Abstract

Objective: Multi-attribute utility measures, such as the Adult Social Care Outcomes Toolkit, are increasingly employed in the long-term care context to evaluate care. Stated preference techniques are often used to elicit preferences for the different quality of life states described by multi-attribute measures. Preferences provide an estimate of the value of each quality of life state and can be used as weights to combine responses in a way that reflects the differential utility of each state. To elicit preferences, newer measures, such as ASCOT, often use best-worst scaling (BWS), a choice-based technique derived from random utility theory. However, BWS has few applications and its acceptability, feasibility and validity – in terms of the extent to which decisions about best and worst states abide by the assumptions of utility theory – are not well understood. The aim of this study is to provide a better understanding of the acceptability, feasibility and validity of the BWS technique.

Data and methods: We combine a BWS exercise with a qualitative approach known as verbal protocol analysis to explore the feasibility, acceptability and validity of the method for eliciting preferences for the ASCOT service user (ASCOT-S) and carer (ASCOT-C) instrument quality of life states. Validity is considered in terms of three axioms of utility theory: completeness, monotonicity and continuity of preferences. An adult sample (n = 20) undertook a BWS experiment involving either the ASCOT-S (8 people) or the ASCOT-C (12 people). The BWS experiment for both ASCOT instruments is based on a fractional-factorial design, consisting of 32 tasks, which were blocked into 4 segments. One task was repeated to test for completeness of preferences, meaning each participant undertook nine tasks. Respondents were asked to ‘think-aloud’ while completing the BWS tasks, but interviewers also used retrospective probing methods to generate a fuller understanding of the decision-making processes used. The transcripts were analysed using thematic analysis. Consistency of the repeated choice task was assessed quantitatively.

Results: Preliminary results indicate participants often used heuristics to aid their decision-making, for example by grouping attributes together or focusing on aspects they considered to be important to them. There was some evidence of non-trading behaviour, and some people may have been constructing their preferences as the task progressed, such as in different contexts. Respondents tended to be fairly inconsistent in their responses to the repeated choice task, being slightly more consistent with their best rather than their worst choices.

Policy implications: The increased level of interest assessing outcomes for long-term care means that measures like ASCOT are becoming more mainstream. The use of the BWS technique is still in its infancy in the field of health and long-term care. We consider how these results help provide insight into the value of BWS for eliciting preferences for multi-attribute utility measures like ASCOT. We also discuss how our study can inform the design of future BWS experiments in the field of long-term care and beyond. This indicator can help policy makers identify valued quality of life states, which can help guide commissioning policy.

Slides