Consumer Responses to Online Decision Aids
Consumer Responses to Online Decision Aids
Healthwise DPs cover 165 topics. Each DP goes through the following product development cycle. The need for a new DP is determined by an expert medical team composed of two to four licensed physicians and content specialists. Factors contributing to the designation of a DP topic include the prevalence of the decision, degree to which the issue is preference-sensitive because of the equivocal nature of the evidence for treatment, the range of negative and positive outcomes related to a specific condition, and the amount of control the patient has in determining the quality of the outcome for his or her health for that specific condition.
Once a health issue has been chosen for a DP, a medical writer is assigned to develop a work plan, which takes 2 to 4 weeks to complete. The writer has expertise in writing in plain language at the sixth-grade level. All writers are guided by a standard protocol for design and placement of content, construction of the interactive elements of a DP, and creation of knowledge questions used to assess the user's understanding of the key issues for the health topic covered by the DP. Content is focused on three or four key ideas distilled from the pertinent medical literature and organized around a balance of what a physician would want his or her patient to understand and what matters most to the patient. Videos, pictures, or other visual illustrations are directly embedded and therefore immediately accessible to the user.
A draft work plan is shared with the expert medical team and reviewed for content and structure. The draft DP goes through several rounds of review with editors, the medical team, and an outside specialist.
Each DP has six elements. Element 1 (Get the Facts) contains a distillation of pertinent medical information with an expandable Frequently Asked Questions pull-down menu for users interested in additional information. Elements 2 (Compare Options), 3 (Your Feelings), and 4 (Your Decision) address the key dimensions of shared decision making. The "Your Feelings" element gives the user an interactive slider tool to rate the importance of three or four reasons a patient or consumer might pursue a certain course of action for the health condition in question versus not pursuing that course. These preference questions are arrived at through review of the pertinent literature on adherence and side effects, clinical experience, and pilot testing with users. Element 4 (Your Decision) asks the user to assess the direction in which he or she is leaning by using an interactive slider tool. Element 5 (Quiz Yourself) queries the user on three questions key to the decision being made and level of confidence in the decision on a 5-point scale. Each of these "knowledge questions" receives a correct (coded 2) versus incorrect rating (coded 1) or an "I am not sure" (coded 0). These knowledge questions were asked before and after participants used the decision aid in the study reported here. The sixth and final element of the DP is a printable, one-page summary of the user's responses to questions about preferences.
All Healthwise DPs meet the International Patient Decision Aids Standards. The format of the DPs has gone through iterative usability and functionality testing; all DPs are based on the Healthwise Knowledgebase, which is updated quarterly by impartial experts and are URAC-accredited (Utilization Review Accreditation Commission).
Participants were recruited through Zoomerang, a commercial, web-based survey service (now combined with Survey Monkey). Study participants were recruited for a 2-week block, beginning in mid-August 2009 for the statins and aspirin decision aids and beginning in late September 2009 for the MRI decision aid. After being designated as eligible for participation, participants were introduced to an online session with a statement about the importance of honest and anonymous feedback and that the average length of time was 20 minutes to complete the survey.
All participants agreed to a standard statement about personal privacy and use of their responses for research purposes (Zoomerang/Survey Monkey privacy policy, available at http://www.surveymonkey.com/mp/policy/privacy-policy/). Consent was gained a second time with users responding to the question "Do you agree to the uses of your survey responses as noted above?"
As part of the sign-up protocol, potential participants were screened by Zoomerang for background characteristics (e.g. minimum age, to assure they were not employed as a health professional and health status via self-report of diagnosis). The initial pool of users was then sent to a Healthwise website where another validation of health and and employment status was completed. Completed survey responses were checked for time spent on the website and for patterns of responses. User responses lasting less than 3 minutes and those showing a pattern of response bias were eliminated from the data file. For the current study, 3% of users were eliminated prior to analysis.
Questions about knowledge were analyzed with a mixed design with one "within-subjects" factor (pre-post) and one "between-group factor" (health condition). The IBM SPSS Statistics 21 general linear model with repeated measures was used for the analysis. The multi-variate results for Pillai's trace (V) and related statistics are reported. Since there were only two repeated data points per participant, the question of sphericity did not apply. A first step in the analysis was to test the contribution of four socio-demographic factors to pre-post changes in the knowledge questions. Gender and educational level were statistically significant (P = .01) for the second knowledge question.
Descriptors of the DPs were analyzed as dependent measures with a one-way analysis of variance with health condition as a three-level factor. Post hoc comparisons for significant "between-subjects" main effects were analyzed with the Bonferroni test. As with the knowledge questions, the four socio-demographic factors were first analyzed as potential predictors of differences on the descriptive variables. Of these factors, race/ethnicity was significant (P ≤ .05) for five of the eight descriptors. The last set of questions on user assessment of the DP was analyzed with Pearson's chi-square test for the association between health condition and these questions.
Methods
Development of Decision Aids
Healthwise DPs cover 165 topics. Each DP goes through the following product development cycle. The need for a new DP is determined by an expert medical team composed of two to four licensed physicians and content specialists. Factors contributing to the designation of a DP topic include the prevalence of the decision, degree to which the issue is preference-sensitive because of the equivocal nature of the evidence for treatment, the range of negative and positive outcomes related to a specific condition, and the amount of control the patient has in determining the quality of the outcome for his or her health for that specific condition.
Once a health issue has been chosen for a DP, a medical writer is assigned to develop a work plan, which takes 2 to 4 weeks to complete. The writer has expertise in writing in plain language at the sixth-grade level. All writers are guided by a standard protocol for design and placement of content, construction of the interactive elements of a DP, and creation of knowledge questions used to assess the user's understanding of the key issues for the health topic covered by the DP. Content is focused on three or four key ideas distilled from the pertinent medical literature and organized around a balance of what a physician would want his or her patient to understand and what matters most to the patient. Videos, pictures, or other visual illustrations are directly embedded and therefore immediately accessible to the user.
A draft work plan is shared with the expert medical team and reviewed for content and structure. The draft DP goes through several rounds of review with editors, the medical team, and an outside specialist.
Each DP has six elements. Element 1 (Get the Facts) contains a distillation of pertinent medical information with an expandable Frequently Asked Questions pull-down menu for users interested in additional information. Elements 2 (Compare Options), 3 (Your Feelings), and 4 (Your Decision) address the key dimensions of shared decision making. The "Your Feelings" element gives the user an interactive slider tool to rate the importance of three or four reasons a patient or consumer might pursue a certain course of action for the health condition in question versus not pursuing that course. These preference questions are arrived at through review of the pertinent literature on adherence and side effects, clinical experience, and pilot testing with users. Element 4 (Your Decision) asks the user to assess the direction in which he or she is leaning by using an interactive slider tool. Element 5 (Quiz Yourself) queries the user on three questions key to the decision being made and level of confidence in the decision on a 5-point scale. Each of these "knowledge questions" receives a correct (coded 2) versus incorrect rating (coded 1) or an "I am not sure" (coded 0). These knowledge questions were asked before and after participants used the decision aid in the study reported here. The sixth and final element of the DP is a printable, one-page summary of the user's responses to questions about preferences.
All Healthwise DPs meet the International Patient Decision Aids Standards. The format of the DPs has gone through iterative usability and functionality testing; all DPs are based on the Healthwise Knowledgebase, which is updated quarterly by impartial experts and are URAC-accredited (Utilization Review Accreditation Commission).
Participant Recruitment
Participants were recruited through Zoomerang, a commercial, web-based survey service (now combined with Survey Monkey). Study participants were recruited for a 2-week block, beginning in mid-August 2009 for the statins and aspirin decision aids and beginning in late September 2009 for the MRI decision aid. After being designated as eligible for participation, participants were introduced to an online session with a statement about the importance of honest and anonymous feedback and that the average length of time was 20 minutes to complete the survey.
All participants agreed to a standard statement about personal privacy and use of their responses for research purposes (Zoomerang/Survey Monkey privacy policy, available at http://www.surveymonkey.com/mp/policy/privacy-policy/). Consent was gained a second time with users responding to the question "Do you agree to the uses of your survey responses as noted above?"
As part of the sign-up protocol, potential participants were screened by Zoomerang for background characteristics (e.g. minimum age, to assure they were not employed as a health professional and health status via self-report of diagnosis). The initial pool of users was then sent to a Healthwise website where another validation of health and and employment status was completed. Completed survey responses were checked for time spent on the website and for patterns of responses. User responses lasting less than 3 minutes and those showing a pattern of response bias were eliminated from the data file. For the current study, 3% of users were eliminated prior to analysis.
Data Analysis
Questions about knowledge were analyzed with a mixed design with one "within-subjects" factor (pre-post) and one "between-group factor" (health condition). The IBM SPSS Statistics 21 general linear model with repeated measures was used for the analysis. The multi-variate results for Pillai's trace (V) and related statistics are reported. Since there were only two repeated data points per participant, the question of sphericity did not apply. A first step in the analysis was to test the contribution of four socio-demographic factors to pre-post changes in the knowledge questions. Gender and educational level were statistically significant (P = .01) for the second knowledge question.
Descriptors of the DPs were analyzed as dependent measures with a one-way analysis of variance with health condition as a three-level factor. Post hoc comparisons for significant "between-subjects" main effects were analyzed with the Bonferroni test. As with the knowledge questions, the four socio-demographic factors were first analyzed as potential predictors of differences on the descriptive variables. Of these factors, race/ethnicity was significant (P ≤ .05) for five of the eight descriptors. The last set of questions on user assessment of the DP was analyzed with Pearson's chi-square test for the association between health condition and these questions.
Source...