Most Important Reason By Judith T. Lynch et al.

By Judith T. Lynch, Anne E. Kenyon, Jean Wang, Stephanie Rizk, Scott Scheffler, Katherine R. Jackson, and Mildred C. Duke

With all the changes that are occurring as a result of the Medicare Modernization Act, it is more important than ever to have a window into the quality of care that Medicare beneficiaries receive—from their perspective—and how these changes influence their choice of Medicare health plan options.

The Medicare CAHPS® Disenrollment Reasons Survey investigates one aspect of this question by collecting data from beneficiaries who have recently left a managed care health organization about the reasons they disenrolled from their Medicare managed care health plan.

The Reasons Survey was first implemented in 2000. Each calendar quarter, the survey’s project team selects a sample of approximately 15,000 adults who voluntarily disenrolled from a Medicare Advantage (MA) health plan the preceding quarter. They are questioned using a mail survey, with telephone follow-up of nonrespondents.

In the mail survey, beneficiaries are first asked a series of thirty-four closed-ended items about reasons they disenrolled, and are then asked to report their one “most important reason” in an item that uses an open-ended response format. They are also asked to mention any other reasons not already marked or indicated in the preprinted items. The “most important reason” question is one of the most critical analytic items in the survey.

There are advantages to using an open-ended response format that are particularly applicable to the Reasons Survey, but there are disadvantages that are especially problematic as well. As Don Dillman pointed out in 1978, open-ended response formats require that respondents recall a past experience, reorganize their thoughts about that experience, and then find the words or terms with which to describe it. This response task, according to Dillman, is difficult for most respondents, especially those “with low educational attainment or who lack experience in communicating ideas to other people.”

Dillman’s observations are especially pertinent to the Reasons Survey, in which the majority of the target population is elderly (ages sixty-five years or over). While the sample is representative of beneficiaries who disenroll from an MA health plan, the primary response task is to think about and report actions or events that might have occurred weeks or months prior to the actual decision to leave the plan and which later influenced and led to the decision. The response task is made even more difficult by the fact that survey items are about Medicare, Medicare health plan (insurance) coverage, and reasons for leaving an MA health plan, which are generally considered to be very complex topics.

Although the quality of the responses to the “most important reason” (MIR) and “other reasons for leaving” items have in the past been generally good and adequate for analysis, we consistently found in each survey year that we could not use data from a fair number of the respondent cases, either because no reason for leaving the plan was reported, or because the responses to the MIR and/or Other Reasons items were unrelated to reasons for leaving the plan. Several recurring problems adversely affected the usability of some of the data reported in these open-ended items:

Based upon these observations, the project team conducted a qualitative assessment to understand better the ease or difficulty that beneficiaries have in citing a most important reason. Using a combined mail survey and telephone follow-up approach, project staff interviewed a sample of respondents to the 2003 survey about their thought processes and ability to provide a MIR. In our interviews with individuals who did not report a most important reason, we realized that, in some cases, respondents were confused by the purpose of the two items. Most indicated that they unintentionally left the MIR blank while writing text that described their most important reason in the Other Reasons item.

Further, we learned that, for some respondents, providing just “one” most important reason was a difficult task, as the “reason” that they left involved a number of factors which, in their minds, reflected a single reason. For example, while the statement, “I left because the premiums and copayments were too high” might reflect an overall reason to a respondent of, “It cost too much,” CMS and MA administrators are interested in knowing which of the two—premiums or copayments—was most important to the respondent.

Most Important Reason: Helping Older Respondents Provide Better Data