The Community Tracking Study: A Focus on Change in the Health System

Objectives of the Study

Recognizing that health care delivery is fundamentally local in nature, the Center for Studying Health System Change launched the Community Tracking Study to investigate what is happening in health care financing and delivery at the community level. The study focuses on changes in the health care system across 60 sites that are representative of the nation as a whole. Twelve of these communities are being studied in depth, while the remaining 48 are examined less intensively. Together, these sites permit researchers to generalize findings to the national level and to analyze the relationship between health system characteristics and their effects on people. Data collection and analysis for the Community Tracking Study are organized in two-year cycles, with the first cycle establishing a baseline beginning in the spring of 1996.

The Community Tracking Study, funded by The Robert Wood Johnson Foundation, is structured to answer three fundamental questions:

First, how is the organization of the health system changing? This encompasses how the organizational relationships among health plans, physicians, hospitals, and safety net providers are being restructured, how these organizations respond to shifts in market structure, and what role employers, consumers, and public policy play in driving health system changes.

Second, how are outcomes changing? That is, how are outcomes such as insurance coverage, access to care, use of services, health care cost, quality, and satisfaction evolving over time? And how do these changes differ across communities and population subgroups?

Third, how are health system changes related to differences in outcomes? From both qualitative and quantitative perspectives, what is the connection between health system characteristics and insurance coverage, access, service delivery, cost, and quality?

Community Focus

A community-level focus is central to the design of the Community Tracking Study. Health care delivery varies from community to community for many reasons, including local history, culture, and state policy. Collecting and analyzing data at the local level is essential for understanding institutional changes in the delivery system and their effects on people.

Two categories of communities were selected through a process of stratified random selection; taken together, they provide a representative profile of change across the United States. Twelve communities are studied in depth: Boston, Cleveland, Greenville (S.C.), Indianapolis, Lansing (Mich.), Little Rock, Miami, Newark (N.J.), Orange County (Calif.), Phoenix, Seattle, and Syracuse. Intensive case studies are conducted in these communities, and survey sample sizes are large enough to draw conclusions about health system change for each individual site.

An additional 48 communities are studied less intensively. No case studies are conducted in these communities, and survey sample sizes are smaller. Because the smaller samples do not support meaningful site-specific estimates, data from these 48 sites are not analyzed individually, and the sites themselves are not publicly identified.

Site Selection Process

The process of random site selection ensures that the sites are representative of the U.S. population and protects against researcher bias. Potential sites were stratified by region and by community size and type (metropolitan versus nonmetropolitan). The 12 high-intensity sites were selected randomly from among the 60 sites that were metropolitan areas with more than 200,000 people. The 48 low-intensity sites include small metropolitan and rural areas as well.

The information gathered from the 48 less intensively studied communities strengthens the study in several important ways. First, it enables the case studies conducted in the 12 intensive sites to be placed in the broader context of national health system change. This overcomes a common weakness of case study research -- findings that reflect unique features of a small number of communities visited.

Second, it permits statistically reliable conclusions to be drawn about health system change at the national level. Drawing inferences from samples in only 12 communities would carry an unacceptable degree of statistical error; expanding the number to 60 reduces this error to a manageable level.

Third, it permits multivariate analyses that relate system change to changes in individuals' health care experiences. In the conceptual framework guiding the study design, variables related to individuals' experience with the health system -- access, service use, quality, satisfaction -- depend not only on individual factors such as insurance status but also on community-level characteristics. For instance, the nature of access available to an uninsured person may depend on the financial well-being of both safety net and mainstream providers. Increasing the variation in community-level variables from 12 to 60 distinct levels makes such multivariate analyses feasible.

Data Sources

Data for the Community Tracking Study are drawn from a variety of sources. Site visits gather qualitative information from major components of the health care system, including purchasers, state governments, health plans and insurers, hospitals, physician organizations, community health centers, public health departments, and consumer organizations. These site visits provide a rich understanding of health system change and aid in interpreting quantitative data. The Lewin Group, working with Center staff, conducts the site visits.

Much of the information on the effects of health system change on people comes from surveys of households and physicians that the Center conceptualized and directs. The household survey collects data on access to care, how individuals enter the health system, use of care, financial burdens, satisfaction with care, selected quality indicators, and attitudes that predict demand for insurance. Respondents are asked for the name of their insurance plan, and health insurers are in turn asked to provide characteristics of that plan. Mathematica Policy Research, Inc. conducts the household survey.

The physician survey offers an additional perspective on changes in patient care. Practice styles are measured through clinical vignettes that assess physicians' decision making concerning discretionary procedures and referrals. Physicians' perceptions of their ability to meet patients' needs and expectations -- including technical quality and clinical autonomy -- are also measured. The Gallup Organization conducts the physician survey.

Analysis of hospital discharge records (in communities located in states that maintain such data systems) allows the study of care delivery and quality-of-care indicators. Hospital discharge records may also serve as indicators of access, such as admission rates for conditions likely to result from inadequate ambulatory care.

Finally, surveys of various health system organizations are conducted to complement the data from site visits. These surveys permit greater depth and objectivity on selected issues and broader coverage through larger respondent samples. Organization surveys and secondary data provide information about health system organization in the 48 less intensively studied communities. Surveys of hospitals, physician groups, and health plans and insurers are developed and fielded. An employer survey conducted by RAND and the Research Triangle Institute collects information on health insurance premiums, types of health plans offered, and employer strategies for purchasing health care coverage.

Tracking Changes in Outcomes

One way to organize the information collected from these data sources is by type of outcome. Center researchers track insurance coverage, access, service delivery, cost, and quality using data from the household, physician, and employer surveys.

Insurance Coverage. Household survey respondents are asked whether they have insurance and, if so, about specific attributes of their coverage -- for example, whether they must use doctors from a defined list or can see other providers without additional charges. Supplemental data on insurance -- whether employers offer coverage and what types of products they make available -- come from the employer survey. From this information, the Center can estimate and track over time the percentage of the population that is insured versus uninsured, as well as key characteristics of insurance plans for those with coverage.

Access. The household survey yields insights into traditional indicators of access, including the ease and convenience of obtaining health care services. From the physician survey, researchers learn how readily physicians can refer patients to specialists, whether they accept Medicaid patients, and how much charity care they provide. Hospital discharge records also help track access; a high rate of hospital admission for asthma, for instance, can serve as an indicator of poor access to primary care.

Service Delivery. Among all the changes taking place in the health system, changes in the delivery of care are those most likely to directly affect people. The household survey tracks how many times respondents see a doctor and what types of health services they receive. The physician survey provides indicators of how medicine is practiced, including information about scope of care and physicians' perceptions of managed care's effects. Hospital discharge records contribute to the picture of service delivery by providing information on rates for surgical procedures that vary widely in use by geography.

Cost. Changes in cost are studied primarily from the perspective of payers. Data on premiums for private insurance and employees' share of premiums come from the employer survey, while information about out-of-pocket expenses comes from the household survey. Premiums are tracked by type of insurance, which allows researchers to study the extent to which changes in overall premiums are driven by shifts among insurance types. Per capita Medicaid and Medicare expenditures are drawn from government data.

Quality. The household survey addresses the personal dimension of quality: how satisfied people are with their care, what preventive services they receive, and how they rate their health status. The physician survey captures physicians' perceptions of their ability to deliver quality care. Additional quality measures come from hospital discharge records, including rates of fetal deaths, newborns with serious health conditions, and postsurgical complications.

Timetable for Work

To track changes in health systems and outcomes, the 60 communities are followed over time. The initial site visits, conducted during the second half of 1996, are followed up with telephone calls to key respondents and ongoing monitoring of local media and state policy. The second round of site visits was planned for 1998, with subsequent rounds at two-year intervals. Surveys also follow a two-year cycle, with fielding beginning in 1996. Organization surveys are conducted in 1997.

For the household and physician surveys, half of the 1998 sample consists of respondents to the 1996 survey, and half are new respondents. This mixed design represents a compromise between a pure longitudinal panel and an independent cross-section each cycle. The longitudinal cohort allows analysis of change for individuals and institutions at two different time points. The new cross-section ensures that the sample remains representative of each community's current population.

Center staff prepare papers and reports with analyses sequenced according to data availability. The first reports -- site-specific descriptions of health system organization and patterns of change for the intensively studied communities -- were published in the following year. These were followed by cross-site comparisons on particular issues such as the integration of health care delivery. A national baseline description, scheduled for fall of 1997, reported data from the household, physician, and employer surveys on topics including insurance coverage, access to care, service use, and satisfaction.

Data from the organization surveys and site visits were used to describe local market characteristics -- such as the extent of market consolidation and integration -- by early 1998. These data were then analyzed alongside data from the household and physician surveys to examine individuals' experiences with the health system in markets that differ according to characteristics such as the extent of capitation and consolidation.

Sources and Further Reading

Robert Wood Johnson Foundation -- The foundation that funded the Community Tracking Study.

Health Affairs -- Peer-reviewed health policy research and analysis.

Kaiser Family Foundation -- Health policy research, polling, and data on key health issues.

Commonwealth Fund -- Research on health care system performance.