Back Pain Spending Up, Relief Down? Or, Did the Media Fall for a False Conclusion?

Me: The name of the paper we are reviewing is "Expenditures and Health Status Among Adults with Back and Neck Problems," authored by Deyo, Mirza, Martin, et al., in the February 13, 2008 issue of JAMA. In the article, the conclusion of the study alleges that patient outcomes have not improved from 1997 to 2005 in spite of an increase in medical expenditures for spine care. Did the authors really prove this conclusion?

Fessler: The conclusions of this manuscript are neither rational nor supported by the data reported. The main thing that the authors had to prove was that there was a statistically significant difference between the expenditure increases between the study groups (spine vs. non-spine problems) over the years of the study. According to their paper, the difference was not statistically significant. (p=.07) For this fact alone, the paper should have been rejected for publication. Yet, they ignore this fact and make broad sweeping and misleading claims about the conclusions.

Me: How could they make such claims? Lead us through the issues.

Fessler: Actually, they had several methodological problems with the collection and the analysis of the data. Several were serious enough to invalidate the entire paper.

Me: How did how they collect the cost data?

Goldstein: First of all, they utilized a database called the Medical Expenditure Panel Survey (MEPS), which involves a questionnaire that is sent to thousands of households around the United States. People are asked to fill out the questionnaire on behalf of their household. They are asked questions about their general health, the type of care received and if they are suffering from any health conditions or if someone in their household has a condition, what health care services were used, how much they spent on that care, etc. Basic demographic data is also collected. They are not asked about their health status or about specific treatments that they had for specific conditions. It's more of a general questionnaire.

Peloza: This is where one of the major problems comes in to play. Specifically from the MEPS Web site, regarding the validation of medical conditions, the answer is 'No, medical conditions are reported by household respondents and are not validated with diagnoses or conditions reported by the provider in the Medical Provider Component. Several published papers have addressed the comparability of household and provider reports of medical conditions.' Therefore, this methodology is entirely nonscientific. In addition, from the MEPS Web site "The MEPS data cannot be used to make estimates of disease, prevalence of health conditions, or mortality/morbidity. The full-year expenditure data can be used to provide estimates of expenditures for persons with events associated with a specified condition. However, these expenditures are not absolutely tied to a single condition.

Me: To summarize, it's a self-reported survey tool designed to gather some general expenditure and utilization data on a number of people in the U.S.

Fessler: That is correct. And it is important to note that this survey tool is not a valid tool to assess a patient's condition, their treatment and certainly not their outcome. The survey and database are just not set up to be used this way. And it has never been subjected to the routine validity testing that is required for assessment tools for scientific publications. Another point to remember is that self-reported data is generally accepted as a weak source of accurate data.

Me: If the MEPS data is so general in nature, how did the authors relate it to specific back, neck and spinal conditions?

Fessler: They used another database that has the ICD-9 codes and they "mapped" the survey responses from the MEPS database into the specific condition codes. However, these databases have two different sets of patient populations, so I'm not sure how the authors thought this could be reasonable or valid.

Berven: As a result of this "mapping," 52.9% of the patients in their study fell into a category labeled "unspecified disorder of the back." However, we cannot tell what the person's problems were. Without knowing which pathologies were being treated, how could any assumptions be made?

Fessler: I agree. There is no way to draw a meaningful conclusion about back and neck problems because the population of patients was so poorly defined.

Me: Do they know how many had back problems and how many had neck problems?

Berven: There is no way to know. There is also no way of knowing if the patient's problem was spine-related or muscle-related.

Me: Let's shift to the expenditure discussion. We know that it's impossible to tell what patient conditions are being analyzed, so for this broad group of people with a back pain complaint, what does the paper reveal about their expenditures?

Goldstein: Right away we can see that the paper is addressing nonoperative costs. They didn't elaborate much on the inpatient costs, and most of the increase was from outpatient expenses, pharmaceutical and "other." We know from the inpatient charge databases that approximately $7 billion each year is spent on spinal surgery charges in the United States. So, if you apply this to their $85.9 billion estimate, it means that 92% of the cost to treat back/neck problems is spent on non-operative care.

Polly: It is interesting that people are shocked by this $85.9 billion number and they quickly blame it on surgical expenses, when it is really being spent on nonoperative treatment. Is it too high? Is it too low? There is no way of knowing. We do know that denying care may lower the overall expenditure, but is unethical.

Ondra: Of course it is less expensive to withhold care from a patient than it is to actually prescribe a course of therapeutic treatments, including surgery. But, what is the cost for that patient who has to give up their livelihood and daily activities?

Me: Are there any conclusions or implications about the cost of surgery?

Peloza: They hardly mention anything in the paper about inpatient or surgery costs. Plus, because it is impossible to map the surveys to specific treatments, and because they chose not to include the data on comorbities (other unrelated illness) it is impossible to tell if these people had inpatient experiences for a spinal reason or for some other reason. They even admitted that when the comorbidity was included as covariate in the analysis, it "weakened the diverging trend between the respondents." This would be important to know.

Berven: Not accounting for the comorbidities distorts the entire study results, not just the people with inpatient experiences.

Fessler: When they accounted for the comorbidities their trend disappeared, so to reach their desired conclusion they eliminated it from their analysis. In my mind, this invalidates their entire project.

Me: How so?

Fessler: The patients with spinal problems were sicker than the patients without spinal problems, which is no big surprise. This fact was statistically significant. However, because they are sicker and have comorbidities, we have to account for treating all of the issues that the patient has, not just the back problem. So, when these patients filled out the questionnaire, they listed all of their conditions. In addition to a back pain issue, they could have also had diabetes, hypertension, allergies, or something else that would require prescription medication, doctor visits, etc.

To prove their conclusion, the authors would have to prove how much of the increased healthcare expenditures were attributable to the patient's back problem, not their diabetes or other problems. This is where they got into trouble. When they did the statistics accounting for the comorbidities they lost their difference, which means that they couldn't show a statistically significant difference in the cost between a person with a spine problem and a person without a spine problem. This would have kept them from stating their ultimate conclusion.

So, they chose to ignore the fact that the person with a spine problem might be spending money on medications and treatments for their other health issues. Instead, they assigned the entire expenditure increase to their spine problems alone. This is bad statistics and bad science. As I said before, this should have invalidated the study and disqualified it for publication.

Me: What about surgical costs? They mention that inpatient costs increased in the time period, but they don't go any further with it.

Berven: In this paper, they aren't even addressing surgery costs and its efficacy. They are focusing on this nonoperative care path. Unfortunately the efficacy of many non-operative treatments is not well documented, and the information is really limited. Perhaps that is another meaningful item that we are learning with this exercise. Just as we are studying, documenting and learning more about the efficacy of surgical therapies, we need to also understand more about the nonoperative side.

Me: The other side of their conclusion states that health outcomes did not improve for the patients with spine problems. How did they measure the patient's outcome?

Peloza: They used the SF-12 survey data, which is a reasonable tool to use in a prospective, randomized, controlled trial. In this situation they took an SF12 database from 2000 to 2005 and they looked at the patients who had back and neck problems or complaints. Keep in mind that these patients are not the same patients who answered the expenditure questionnaire. This is a different set of people.

Me: Did they find a difference in their health status over those 5 years?

Peloza: The values for the physical component score (PCS) were in the range of 45 and the mental component score (MCS) set were in the range of 49.

Me: So, 45 for PCS and 49 for MCS. How close is that to a normal, healthy person.

Berven: Their scores are within the 90% normative range of 50. In other words, these people were close to normal. Regardless of their treatment, they just don't have much room to improve. So, the comment in their conclusion about clinical improvements being marginal is very misleading. The patients were so close normal to begin with, that of course only a marginal improvement was shown.

Fessler: That's right. They didn't have much opportunity to show an improvement.

Me: So, the authors report no statistical significance in the expenditure difference between patients with and without spine problems and the patients are very close to normal health already. Yet, the conclusion of the paper states, "These spine-related expenditures have increased substantially from 1997 to 2005, without evidence of corresponding improvement in self-assessed health status." Help me out here, guys.

Fessler: It's shocking that this was even published. Their data does not support their conclusion.

Goldstein: It's disappointing because there are probably some things that we could have learned here with better information.

Berven: I can't explain their conclusion with their data.

Ondra: This is not a database that a scientist or surgeon can use to make an adequate judgment on patient outcomes. We would use a clinical trial instead to make this type of assessment.

Polly: If the article was addressing cancer patients, people would be appalled by the concept of letting patients suffer and perhaps even die to save money for the healthcare system.

Peloza: It's fraudulent.

Polly: In the spinal surgery community we subscribe to the concept of making decisions based on clinical data and evidence. Before we make a decision with our patients, we want to understand the data about a therapy that will help them, not hurt them. We call this "evidence-based decision-making." However, in this article it appears that the authors are doing the exact opposite, starting with a predetermined or biased decision, then trying to back it up with evidence from a questionable and unscientific source. It defies logic.

Peloza: That would make it "Decision-based evidence-making!"

Me: Since this paper hardly addresses the cost and outcome of spinal surgery, why did so many of the stories in the media target spinal surgery as the problem?

Goldstein: That is a great question, since the paper hardly analyzed any data about back surgery and inpatient costs at all.

Peloza: The only place in the paper they refer to spinal surgery is in their comment section. They list a myriad of items that "may" increase costs. However, they don't address it, they don't quantify it and they don't analyze it.

Me: If they didn't address the cost of spinal surgery in the paper, then why did their media campaign target spinal surgery as a major cause for the increase in health expenditure?

Ondra: I'm not sure why they targeted spinal surgery when their paper had little to do with spinal surgery. Only a small percentage of patients ever require or receive surgery; the effectiveness of surgical treatment is the best documented, successful management of severe spine disease. Perhaps further research is needed regarding the best and most cost-effective care for back disease that requires nonoperative care and consumes 92% of the back pain health dollar.

Me: Thank you, gentlemen. It's important to challenge and question things that don't make sense so that we can collectively help patients who could benefit from modern health care therapies.