D ecide F or Y ourself . PASSIVE REPORTING ON PASSIVE SM OKE by Jacob Sullum L ast F ebruary the administrator of the Environmental Protection Agency told a House subcommittee that Congress should ban smok­ ing in places of business. Testifying in favor of the Smoke-Free Environment Act. which would forbid smoking in buildings open to the public, Carol Browner relied heavily on the EPA report that declared environmental tobac­ co smoke (ETS) to be "a known human lung carcinogen." Since it was released in January 1993, this 510-page document has become a favorite prop of the anti-smoking movement. It has helped justify smoking bans in government agencies— including the Department ot Defense—in cities such as Los Angeles and San Francisco, and in states such as Maryland and Washington. Because the EPA s prelimi­ nary conclusions about ETS were first publi­ cized in 1990, the report had an impact even before it appeared in its final form. "Hundreds of local ordinances have been passed or intro­ duced in virtually every area of the country since 1991," Browner testified. "In the year since publication of the EPA report...we have seen a rapid acceleration ot measures to protect non-smokers in a variety ot settings And in March, the U S. Occupational Safety and Health Administration (OSHA) proposed a ban on smoking in indoor workplaces, including bars and restaurants. In light of the legislation and policy changes it has generated, the EPA s Respiratory Health Effects o f Passive Smoking: Lung Cancer and Other Disorders may be the most influential report ever issued by the agency. As one might expect, it has received extensive coverage from major newspapers. Between May 1990 and February 1994, the New York Times. Los Angeles Times. The Wall Street Journal and The Washington Post ran more than 100 news stories about ETS. ot which about 45 focused on the EPA report. Yet almost without excep­ tion, the coverage has been one-sided, credu­ lous and superficial. Even before the EPA released its report, journalists were quick to accept the claim that secondhand smoke kills. And despite serious questions about the report’s assertion that ETS causes lung cancer and the process by which the EPA reached that conclusion, leading U.S. newspapers have treated this assertion as scientific fact. In so doing, not only have they exaggerated what is known about the effects ot ETS, but they have missed an important story about the corruption of science by the political crusade against smoking. To uncover the facts would not have required a lot of digging. They were repeatedly outlined by representatives of the tobacco industry tor anyone who would listen. Indeed, that was a big part of the problem. "The tobacco industry has established a reputation for disseminating misinformation," says Michael Fumento. one of the few journalists who took a critical look at the science behind the EPA s report. "At the very least. |the industry | has been known to put a twist on material that isn’t warranted. In a MEDIACRITIC I r. J Faced with evidence that was weak, incon­ That impression is supported by the fact that sistent and ambiguous, the EPA finessed some the EPA put together a "policy guide" for important points and gave the data a vigorous reducing workplace exposure to ETS well massage to arrive at the conclusion that ETS before it had officially decided that ETS was a causes lung cancer. To begin with, the EPA hazard. The first draft of the guide was used an unconventional definition of statisti­ released in June 1990, three-and-a-half years before the EPA released the final version of cal significance. In previous risk assessments the EPA had always used the traditional stan­ its risk assessment. William Reilly, then administrator of the EPA. told The Wall Street dard. But in the case of ETS, the agency abandoned the usual definition of statistical Journal in January 1993 that he delayed release of the policy guide in its final torm significance and called a result significant it the probability that it occurred by chance was because he didn't want it to "look like we're trying to torque the science." 10 percent or less—a change r r o c- j Reilly had reason to be con- that in effect doubles the odds 1 DC E P A [ m e s s e d cerned about that perception. of being wrong. In March 1992, an expert Even according to the panel that he convened had b ro a d e r d e fin itio n , only is s u e d a r e p o r t c a lle d one of the 11 U.S. studies that Safeguarding the Future: the report analyzes found a Credible Science. Credible statistically significant link between ETS and lung cancer c a u s e s l u n o C O IlC e r. D ecisions. Among other things, the panel concluded And according to the usual that "EPA science is of definition, none of them did. uneven quality, and the agency's policies and In order to bolster the evidence, the EPA departed from its usual risk-assessment proce­ regulations are frequently perceived as lack­ dure by combining the results from these 11 ing a strong scientific foundation ' It cau­ studies in a "meta-analysis." This technique is tioned that "science should never be adjusted appropriate only when the underlying studies to fit policy, either consciously or uncon­ sciously." are comparable in method and structure. Enstrom says using meta-analysis for stud­ espite these and other warning signs, ies such as those examined by the EPA “is not the coverage by the major newspapers a particularly meaningful exercise," since the was generally unskeptical of the studies are apt to differ in the way they define agency's conclusions and dismissive ot the sm okers, the types of lung cancer they tobacco industry's criticism. The typical story include, the confounding variables they take into account and so on. "It's just fraught with opened with the government's claims, elabo­ dangers." In any event, the result of the EPA's rated on them for several paragraphs, quoted anti-smoking activists who agreed with the meta-analysis is significant only under the EPA and described the tobacco industry s weak definition adopted especially for these data. By the conventional standard, the meta­ response in a paragraph or two. The tobacco industry’s comments usually amounted to lit­ analysis does not support the claim that ETS causes lung cancer Furthermore, had the EPA tle more than denial, and no independent sources were provided to back them up. News included in its meta-analysis a large U.S. study published in 1992, the result might not consumers were left with the impression that, aside from industry representatives, no one have been significant even by the revised had doubts about the EPA's position on the standard. health effects of ETS. The contrivances employed by the EPA, But as Michael Fumento showed in his which a July 31, 1992 Science article described as “fancy statistical footwork," indi­ January 28, 1993 story for Investor's Business Daily, this was clearly not true. "Some scien­ cate that the agency was determined to reach tists and policy analysts who say they couldn't the conclusion that ETS kills non-smokers. some key points to conclude that passive smoke D MEDIACRITIC sense, it was the boy who didn't cry wolf—the guy who year after year saw a wolf and claimed there was no wolf there. When he says, 'Look, there's no wolf there,- the media are not going to be quick to believe that." In fact, most reporters were so disinclined to believe the tobacco industry that they simply assumed there was a wolf, without attempting to verify its existence. On January 6, 1993, Los Angeles Times writer Rudy Abramson report­ ed: “The most bitter resistance to the EPA's move to link secondary smoke and lung cancer has been waged by Philip Morris Co., a lead­ ing cigarette manufacturer, and by the Tobacco Institute, the industry’s chief lobbying organi­ zation. Some 30 years after the landmark sur­ geon general's report on smoking and health, the industry continues to argue that there is no scientific proof of a link between cancer and smoking.” The message of this juxtaposition is clear: Since the tobacco industry has refused to acknowledge that smoking causes lung cancer, people should not give credence to their claims about ETS and lung cancer. This argument, which showed up repeatedly in coverage of the EPA report, seeks to simultaneously discredit criticism of the agency’s position and bolster the case against ETS. It implies not only that the tobacco industry is lying, but that the evi­ dence of a link between ETS and lung cancer is just as strong as the evidence of a link between smoking and lung cancer. This analo­ gy is very misleading. James Enstrom, a professor of epidemiology at UCLA, notes that thousands of studies have examined the link between smoking and lung cancer. Virtually all of them have found posi­ tive associations, statistically significant in the vast majority of cases. This is an important point. In any study that tries to measure the association between a sus­ pected risk factor and disease rates, there is always the possibility that an observed differ­ ence between the exposed group and the con­ trol group occurred simply by chance and had nothing to do with the risk factor. Researchers do statistical tests to account for this possibili­ ty. By convention, epidemiologists call a result significant if the possibility that it occurred by chance is five percent or less. The associations between smoking and lung cancer are sizable as well as statistically significant: Recent stud­ ies indicate that the average male smoker is 20 times more likely to develop lung cancer than a male non-smoker, while the risk ratio tor women is about 10 to one. The figures are even higher for heavy smokers. These errors in stories about the EPA report reflect a general tendency in cover­ age of the ETS controversy to exaggerate evidence and m inim ize criticism . An example is a May 29, 1990 New York Times story by Lawrence K. Altman. U nder the h ead lin e, “The E vidence Mounts on Passive Smoking,” Altman described a growing scientific consensus that ETS is a health hazard. He quoted one scientist who said "the links between passive smoking and health problems are now as solid as any finding in epidemiol­ ogy,’’ and another who claimed “there’s no question” that ETS causes heart dis­ ease. Both assertions are controversial, to say the least, but Altman did not offer specific rebuttals from anyone. In the 44- paragraph article, he devoted only three paragraphs to skeptics, both identified with the tobacco industry. And Altman himself exaggerated what the evidence tells us. In the second para­ graph, he asserted that “the studies show” ETS “causes death not only by lung can­ cer, but even more by heart attack." Thus, he declared at the outset of the story that the case was closed on ETS. "The EPA reviewed 24 epidemiological studies of passive sm oking and lung cancer, 11 m ore than in the Surgeon G en eral's Report in 1986,” he wrote, describing an early version of the risk assessment. "The newer studies confirm [the results] in the first 13 studies." The reader is not likely to guess from this summary that the vast majority of these studies failed to find a significant link between ETS and lung cancer. Altman is not alone in failing to discuss statistical significance. Consider Jane E. Brody’s January 8, 1992 New York Times story about a study directed by Elizabeth Fontham of Louisiana State University Medical Center. The headline read: “New Study Strongly Links Passive Smoking and C an ce r.” B rody rep o rted : “The study, the largest of its kind, found a 30 percent higher risk of lung cancer if the women's husbands smoked, a risk that y contrast, the EPA report was based on 30 epidemiological studies that looked for a link between ETS and lung cancer, mainly by comparing disease rates among non-smoking women living with smokers to disease rates among women living with non- smokers. Most of the studies found positive associations, but they were statistically signifi­ cant in only six studies. (Nine found that living with a smoker was associated with a reduced risk of lung cancer, but these results were not statistically significant.) And all ot the positive associations were weak by epidemiological standards, typically yielding risk ratios of less than three to one. The EPA estimated that a woman who lives with a smoker is 1.19 times as likely to develop lung cancer as a woman who lives with a non-smoker. "Comparing that to a 10 to one ratio, you can see it's minute." Enstrom says. “It’s at least one order of magni­ tude different from the active smoking data. With risk ratios this small, it's difficult to rule out confounding variables, such as diet and other sources of pollution, that might account for an observed association. "At least 20 confounding factors have been identified as important to the development of lung cancer,” wrote Gary L. Huber, a professor of medicine at the University of Texas Health Science Center, and two colleagues in the July 1991 issue of Consumers' Research. “No reported study comes anywhere close to controlling, or even mentioning, half of these." Enstrom is not optimistic that future research will clarify the issue. "You're talking about ratios that are so close to 1.0 that it’s really beyond the realm of epidemiology," he says. "You could do more studies, and you could probably arrive at more precise ratios, but as to whether those ratios would mean any thing, I doubt it....You're basically down in a noise- level situation, and whether you can really see a true signal above the noise is doubtful' B rose with the number of cigarettes and years of exposure.” Brody failed to note that this overall association was not sta­ tistically significant (that is, the probabili­ ty that the result occurred purely by chance was greater than five percent). Although Fontham et al reported statisti­ cally sig n ific a n t asso c ia tio n s for a few subgroups, the risk ratios were all under 2.5, so it is wrong to say that the study “strongly links passive smoking and cancer.” nother common error involves con­ fusing correlation with causation. In 1991, for exam ple, the U.S. Centers for Disease Control (CDC) did a survey that, among other things, asked parents to assess their children’s health. The CDC reported that 4.1 percent of the children who lived in households with sm okers were said to be in “ fa ir” or “poor” health, compared to 2.4 percent of the children who lived in households without smokers. From this information it is im p o ssib le to co n clu d e anything about the effects of ETS, since the study did not control for variables that might account for the difference in reported health. Poverty is the most obvious exam­ ple. Research shows that people with lower incomes are both more likely to smoke and more likely to be in poor health. Yet on June 19, 1991, the New York Times, The Wall S treet Journal, Los Angeles Times and The Washington Post all ran stories under headlines asserting that the study had found that smoking in the home harms children. Only the New York Times and The Washington Post noted that the study did not control for incom e, and only the Post m ade the importance of this fact clear. The errors that appear in these and other stories about ETS are not random, of course. They consistently weigh in favor of the view that ETS is a serious health hazard. Reporters are receptive to that view for a number of reasons. Even if they d o n ’t p erso n ally d isap p ro v e of A MEDIACRITIC When reporters choose sides on the basis smoking, they are well aware of its dan­ gers. If a lot o f tobacco sm oke hurts of trust, they fail to make independent smokers, it seems plausible that a little assessments of the arguments of both sides. would hurt non-smokers, though not as So readers of stories about ETS might wish to keep in mind the following points: much. Since most jo u rnalists do not have □ The Im portance of S tatistical backgrounds in statistics or epidemiolo­ Significance. When researchers do not come gy, they rely on other people to assess the up with statistically significant results, (hey issue. The most conspicuous sources for tend to underplay this fact, for obvious rea­ stories about ETS work for the tobacco sons. Stories should be examined to see in d u stry , the g o v ern m e n t and a n ti­ whether they disclose, as a good report smoking groups. Reporters don’t trust the should, whether a result is statistically sig­ nificant. Epidem iological tobacco com panies. But studies include “confidence in contrast to the skepti­ When someone intervals” that indicate there cism th ey b rin g to the pronouncem ents of other cites a “pattern ” is a 95 percent probability that the true risk ratio lies government agencies and two numbers. If the s p e c ia l-in te re st gro u p s, or “trend in the between lower number is 1.0 or less, they do tend to trust pub­ lic health authorities such data’ , ’ it’s time to the result is not significant, even if the authors of the as th e EPA and a n ti­ sm oking o rg a n iz a tio n s look more closely. paper play it up in the abstract. su ch as th e A m e ric a n When researchers don’t get significant Cancer Society. The governing assump­ tion seems to be that the tobacco compa­ results overall, they sometimes slice up the nies are trying to maintain profits, while data into subgroups, seeing if they can find a the government and anti-smoking groups significant association at certain levels of are interested in promoting public health exposure, for certain kinds of cancer and so on. But the more such comparisons they do, and getting out the facts. the less likely it is that any association they ut sometimes these two missions find will be meaningful, since there is a five- conflict. Public health officials may percent chance of being wrong each time. be inclined to shade the truth a bit if Furthermore, the subgroup data for ETS and lung cancer are often contradictory: One it helps to discourage smoking by making it less acceptable. In her testimony last study will find a significant result for adeno­ February, EPA Adm inistrator Browner carcinoma lung cancer but not for other said the main benefit of the Smoke-Free types of cancer, or for spousal smoking but Environment Act would be its impact on not for childhood exposure, while another smokers. “The reduction in smoker mor­ study will find the opposite. tality due to smokers who quit, cut back or □ The P itfalls of C o rrelatio n versus do not start is estimated to range from C ausation. Even a statistically significant about 33,000 to 99,000 lives per year,” association between A and B does not prove she said. And six former surgeons general, that A causes B. A and B could both be the New York Times reported, “echoed the associated with another factor or set of fac­ theme that this simple measure could do tors. An article in the July 28, 1993 Journal more for the public health than any other o f the American Medical Association report­ bill in years.” So, just as the tobacco com­ ed that, allowing for differences in smoking panies have an interest in minimizing the rates, restaurant workers are 50 percent more dangers of ETS, the government and the likely to get lung cancer than people in other anti-smoking groups have an interest in occupations. The study controlled for smok­ ing but not for a wide range of other factors maximizing them. B MEDIACRITIC MEDIACRITIC care less about tobacco company profits or even the rights of smokers are worrying aloud that the EPA report is paving the way for jus­ tifying new health-based government regula­ tions and programs without any real science behind them," he wrote. The story quoted a series of credible sources, including epidemi­ ologists and statisticians, who questioned the quality of the evidence linking ETS to lung cancer and took the EPA to task for manipu­ lating the data to make its case. Fumento cited a 1992 article from Toxicologic Pathology in which Alvan Feinstein, an epidem iologist at Yale University, reported a com­ ment by a leading public- health researcher: "Yes, it's rotten science, but it’s in a worthy cause, It will help us get rid o f c ig a r e tte s and become a smoke-free society." It's difficult to understand why virtually no one followed Fumento's lead, especially since similar questions about the report were raised that summer in con­ gressional hearings and in a tobacco industry law suit challenging the EPA's findings. During the year after Fum ento’s piece appeared, only one story in a major newspa­ per dealt with the issues he raised in a less than perfunctory way. In a July 28, 1993 arti­ cle about the tobacco industry’s lawsuit. Wall Street Journal reporter Jerry E. Bishop made it clear that questions about statistical signifi­ cance and confounding variables are legiti­ mate and not easily dismissed. Although he did not quote any critics of the report who were not affiliated with the tobacco industry, he at least showed that statisticians disagree about the quality of the EPA's work. By contrast, a June 23, 1993 story by Journal reporter F.ben Shapiro unfairly and erroneously attacked one of the industry’s major claims, that the EPA excluded from its meta-analysis a large U.S. study, published in the November 1992 issue of the American Journal o f Public Health, that would have changed the report's conclusions. Shapiro wrote that the study, which was included in a tobacco industry press package about the law- that could affect lung cancer rates. Yet cov­ erage in The Washington Post, the New York Times and Los Angeles Times supported the author’s conclusion that the higher incidence of lung cancer should be blamed on higher levels of tobacco smoke in restaurants. The concern about confounding variables is especially important when risk ratios are small. Epidemiologists generally consider an association "w eak" when the ratios are between 1.0 and 3.0. In the restaurant study, the risk ratio emphasized by the author was about 1.5. “Anything with a risk ratio of less than 3.0. I don't trust," Fumento says. “It’s like measuring the width of a hair with a standard 12-inch ruler. You can’t do it. The little markings are tot) big. So it is with epi­ demiology. It’s a blunt tool. □ Weasel Words. Readers should be alert to q u a lifie rs and hedging; so should reporters. In the restaurant study, tor exam­ ple, the author wrote: “The epidemiologic evidence suggested that there may be a 50 percent increase in lung cancer risk among food-service workers that is in part attribut­ able to tobacco smoke exposure in the work­ place." (Emphasis added.) The 1991 report of the CDC survey of children’s health said the results “show an apparent pattern sug­ gesting that, fo r most children, fair or poor health appears to be associated with various exposures to cigarette smoke. (Emphasis added.) When someone cites a “ pattern" or a "trend in the data." it’s time to look more closely. In rigorous science, close doesn't count. J Discrepancies. When two versions of a verifiable fact diverge sharply, readers should reserve judgment. For instance, an Associated Press story that appeared in the New York Times on June II, 1992, quoted a physician who appeared at an American Heart Association (AHA) press conference as saying that "thousands of studies have shown that secondary smoke increases the risk of heart and lung disease." The Tobacco Institute, on the other hand, "insisted that fewer than 100 studies had been done on the effects of secondary smoke." In fact, about a dozen studies had found a significant link between ETS and lung cancer or heart dis­ suit, “actually appears to support the EPA's decision. The report...concludes that there is 'a small but consistent elevation in the risk of lung cancer in non-smokers due to passive smoking.’” Thus Shapiro implied that the results of the study supported the claim that ETS causes lung cancer. But the sentence from which he quoted actually says that “our study and others conducted during the past decade suggest a small but consistent eleva­ tion in the risk of lung cancer.” (Emphasis added.) In fact, the study itself did not find a statistically significant associ­ ation between ETS and lung cancer. That is why the tobac­ co com panies argued that it would have underm ined the E PA 's c a se . S h a p iro a lso sm u g ly q u o te d the researchers' opinion that "the proliferation of federal, state and local regulations that restrict smoking in public places and work sites is well-founded." This editorial comment does not change the data. Many other stories raised false doubts about the arguments of the EPA’s critics. In the July 22, 1993 New York Times, for example, Philip J Hilts reported that Representatives Thomas J. Bliley, Jr. (R-VA) and Alex McMillan (R-NC) “suggested that the EPA’s study of several studies, or ‘meta-analysis,’ used a lower standard of statistical proof than nor­ mally used in assessing danger scientifically." Despite the implication of the word suggest­ ed, this is not an arguable point, although the report's detractors and supporters disagree about its importance. Hilts also stated that “about 30 studies were reviewed, of which 24 showed that secondhand smoke was a risk"— just the opposite was true. And he had the congressmen conceding the very point they were disputing: "The biggest study, the two lawmakers noted, found statistical proof that secondhand smoke caused cancer with cer­ tainty only in those people subjected to the most smoke." No study has ever found "statis­ tical proof that secondhand smoke caused cancer with certainty." (In fact, it is impossi­ ble for an epidemiological study to provide such proof.) Reporters don’t trust the tobacco companies. But they do trust the EPA. ease. In this case, the reporter misunderstood his source, and a phone call to the AHA would have cleared up the matter. In other cases, it might be necessary to consult an independent authority fam iliar with the research. Reporters will soon have an opportunity to do better. In testimony last February. EPA Administrator Carol Browner predicted that the Smoke-Free Environment Act would save the lives of 5,000 to 9.000 non-smokers each year. Dave Mudarri of the EPA’s Indoor Air Division says fewer than 2,200 of these represent lung-cancer cases; the rest are heart-disease deaths. Yet the evidence of a link between ETS and heart disease is even weaker than the evidence of a link between ETS and lung cancer, and the EPA has never done a risk assessm ent in this area. The agency’s full report on the impact of the Smoke-Free Environment Act was sched­ uled to be released in the spring. If reporters want to get at the truth, they cannot continue to act as if only one side in this debate has an ax to grind. They need to be just as skeptical about the EPA and the Coalition on Smoking or Health as they are about Philip Morris. “I treat sources like lawyers, like advocates in a court of law, ’ Fumento says. In a court of law the jurors take for granted that each side has an agen­ da, but that does not stop them from weigh­ ing the argum ents. Sim ilarly, reporters should not dism iss a statem ent sim ply because it comes from the Tobacco Institute. Writing in Toxicologic Pathology. Yale epidemiologist Alvan Feinstein cautioned his fellow scientists against automatically believing everything the “good guys” say and rejecting everything the “bad guys" say. His message applies to journalists as well as scientists: “If public health and epidemiolo­ gy want to avoid becoming a branch of poli­ tics rather than science, the key issues are methods and process, not the ‘goodness’ of the goals or investigators. In science even more than law. the ‘bad guy’...should always have the right to state his case, and a well- stated case has the right to be heard, regard­ less of who pays for it."* Jacob Sullum is managing editor of Reason magazine. O 1994 Forbes MediaCrMtc Reprinted with permission MEDIACRITIC S econdhand S moke : C onsider T he F acts , T hen D ecide . In January 1993, the EPA issued its report declaring that secondhand smoke is harmful to non-smokers. Since that time, this report, accepted in large part without question, has caused considerable concern among smokers and non-smokers alike. And while these concerns grew, the flaws in the EPA’s use of science remained largely unpublicized. Finally, these flaws are being publicly discussed. In this meticulously researched article in the current issue of Forbes MediaCritic, Jacob Sullum, Managing Editor of Reason magazine explains why the public never got the full story about the EPA report. He also details exactly how the EPA disregarded established methods of statistical analysis to arrive at a politically motivated conclusion about secondhand smoke. Since the EPA’s report has been the basis for a flurry of smoking restrictions, we believe that smokers and non-smokers need to have both sides of the story in order to make up their own minds. After all, recent polls show that most Americans prefer accommodation and common courtesy to more smoking regula­ tions and outright bans. For a full copy of this article and more information, please call 1 800 852-5325. PHILIP MORRIS U.S.A. I n A ny C ontroversy , F acts M ust M atter . > I ■ .v k '■ I © 1994 JcS Philip Morris Inc. H O « • ' Ì4 J s» w xTÍ>- ’S •. » -C x / Jh'> & ¿ O ’ idCKlftii