A Measure of Media Bias (research shows Drudge/Fox centrist, NYT far liberal)

Tim Groseclose Department of Political Science, UCLA, and Graduate School of Business, Stanford University

We are grateful for the research assistance by Aviva Aminova, Jose Bustos, Anya Byers, Evan Davidson, Kristina Doan, Wesley Hussey, David Lee, Pauline Mena, Orges Obeqiri, Byrne Offut, Matt Patterson, David Primo, Darryl Reeves, Susie Rieniets, Tom Rosholt, Michael Uy, Michael Visconti, Margaret Vo, Rachel Ward, and Andrew Wright. Also, we are grateful for comments by Mark Crain, Tim Groeling, Wesley Hussey, Chap Lawson, Jeff Lewis, and Barry Weingast.

¡°The editors in Los Angeles killed the story. They told Witcover that it didn¡¯t ¡®come off¡¯ and that it was an ¡®opinion¡¯ story. ¡The solution was simple, they told him. All he had to do was get other people to make the same points and draw the same conclusions and then write the article in their words.¡± (emphasis in original) Timothy Crouse, Boys on the Bus, 1973, p. 116.

Do the major media outlets in the U.S. have a liberal bias? Few questions evoke stronger opinions, and we cannot think of a more important question to which objective statistical techniques can lend their service. So far, the debate has largely been one of anecdotes (¡°How can CBS News be balanced when it calls Steve Forbes¡¯ tax plan ¡®wacky¡¯?¡±) and untested theories (¡°if the news industry is a competitive market, then how can media outlets be systematically biased?¡±).

Few studies provide an objective measure of the slant of news, and none has provided a way to link such a measure to ideological measures of other political actors. That is, none of the existing measures can say, for example, whether the New York Times is more liberal than Tom Daschle or whether Fox News is more conservative than Bill Frist. We provide such a measure. Namely, we compute an ADA score for various news outlets, including the New York Times, the Los Angeles Times, USA Today, the Drudge Report, Fox News¡¯ Special Report, and all three networks¡¯ nightly news shows.

Our results show a very significant liberal bias. All of the news outlets except Fox News¡¯ Special Report received a score to the left of the average member of Congress. Moreover, by one of our measures all but three of these media outlets (Special Report, the Drudge Report, and ABC¡¯s World News Tonight) were closer to the average Democrat in Congress than to the median member of the House of Representatives. One of our measures found that the Drudge Report is the most centrist of all media outlets in our sample. Our other measure found that Fox News¡¯ Special Report is the most centrist. These findings refer strictly to the news stories of the outlets. That is, we omitted editorials, book reviews, and letters to the editor from our sample.

To compute our measure, we count the times that a media outlet cites various think tanks. We compare this with the times that members of Congress cite the same think tanks in their speeches on the floor of the House and Senate. By comparing the citation patterns we can construct an ADA score for each media outlet.

As a simplified example, imagine that there were only two think tanks, one liberal and one conservative. Suppose that the New York Times cited the liberal think tank twice as often as the conservative one. Our method asks: What is the estimated ADA score of a member of Congress who exhibits the same frequency (2:1) in his or her speeches? This is the score that our method would assign to the New York Times.

A feature of our method is that it does not require us to make a subjective assessment of how liberal or conservative a think tank is. That is, for instance, we do we need to read policy reports of the think tank or analyze its position on various issues to determine its ideology. Instead, we simply observe the ADA scores of the members of Congress who cite the think tank. This feature is important, since an active controversy exists whether, e.g., the Brookings Institution or the RAND Corporation is moderate, left-wing, or right-wing.

One of the most curious and surprising statistics in all of American politics is that an overwhelming number of journalists are liberal. For instance, Elaine Povich (1996) reports that only seven percent of all Washington correspondents voted for George Bush in 1992, compared to 37 percent of the American public. Lichter, Rothman and Lichter, (1986) and Weaver and Wilhoit (1996) report similar findings for earlier elections.

The reason this statistic is curious and surprising is that many consider the media the watchdog of government, sometimes calling it the ¡°Fourth Branch of American Government.¡± If so, it is by far the least representative of the branches. These statistics suggest that journalists, as a group, are more liberal than almost any congressional district in the country. For instance, in the Ninth California district, which includes Berkeley, twelve percent voted for Bush, nearly double the rate of journalists. In the Eighth Massachusetts district, which includes Cambridge, nineteen percent voted for Bush, more than triple the rate of journalists. In the 14th California district, which includes Palo Alto, 26 percent voted for Bush, more than four times the rate of journalists.

It is interesting to compare the unrepresentative nature of the media to the purported unrepresentative nature of the U.S. Senate. Some have noted that the U.S. Senate is unrepresentative of voters because small states are overrepresented. Further, since small states in the U.S. tend to be more conservative than large states, this causes a conservative bias in the Senate. However, even if the entire U.S. Senate were chosen only by voters from Mississippi, the most conservative state in the union in 1992 (50 percent voted for Bush), such an electorate would still be significantly more representative than the Fourth Branch of Government.

Of course, however, just because a journalist has liberal or conservative views, this does not mean that his or her reporting will be slanted. For instance, as Kathleen Hall Jamieson (2000, 188) notes,

One might hypothesize instead that reporters respond to the cues of those who pay their salaries and mask their own ideological dispositions. Another explanation would hold that norms of journalism, including `objectivity¡¯ and `balance¡¯ blunt whatever biases exist.¡±

It is an unwritten law of current political journalism that conservative Republican Presidential candidates usually receive gentler treatment from the press than do liberal Democrats. Since most reporters are moderate or liberal Democrats themselves, they try to offset their natural biases by going out of their way to be fair to conservatives. No candidate ever had a more considerate press corps than Barry Goldwater in 1964, and four years later the campaign press gave every possible break to Richard Nixon. Reporters sense a social barrier between themselves and most conservative candidates; their relations are formal and meticulously polite. But reporters tend to loosen up around liberal candidates and campaign staffs; since they share the same ideology, they can joke with the staffers, even needle them, without being branded the ¡°enemy.¡± If a reporter has been trained in the traditional, ¡°objective¡± school of journalism, this ideological and social closeness to the candidate and the staff makes him feel guilty; he begins to compensate; the more he likes and agrees with the candidate personally, the harder he judges him professionally. Like a coach sizing up his own son in spring tryouts, the reporter becomes doubly severe. (1973, 355-6)

However, a strong form of the view that reporters offset or blunt their own ideological biases leads to a counterfactual implication. Suppose it is true that all reporters report objectively and their ideological views do not color their reporting. If so, then all news would have the same slant. Yet, few would disagree that Fox News or the Washington Times has a more conservative slant the New York Times.

A large number of economic studies give theoretical reasons that bolster the view that the media does not have a systematic bias. (See xx, xx, xx and xx). The idea is that if there were a systematic bias, then an entrepreneur could form a new media outlet that does not have a bias. This outlet would drive the others out of business. This is a compelling argument, and even the libertarian Cato Journal has published an article agreeing with the view: In this article, the author, Daniel Sutter (2001), concludes that, although it might be possible for a systematic bias to exist in the network news (since, before cable television, there were strong barriers to entry in this industry), such a bias is impossible, or at least very unlikely, for the newspaper, radio, or magazine industry.

However, contrary to the views and evidence cited above, we find a significant liberal bias in our sample of media outlets. This presents a challenge to economic theorists. Given that there is a systematic liberal bias the news market, at least one of the assumptions in the above theoretical studies must be inaccurate.

The web site, www.wheretodoresearch.com lists 200 of the most prominent think tanks in the U.S. Using the official web site of Congress, http://thomas.loc.gov, we and our research assistants searched the Congressional Record for instances where a member of Congress cited one of these think tanks. We looked for instances where the legislator cited a view or a fact stated by a member of the think tank. We then counted the sentences in the citation. We also recorded the average adjusted ADA score of the member who cited the think tank.

Along with direct quotes, we sometimes included sentences that were not direct quotes. For instance, many of the citations were cases where a member of Congress noted ¡°This bill is supported by think tank X.¡± Also, members of Congress sometimes insert printed material ¡°into the Record,¡± such as a letter, a newspaper article, or a report. If a think tank was cited in such material or if a think tank member wrote the material, we counted it just as if the member of Congress had read the material in his or her speech.

We did the same exercise for stories that media outlets report, except with media outlets we did not record an ADA score. Instead, our method estimates such a score.

Sometimes a legislator or a media outlet noted an action that a think tank had taken¡ªe.g. that it raised a certain amount of money, initiated a boycott, filed a lawsuit, elected new officers, or held its annual convention. We did not record such cases in our data set. However, sometimes in the process of describing such actions, the reporter or member of Congress would quote a member of the think tank, and the quote revealed the think tank¡¯s views on national policy, or the quote stated a fact that is relevant to national policy. If so, we would record that quote in our data set. For instance, suppose a reporter noted ¡°The NAACP has asked its members to boycott businesses in the state of South Carolina. `We are initiating this boycott, because we believe that it is racist to fly the Confederate Flag on the state capitol,¡¯ a leader of the group noted.¡± In this instance, we would count the second sentence that the reporter wrote, but not the first.

Also, we omitted the instances where the member of Congress or journalist only cited the think tank so he or she could criticize it or explain why it was wrong. About five percent of the congressional citations and about one percent of the media citations fell into this category.

In the same spirit, we omitted cases where a journalist or legislator gave an ideological label to a think tank (e.g. ¡°Even the left-wing Urban Institute favors this bill.¡±). The idea is that we only wanted cases were the legislator or journalist cited the think tank as if it were a disinterested expert on the topic at hand. About two percent of the congressional citations and about five percent of the media citations fell into this category.

For the congressional data, we coded all citations that occurred during the period Jan. 1, 1993 to December 31, 2002. This covered the 103rd thru 107th Congresses. We calculated the average adjusted ADA score for each member of Congress over the period 1993 to 1999.

As noted earlier, the media data does not include editorials, letters to the editor, or book reviews. That is, all of our results express the bias of news reporting of media outlets and not, e.g., the editorial pages of newspapers and magazines.

In Table 1 we list the 20 think tanks that are most commonly cited in Congress. The third column of the table lists the average adjusted ADA score of the members who cited the think tank, where this average is weighted by the number of sentences that the legislator cited. The fourth column lists the average score weighted by citations. It is an open question whether the proper level of observation is a sentence or a citation. That is, for instance, if a journalist cites five sentences from the Economic Policy Institute in one story, it is unclear whether this demonstrates the same liberal bias as citing one sentence from the Economic Policy Institute in five separate stories. As a consequence, we report all of our analyses for both levels of observation, sentences and citations.

As a comparison, in Table 2 we list the mean and median adjusted ADA scores of members of Congress for the period that we analyze. The average scores for the House and Senate were respectively 44.5 and 40.0. We calculated these by taking the average adjusted score for each year. Then, for the seven-year period for which we recorded adjusted scores (1993-1999), we calculated the average over these years. We did the same calculation for the median of the House and Senate. These were respectively 39.0 and 36.9.

Table 3 lists the average adjusted ADA score of some well-known moderate members of Congress. It includes the scores of the most conservative Democrat in our sample, Nathan Deal (Ga.), and the most liberal Republican in our sample, Constance Morella (Md.). Although Nathan Deal became a Republican in 1995, the score that we list in the table is calculated only from his years as a Democrat.

The tables shed some light on some much debated topics about the ideological position of various think tanks. First, the table reveals that the position of the Brookings Institution clearly leans left. When we use sentences as our level of observation, the average score of legislators citing Brookings is 50.0, and when we use citations, the average score is 46.2. In contrast, the average score of the House, 44.5, and the average score of the Senate, 40.0, are both more right wing than the average legislator citing Brookings.

Second, contrary to conventional wisdom, the RAND Corporation is fairly liberal. The adjusted ADA score of the average legislator citing it is 53.6, using sentences as the level of observation, and 52.6, using citations as the level of observation. This is significantly to the left of the center of Congress, although not as far left as, say, the Center on Budget and Policy Priorities, the Children¡¯s Defense Fund, or the Economic Policy Institute. We mentioned this finding to some members of RAND, who told us they were not surprised. While RAND strives to be middle-of-the-road ideologically, the more conservative scholars at RAND tend to work on military studies, while the more liberal scholars tend to work on domestic studies. Because the military studies are sometimes classified and often more mundane than the domestic studies, the media and members of Congress tend to cite the domestic studies disproportionately. As a consequence, RAND appears liberal when judged by these citations. It is important to note that this fact¡ªthat the scholars at RAND are more conservative than the numbers in Table 1 suggest¡ªwill not cause a bias to our results. To see this, think of RAND as two think tanks: RAND I, the left-leaning think tank which produces the research that the media and members of Congress like to cite, and RAND II, the conservative think tank which produces the research that they do not like to cite. Our results exclude RAND II from the analysis. This causes no more bias than excluding any other think tank that is rarely cited in Congress or the media.

Perhaps the biggest surprise of Table 1 is the average score for the ACLU. Weighted by citations, the average score, is 42.66, which is near the center of congressional scores. Weighted by sentences, the average score is 34.99, which is to the right of the average member of Congress. The primary reason that the ACLU appears so conservative is that it opposed the McCain-Feingold Campaign Finance bill. Consequently, conservatives tended to cite this fact often. Indeed, slightly more than half of the ACLU sentences cited in Congress were due to one person, Mitch McConnell (R.-Kt.), who strongly opposed the McCain-Feingold bill. If we omit ACLU citations that are due to McConnell, then the average score, weighted by sentences, increases to 70.12. Because of this anomaly, in the Appendix we report the results when repeat all of our analyses but omit the ACLU data. This causes the average score of the media outlets to become approximately one ?? point more liberal.

Because, at times, there is some subjectivity in coding our data, when we hired our research assistants we asked for whom they had voted in the last presidential election (or for whom they would have voted if they did not vote or if they voted for a candidate besides Bush or Gore). We chose research assistants so that approximately half our data was coded by Gore supporters and half by Bush supporters. We, the authors, did very little of the congressional coding, and we did none of the media coding. For each media outlet we assigned research assistants so that approximately half of the data was coded by a Gore supporter and half by a Bush supporter.

Finally, for each media outlet we selected an observation period for the data that we estimated would yield at least 1200 sentences of data. Because there is less data to collect for magazines and television shows (e.g. a transcript for a 30-minute show contains only a small fraction of the sentences that are contained in a daily newspaper), we collected all the dates that were available in Lexis-Nexis for these two forms of media.

We use a fairly complex method to estimate ADA scores for media outlets¡ªit involves maximizing a likelihood function that is similar to a multinomial logit. However, some simple descriptive statistics generate the same general conclusions.

Like we did in Table 1, for the remaining think tanks in our sample we computed the average adjusted ADA score of the legislators who cited them. Next, we split the think tanks into a liberal group and a conservative group, based upon whether the average score of legislators citing the think tank was above or below 42.2, the midpoint of the House and Senate averages.

In Table 4 we list how frequently members of Congress cite the conservative and liberal groups, based upon total sentences. The entire Congress cited the two groups of think tanks approximately evenly. Specifically, of the total sentences that members of Congress cited, 43.1% were from the liberal group. As expected, if we confine our analysis only to Republican members of Congress, then we find that they cite the liberal think tanks less frequently than the entire Congress. Specifically, they cited think tanks from the liberal group, 16.6% of the time. Finally, of the total sentences that the Democrats cited, 81.5% were from the liberal group.

We do a similar analysis with media outlets and list the results in Table 5. Specifically, for each media outlet we list the percentage of sentences that it cited from the liberal group of think tanks. From this percentage, we can compute a back-of-the-envelope estimate of the media outlet¡¯s adjusted ADA score. For instance, note that of the total sentences that the L.A. Times cited from the two groups, 63.5% were from the liberal group. Note that this percentage is approximately halfway between the percentage for the Democrats (81.5) and the percentage for the entire Congress (43.1). Consequently, the back-of-the-envelope estimate for USA Today should be about halfway between the adjusted ADA scores of the average Democrat and the average of Congress. This procedure is illustrated in Figure 1. More specific, the L.A. Times¡¯ percentage was 53.1% of the distance between the percentages for the average of Congress and the average Democrat. Accordingly, we assign an adjusted ADA score which is 53.1% of the distance between 42.2 and 74.1, the scores of the average of Congress and the average Democrat. As Figure 1 illustrates, this analysis assumes that there is a linear relationship between the citation pattern of a media outlet and its adjusted ADA score. However, the analysis allows the slope to differ according to whether the media outlet is to the left or right of the center of Congress. Table 5 lists the back-of-the-envelope scores for the other media outlets. Figure 2 illustrates how these scores compare to various members of Congress.

We repeat this analysis in Tables 6 and 7. However, here the level of observation that we use is the citation, not the sentence.

Like results from the more complex method we execute later, Tables 5 and 7 show a strong liberal bias. All the media outlets except Fox News¡¯ Special Report and the Drudge Report have a score that is left of the center of Congress. And this is true whether one uses citations or sentences as the level of observation or sentences, or whether one defines center of Congress by means or medians, or by House or Senate. Even the Drudge Report is left of center except in the case where we use sentences as the level of observation and we define center of Congress as the House mean. The fact that the Drudge Report is left of center is perhaps the most surprising of the table, given that it broke the Monica Lewinsky scandal, and its author, Matt Drudge has described himself as a libertarian. However, to understand this, it is useful to note the structure of the DrudgeReport. It is mostly a series of headlines with links to articles that are listed on other web sites. Also, it sometimes contains links to articles that Matt Drudge wrote himself. We included both types of articles when collecting data for the Drudge Report. However, in our sample no article written by Matt Drudge ever cited any of the 200 think tanks that we examine. Thus, all data for the Drudge Report comes from links to articles on other web sites. These other web sites included many liberal media outlets, including the New York Times, Los Angeles Times, and USA Today. Therefore, it is not so surprising that the Drudge Report leans left.

The back-of-the-envelope estimates are less than optimal for at least three reasons: (i) they do not give confidence intervals of their estimates; (ii) they do not utilize the extent to which a think tank is liberal or conservative (they only record the dichotomy, whether the think tank is left or right of center); and (iii) they are not embedded in an explicit choice model. We now describe a method that overcomes each of these three deficiencies.

Define yi as the average adjusted ADA score of the ith member of Congress. Given that the member cites a think tank, we assume that the utility that he or she receives from citing the jth think tank is

We assume that eij is distributed according to a Weibull distribution. As shown by McFadden (1974; also see Judge, et. al, 1985, pp. 770-2), this implies that the probability that member i selects the jth think tank is

where J is the total number of think tanks in our sample. Note that this probability term is no different from the one we see in a multinomial logit (where the only independent variable is yi ).

Define cm as the estimated adjusted ADA score of the mth media outlet. Similar to the members of Congress, we assume that the utility that it receives from citing the the jth think tank is

We assume that emj is distributed according to a Weibull distribution. This implies that the probability that media outlet m selects the jth think tank is

Although this term is similar to the term that appears in a multinomial logit, we cannot use multinomial logit to estimate the parameters. The problem is that cm, a parameter that we estimate, appears where normally we would have an independent variable. Instead, we construct a likelihood function from (1) and (2), and we use the ¡°ml maximize¡± command in Stata to obtain estimates of each aj , bj, and cm.

Similar to a multinomial logit, it is impossible to identify each aj and each bj. Consequently, we arbitrarily choose one think tank and set its values of aj and bj to zero. It is convenient to choose a think tank that is cited frequently. Also, to make most estimates of the bj ¡®s positive, it is convenient to choose a conservative think tank. Consequently, we chose the Heritage Foundation. It is easy to prove that this choice does not affect our estimates of cm . That is, if we had chosen a different think tank, then all estimates of cm would be unchanged.

One difficulty that arose in the estimation process is that Stata takes an unwieldy amount of time to estimate all of the parameters. We estimated some versions of the model where we restricted the sample to only the top 15, 20, or 25 think tanks. We found that the time that Stata takes to estimate the parameters is approximately proportional to the number of think tanks squared. Using this formula, we estimate that it would take eight weeks for the program to finish when all think tanks are included. Worse, we want to estimate several versions of the model (eg sentences versus citations as the level of observation, including and not including citations where the legislator mentions an ideological label, etc.). Thus, it would take several months for our computer to produce all of the estimates.

Instead, we estimate the aj ¡®s and bj ¡®s for only the top 25 most cited think tanks in Congress. These think tanks comprised 69.1% of the sentences cited in Congress, and they comprised 66.6% of the total citations. With the remaining think tanks we formed seven mega think tanks, based upon the average score of the legislator citing them, and we construct an additional aj and bj for each of these seven. In the case where we use sentences as the level of observation, if the average score of the legislator citing the think tank was less than 6.2 (and it was not among the top-25 most cited think tanks), then we placed it in the most conservative of the seven mega think tanks. We call this the strong conservative mega think tank. We formed six other think tanks, which we call weak conservative, moderate right, moderate, moderate left, weak liberal, and strong liberal mega think tanks. The cut points that we used to construct these were: 6.2, 12.6, 21.04, 45.5, 59.9 and 71.3. We chose these cut points so that the seven mega think tanks would each have approximately the same number of observations. For the case where our level of observation is citations, we used the cut points: 15.0, 27.7, 39.9, 48.5, 59.25, and 68.0. As a check, we also ran the model when we included only the top-20 or the top-15 think tanks and placed all the rest into one of the seven mega think tanks. We also estimated the model by constructing three or five mega think tanks, instead of seven. In each of these cases the results differed from our main results in only minor ways. For instance, in each case the average adjusted ADA score of the eight media outlets differed less than one-half ?? point from the average score of the results that we report. And in no instance, did a score of any media outlet differ by more than three ?? points from the score we report.

In Tables 8 and 9 we list our estimates of cm , the adjusted ADA score for a media outlet. The two tables differ according to whether one uses, respectively, sentences or citations as the level of observation. The tables also list the standard errors of these estimates along with the period over which the sample is gathered. In Figures 2 and 3 we illustrate these estimates and show how the media outlets compare to members of Congress.

The estimates in the table are very similar to the back-of-the-envelope estimates that we report in Tables 5 and 7. For instance, most differ by less than three points from the back-of-the-envelope estimates.

As before, these results show a strong liberal bias among the media. When we use citations as our level of observation, all media outlets except Fox News¡¯ Special Report are left of center. When we use sentences as our level of observation, all media outlets except Fox News¡¯ Special Report and the Drudge Report are to the left of center. And, depending upon how one defines center, even the Drudge Report is to the left of center. Only if one defines the House mean as the center, is the Drudge Report right of center. If instead one uses the House median, Senate median, or Senate mean, the Drudge is to the left of center.

In discussing left- or right- wing biases of the media, one should be careful how he or she defines center. We think the most appropriate definition refers to a central voter, as opposed to a central member of Congress. Accordingly, we think that it is more appropriate to compare media scores to the House as opposed to the Senate, since the Senate disproportionately represents small states. Next, we think it is more appropriate to use the median House member, instead of the mean. One reason is that, because of The Median Voter Theorem (Black, 1957), one should expect policy to be at the median instead of the mean. Another reason is that comparisons to a mean can be manipulated by the ADA¡¯s choices of roll call votes, whereas comparisons to a median are not subject to such manipulation.

To see this, first note that the ADA has considerable leeway in the roll call votes that it chooses. For instance, suppose it chooses many roll calls such that the cut point of the roll call lies between moderates and extreme liberals. Such a cut point would cause moderates to form a coalition with extreme conservatives on the roll call. (An example of such a roll call would be a bill to ban partial-birth abortions. Here, moderates and conservatives favor the ban, and only extreme liberals oppose it. ) A prevalence of such cut points would cause moderates to have ADA scores more similar to conservatives than liberals. Meanwhile, if it predominantly chose cutpoints on the other side, then the ADA would cause moderates to have ADA scores more similar to liberals than conservatives.

Because of this leeway, with one set of roll calls, the ADA could make a member of Congress or media outlet appear more left-wing than the mean score. However, with a different set of roll calls the ADA could make the same member of Congress or the same media outlet appear more right wing than the mean score. To see this, consider the following example. Suppose there are only five members of Congress. The most left-wing legislator is Member 1, who is more left-wing than member 2, who is more left-wing than member 3, and so on. Suppose media outlet A has an ideology identical to member 2. Consequently, its ADA score (that our method estimates) will be identical to member 2¡¯s score (at least in expectation).

Now suppose that the ADA chooses four roll calls, such that the first roll call has a cut point between members 1 and 2, the second has a cut point between members 2 and 3, and so on. Because the distribution of cut points is uniform, member 1 receives 100 ADA score, member 2 and media outlet A to receive a 75, member 3 receives a 50, and so on. The mean ADA score of the legislators is 50. Thus, this set of roll calls makes media outlet A appear more left-wing than the mean score.

Next, instead suppose that the ADA chooses four roll calls such that each has a cut point between members 1 and 2. This would cause member 1 to receive a 100 score. Media outlet A and members 2, 3, 4, and 5 would receive a 0 score. The mean ADA score in this case would be 20. Thus, this set of roll calls makes media outlet A appear more right-wing than the mean score.

Meanwhile, for this example, regardless of the ADA¡¯s choice of cut points, media outlet A¡¯s score will necessarily be greater than or equal to the median¡¯s score (member 3). That is, unlike the case where we use the mean score as a comparison, it is impossible to make media outlet A appear more right-wing than the median score.

The point of this example is not to suggest that the ADA might intentionally choose roll calls to manipulate a legislator¡¯s or media outlet¡¯s perceived ideology relative to the mean. Rather it is to demonstrate an arbitrariness that exists when one uses a mean score for comparison. The same arbitrariness does not exist with median scores. As a consequence, we think it is appropriate to compare the scores of media outlets with the House median, 39.0.

We now compute the difference of a media outlet¡¯s score from 39.0 to judge how centrist it is. Based on sentences as the level of observation (the results of which are listed in Table 8), the Drudge Report is the most centrist, Fox News¡¯ Special Report is second, ABC World News Tonight is third, and CBS Evening is last.

Given that the conventional wisdom is that the Drudge Report and Fox News are conservative news outlets, this ordering might be surprising. Perhaps more surprising is the degree to which the ¡°mainstream¡± press is liberal. The results of Table 8 show that the Los Angeles Times, the New York Times, USA Today, and CBS Evening News are not only liberal, they are closer to the average Democrat in Congress (who has a score of 74.1) than they are to the median of the whole House (who has a score of 39.0).

Another interesting fact concerns the following claim: ¡°Although the New York Times and other media are liberal, they are balanced by conservative media outlets such as Fox News. Consequently, if one spent an equal amount of time watching Fox News and reading the New York Times, he or she would receive a fairly balanced view of the news.¡± However, Table 8 shows that this is not quite true. Since the New York Times is twice as far from the center as Fox News¡¯ Special Report, to gain a balanced perspective, one would need to spend twice as much time watching Special Report as he or she spends reading the New York Times. (Further as we shall see in Table 9, when one uses citations as the level of observation, one would need to spend an even greater amount of time watching Special Report to gain a balanced perspective.)

A natural question is whether the differences in these rankings are statistically significant. We do not report the variance-covariance matrix of the parameters, however for any two estimated ADA scores of the media outlets, the covariance of the parameters was approximately .07. (The covariance between any two parameters varied between .055 and .079.) Given this, one can compute t-statistics to test the statistical significance of the difference in scores between any two media outlets. For instance, the variance of the difference between the scores of the Drudge Report and ABC World News Tonight is

The difference in their scores is 7.9. Thus, the t-statistic, testing whether the scores are significantly different is 7.9/sqrt(5.04) = 3.52, which is significant at the 1% confidence level. Similar calculations show that the Drudge Report is significantly closer to the center than all other media outlets; ABC World News Tonight and NBC Nightly News do not significantly differ; however, these two network news shows do significantly differ from all the newspapers in our sample and CBS Evening News.

Using citations as the level of observation, Table 9 shows that Fox News¡¯ Special Report is the most centrist news outlet in our sample, the Drudge Report is second, ABC World News Tonight is Third, and CBS Evening News is last.

For these results, the covariance of the estimate between any two media outlets is approximately 1.0. Thus, for instance, to test if the Drudge Report¡¯s score is significantly different from the score of ABC World News Tonight, one uses the formula

Thus, at standard levels of statistical significance, in this case, the scores of the Drudge Report and ABC World News Tonight are not significantly different. Similar calculations show that Fox News¡¯ Special Report is significantly closer to the center than all media outlets except the Drudge Report. Other calculations show that NBC Nightly News does not significantly differs from CBS Evening News at a 5% confidence level, but it does at a 10% confidence level.

We believe that the most innovative and important aspect of our method is our idea to compare news stories with congressional speeches. First, this allows one to provide a baseline for how liberal or conservative a media outlet is. Some previous studies have only tried to determine if a news outlet gave equal coverage to each side of an issue or gave equal treatment to two opposing politicians running for the same office. However, depending on the issue, a media outlet could give equal treatment to both sides, yet still be biased because of the issues it selects. For instance, suppose the issue is whether to make ¡°day-after¡± abortion pills, like RU-486, illegal. Only the most ardent conservatives favor this position. Equal treatment would mean that the story had a conservative slant. Likewise, if a reporter gave equal treatment to the question of allowing partial-birth abortions, he or she would slant the story in the conservative direction. Further, depending on the candidates, equal coverage of the candidates could imply a bias. To see this, imagine David Duke, a former leader of the Ku Klux Klan, were running against a moderate Democrat. If news story gave equal coverage to the two candidates, this would be a right-wing bias. In contrast, if a researcher applies his or her method to congressional speeches, as well as news stories, this gives a baseline for determining how liberal or conservative is the news story. Namely, he or she can report how the media outlet compared to the center of Congress, the average Democrat, the average Republican, etc.

Second, our idea helps researchers to discover when they have found a false negative result. For example, several researchers have conducted tests which conclude that there is no bias in the U.S. news media. (Xx, xx, xx.) Some of these tests strike us as extremely weak¡ªtests which could lead one to conclude no bias when a bias really exists. It is similar to a doctor saying ¡°Your hair looks fine, therefore you must be well,¡± without examining the patient¡¯s nose, throat, blood pressure, etc. One could expose the doctor¡¯s faulty treatment if he or she asked the doctor to perform the same examination on a patient who is known to be ill. We can apply a similar example to media research. Suppose that a researcher applied his or her method to the speeches of Ted Kennedy and concluded that Ted Kennedy does not have a liberal bias. This, we believe, would reveal a problem with that method.

We think it is important for researchers of media bias to apply their method to speeches of members of Congress. Further, if a researcher uses a method that cannot be applied to congressional speeches, we caution that this is a weakness of the method. It would be similar to the above hypothetical doctor saying ¡°My method cannot be applied to the known sick patient. You¡¯ll have to trust my assessment without seeing the test you suggest.¡±

Although we expected to find that most media lean left, we were astounded by the degree. A norm among journalists is to present ¡°both sides of the issue.¡± Consequently, while we expected members of Congress to cite primarily think tanks that are on the same side of the ideological spectrum as they are, we expected journalists to practice a much more balanced citation practice, even if the journalist¡¯s own ideology opposed the think tanks that he or she is sometimes citing. This was not always the case. Most of the mainstream media outlets that we examined (ie all those besides Drudge Report and Fox News¡¯ Special Report) were closer to the average Democrat in Congress than they were to the median member of the House.

Our results contrast strongly with the prior expectations of many others. It is easy to find quotes from prominent journalists and academics who claim that there is no systematic bias among media outlets in the U.S. The following are some examples: ¡°Our greatest accomplishment as a profession is the development since World War II of a news reporting craft that is truly non-partisan, and non-ideological, and that strives to be independent of undue commercial or governmental influence....It is that legacy we must protect with our diligent stewardship. To do so means we must be aware of the energetic effort that is now underway to convince our readers that we are ideologues. It is an exercise of, in disinformation, of alarming proportions. This attempt to convince the audience of the world¡¯s most ideology-free newspapers that they¡¯re being subjected to agenda-driven news reflecting a liberal bias. I don¡¯t believe our viewers and readers will be, in the long-run, misled by those who advocate biased journalism.¡± ¨C New York Times Executive Editor Howell Raines accepting the ¡°George Beveridge Editor of the Year Award¡± at a National Press Foundation dinner shown live on C-SPAN2 February 20, 2003.

¡°¡when it comes to free publicity, some of the major broadcast media are simply biased in favor of the Republicans, while the rest tend to blur differences between the parties. But that¡¯s the way it is. Democrats should complain as loudly about the real conservative bias of the media as the Republicans complain about its entirely mythical bias¡¡± --Paul Krugman, ¡°Into the Wilderness,¡± New York Times, November 8, 2002.

"The mainstream media does not have a liberal bias. . . . ABC, CBS, NBC, CNN, the New York Times, The Washington Post, Time, Newsweek and the rest -- at least try to be fair." --Al Franken. (2003, xx) Lies and the Lying Liars Who Tell Them: A Fair and Balanced Look at the Right.

Black, Duncan. 1958. The Theory of Committees and Elections. London: Cambridge University Press.

Groseclose, Tim, Steven D. Levitt, and James M. Snyder, Jr. 1999. ¡°Comparing Interest Group Scores across Time and Chambers: Adjusted ADA Scores for the U.S. Congress,¡± American Political Science Review. 93 (March): 33-50.

Jamieson, Kathleen Hall. 2000. Everything You Think You Know About Politics ¡ and Why You¡¯re Wrong. New York: Basic Books.

Judge, George G., W. E. Griffiths, R. Carter Hill, Helmut Lutkepohl, and Tsoung-Chao Lee. 1985. The Theory and Practice of Econometrics. New York: John Wiley and Sons.

Lichter, S.R., S. Rothman, and L.S. Lichter. 1986. The Media Elite. Bethesda, MD: Adler and Adler.

McFadden, Daniel. 1974. ¡°Conditional Logit Analysis of Qualitative Choice Behavior,¡± in P. Zarembka, ed. Frontiers in Econometrics, pp. 105-42. New York: Academic Press.

Povich, Elaine. 1996. Partners and Adversaries: The Contentious Connection Between Congress and the Media. Arlington, VI: Freedom Forum.

Sutter, Daniel. 2001. ¡°Can the Media Be So Liberal? The Economics of Media Bias.¡± The Cato Journal. 20 (Winter): 431-51.

Weaver, D.H. and G.C. Wilhoit. 1996. American Journalist in the 1990s. Mahwah, NJ: Lawrence Erlbaum.

Thank you indeed for posting this high quality study. I am impressed by the measure they use. While some folks will try to pick at it, the measure appears robust and objective.

It is interesting to note with regard to the argument that a free market would produce a relatively unbiased media that this point might still be right. As noted, the broadcast media was far from a free market. However, what could have also been noted is that for the past 50 or so years the newspaper business has also not really been free market as with circulation falling and papers going out of business, the barrier to new entry was very high. In those outlets where there is a free market (cable news, radio, internet, book publishing) the conservatives are holding their own or more. The reason the 'or more' is that the conservative fraction of the market was so badly served by the mainstream media.

This article discusses a method of extracting an "ADA rating" for journalists based on how their citations of liberal and conservative think tanks compare to those of members of Congress for whom actual ADA ratings are available. Strikes me as a nice piece of analysis - even if my eyes did glaze over when they got into the mathematical details of their statistics.

One of the most curious and surprising statistics in all of American politics is that an overwhelming number of journalists are liberal . . .
The reason this statistic is curious and surprising is that many consider the media the watchdog of government, sometimes calling it the ¡°Fourth Branch of American Government.¡± If so, it is by far the least representative of the branches.
Our greatest accomplishment as a profession is the development since World War II of a news reporting craft that is truly non-partisan, and non-ideological, and that strives to be independent of undue commercial or governmental influence....It is that legacy we must protect with our diligent stewardship. To do so means we must be aware of the energetic effort that is now underway to convince our readers that we are ideologues. It is an exercise of, in disinformation, of alarming proportions. This attempt to convince the audience of the world¡¯s most ideology-free newspapers that they¡¯re being subjected to agenda-driven news reflecting a liberal bias. I don¡¯t believe our viewers and readers will be, in the long-run, misled by those who advocate biased journalism.¡± ¨C New York Times Executive Editor Howell Raines accepting the ¡°George Beveridge Editor of the Year Award¡± at a National Press Foundation dinner shown live on C-SPAN2 February 20, 2003. when it comes to free publicity, some of the major broadcast media are simply biased in favor of the Republicans, while the rest tend to blur differences between the parties. But that¡¯s the way it is. Democrats should complain as loudly about the real conservative bias of the media as the Republicans complain about its entirely mythical bias¡ --Paul Krugman, ¡°Into the Wilderness,¡± New York Times, November 8, 2002.
"The mainstream media does not have a liberal bias. . . . ABC, CBS, NBC, CNN, the New York Times, The Washington Post, Time, Newsweek and the rest -- at least try to be fair." --Al Franken. (2003, xx) Lies and the Lying Liars Who Tell Them: A Fair and Balanced Look at the Right.

IOW, "The fox says he guards the chicken coop. Ergo, it is surprising that chickens disappear from the coop when the fox is 'on duty'."

Why Broadcast Journalism is
Unnecessary and Illegitimate

Do the major media outlets in the U.S. have a liberal bias? Few questions evoke stronger opinions, and we cannot think of a more important question to which objective statistical techniques can lend their service. So far, the debate has largely been one of anecdotes ("How can CBS News be balanced when it calls Steve Forbes tax plan wacky?") and untested theories ("If the news industry is a competitive market, then how can media outlets be systematically biased?".

Obviously that last untested theory "If the news industry is a competitive market" is wrong.
The news industry is competitive only in ways which do not cut the revenue of journalism as a whole. It's perfectly fair, in that world, to try to deliver the news faster than anyone else - what is considered unethical is to select the news on the basis of what is important and true rather than on the basis of what will sell the most papers.
Ironically, what sells the most papers is reports which call the safety of the reader into question. Which, IOW, implicitly make the case that the reader would be a helpless babe in the woods if not for the fearless reporter protecting him from otherwise unsuspected dangers. Such stories are inherently anti conservative, but the conservative public as well as the liberal public can't pass by the newsstand without reading that kind of story, even as he dissects it and ultimately dismisses it.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.