Readers of this site will likely know I am critical of the rise in "fact checking" as I feel what these "fact checkers" do is often more akin to punditry than fact checking. One such organization, Politifact, did a Q&A session on Twitter today where I happened to see it say:
We come from tradition of newspaper reporting where we put our personal opinions aside at work. I think that helps us fact-check better. https://t.co/1TQJRkeaCr
— PolitiFact (@PolitiFact) December 16, 2016
In response to claims it is liberally biased. I felt this tweet exaggerates the impartiality of Politifact due to seeing a number of articles it published which seemed biased against conservative or toward liberal views. As such, I asked:
.@PolitiFact And what do you do when people have specific complaints about your "fact checks"? Because there are a number that are wrong.
— Brandon S? (@Corpus_no_Logos) December 16, 2016
The answer I got directed me to e-mail Politifact with any specific concerns. I did so. I initially planned to give a sampling of issues in a number of "fact checks," but in the process of finely parsing one article I selected, I discovered so many problems there was little space. Indeed, there were enough problems with that article I couldn't even discuss them all in my e-mail.
I don't know what to expect from Politifact regarding this e-mail, but since I went through the trouble of writing it, I figured I might as well post it online for people to see. Perhaps it will give some insight as to why I don't hold much respect for "fact checkers." The e-mail discusses only a tiny fraction of the issues I've seen in Politifact's fact checking, and even so, it runs over a 1,000 words. Oh, and yes, I do realize there are a few typos in it. That's what I get for not having an editor. (I count three. How many do you count?)
I have noticed a number of errors in the "Fact checks" on your site, and it was suggested to me I should contact you here about them. I will focus primarily on articles about scientific issues due to my personal knowledge and views on science commonly being portrayed as partisan.
To begin, I would like to highlight a relatively trivial example regardin one Rush Limbaugh. A "fact check" on your site lists this as what is being checked:
The presence of gorillas calls into question the concept of evolution.
— Rush Limbaugh on Tuesday, May 31st, 2016 in the Rush Limbaugh Show
A naive person might think that this statement is attributed to Rush Limbaugh means he actually said it. He did not. Unlike most things being checked on your site, this statement is not placed in quotation marks because it is a paraphrase rather than a quotation. What Limbaugh actually said is:
"A lot of people think that all of us used to be gorillas, and they're looking for the missing link out there," Limbaugh said. "The evolution crowd. They think we were originally apes. I've always had a question: If we were the original apes, then how come Harambe is still an ape, and how come he didn't become one of us?"
The paraphrase given for this is inappropriate. While Limbaugh may have meant to use his question to create the rhetorical effect given in that paraphrase, there are many other potential interpretations. For instance, a teacher might ask the same question in order to get students to think about details how of evolution works. A person who knows little about evolution might ask the question because they sincerely want to know the answer.
Limbaugh's statement is rated as "False" because the "fact check" assumes one possible interpretation of the purpose of Limbaugh's question is the only possible one, to the extent it never even attempts to justify that assumption. That is inappropriate.
In an article on global warming:
There are multiple inaccurate remarks. A notable example is a strange transition the article goes through, demonstrated by these two sentence:
"Santorum is likely referring to Richard Tol, an economist at the University of Sussex who’s been a vocal critic of the 97 percent figure."
"Santorum’s claim confuses several points. First, the critic of the 97 percent he’s referring to isn’t the "head" of the UN’s climate panel, but an economist who has collaborated with but has since left the IPCC."
It is difficult to see how one could argue "Santorum is likely referring" to a person then conclude as fact Santorum is referring to "an economist" of any sort. If one says they are not certain who Santorum is referring to, it is misleading to state as fact he was referring to a specific individual. It is the probable conclusion, but "fact checkers" should not state as fact things which are merely probable.
There are specific errors in the article as well. I won't try to document them all, but I'll list a few. The article says:
Santorum is likely referring to Richard Tol, an economist at the University of Sussex who’s been a vocal critic of the 97 percent figure. He is not, as Santorum claims, the "head" of the IPCC, though he was the convening lead author of a chapter of the IPCC’s fifth report. He has since parted ways with the IPCC.
Richard Tol did not part ways with the IPCC. The article linked to to support that claim does not say he had. What it says is Richard Tol "was withdrawing from the team writing the summary" of the IPCC Working Group 2 Fifth Assessment Report. That Tol withdrew from a team working on one aspect and one Working Group's report does not mean he parted ways with the IPCC as a whole. Indeed, Tol remained a lead author of a chapter of that very report.
The article also says
The study Santorum is attempting to describe is a 2009 survey by Peter Doran, a professor of earth science at Louisiana State University. About 90 percent of around 3,000 surveyed earth scientists said they think climate change is happening, and about 82 percent said human activity is contributing to it.
The 97 percent figure comes from a subsample of climate scientists in Doran’s study, and Santorum correctly describes its small size: 74 out of 77 respondents said they agreed that climate change is man-made.
This contains multiple inaccuracies: 1) The second question did not ask if "human activity is contributing to" global warming, but rather, if "human activity is a significant contributing factor to changing mean global temperatures." It is not appropriate to conflate "contributing to" and "a significant contributing factor. A person could reasonably believe there are contributing factors which are not significant.
2) It is misleading to say "74 out of 77 respondents" said anything. This is true on two counts. First, the reported results of the Doran & Zimmerman study say 75 out of 77, not 74 out of 77.
3) 79 people met the criteria to be included in this subset. All 79 were asked if they believed global mean temperatures had generally risen, generally fallen or remained relatively constant compared to levels prior to 1800. Two of the 79 answered "remained relatively constant." They were then filtered out and not asked the follow-up question. Excluding them in this way without warning people is inappropriate. The actual numbers would change from 96.1% to 93.7%. (This same pattern of filtering out respondents was used for the entire survey, meaning all of its reported results are similarly distorted if one provides no caveat.)
Cook’s study found that among over 4,000 studies that took a position on man-made climate change, 97.1 percent "endorsed the consensus position that humans are causing global phase" and 97.2 of 1,300 authors who responded agreed with the position.
There is no such thing as "global phase." The quotation should read "global warming." It appears several words from the original quotation have been inadvertently removed during the editing process.
Additionally, Cook and his friends did not state anything about "over 4,000 studies." They specifically refrained from examining studies as a whole. Instead, they looked at the abstracts (basically, summaries) for scientific papers. While summaries may in many cases express the same view on the "consensus" as the body of the paper might, the body of papers will say many things not included in their summary. In many cases, that might include a position on the "consensus." It is inappropriate to conflate results for the summaries of papers with results for the papers as a whole.
Finally, it is worth noting the "consensus" position expressed by Cook et al does not match that expressed by Doran & Zimmerman. Cook et al included in its consensus any statement which at least acknowledged the greenhouse effect is real. Many papers were included in its "consensus" simply because they said things akin to, "Methane is a greenhouse gas." Statements like those do nothing to establish human impact is a signficant contribution to global warming. Readers are unlikely to notice this distinction unless it is highlighted.
If you have any questions or wish for further information regarding anything I have said, feel free to ask. I'd be happy to help. I think fact checking is a very important thing. I just can't view what you do as actually being fact checking given this e-mail highlights only a small sample of the many issues I have found with your articles.
I strongly suspect fact checking your "fact checks" COuld be a full-time job. Not that anyone would pay for it.