Goals: To develop students’ knowledge of best practice when it comes to fact-checking.

Human-rights related: Protecting access to information.

Media: Print, online media.

Length: 1 hour

Tools: Computer and projector to present slideshow; handouts for class exercise.

Preparation: Prepare slides based on information below; choose news article and print copies for students to use in class exercise.

Process: 1. Re-cap from lesson 1 (basics of fact-checking) – 5 mins

Briefly discuss learnings from previous lesson.

2. Introduce techniques for fact-checking best practice – 35 mins

Data & Society Research Institute‘s tips for best practice in reporting

(https://datasociety.net/output/oxygen-of-amplification/):

Internalize the idea that social media does not constitute a “person on the street” scenario, nor is an embedded tweet or Facebook post akin to a pulled quote. Not only is this information unreliable (the profile might be a bot, the person might be joking in ways inscrutable to the reporter, etc), but by collating average citizens’ tweets, reporters are directing readers to those citizens’ profiles, and opening them up to direct, targeted harassment.

Avoid pulling a handful of social media posts and then attributing that perspective, positive or negative, to “the internet.” Any conceivable perspective could be supported by that approach, and does not a critical mass make—although reporting on it as such could artificially create exactly that.

Reporters should talk to sources for digital culture stories at length, ideally face- to-face, whenever possible. According to The New York Times’ Farhad Manjoo, this approach yields greater insight into the totality of that person’s perspective, since a person’s online performative self may not accurately reflect that person’s true perspectives and motives, and/or may obscure details that would help shed light on the person’s digital footprint.

Laura Norkin, formerly of Refinery29, encourages reporters to “ask yourself why, and why now.” What is the point of having this conversation today? As with all good reporting, but particularly when the topic butts up against networked manipulation campaigns, if there is any doubt about the relevance of the story, or the ethics of covering it, reporters and their editors should ask someone.

It’s not just that journalists play an important role in the amplification of information. What gets reported – and what doesn’t – becomes part of broader cultural narratives, and those broader cultural narratives directly impact the lives of countless citizens. For this reason, reporters, editors, and publishers alike should prefigure every professional decision with the recognition that individual journalists are an integral part of the news being reported. There is no escape for anyone.

Who should you ask when verifying information?

• Data sources – Depending on the sort of claim you are checking, you may seek information from government papers and official statistics, company records, scientific studies and health research databanks, through to school records, development charity accounts, religious orders’ papers and others besides. As with all sources of information, it is important to know all you can about the organisation that gathered and holds the data before you use it

• Experts – Depending on the topic – if the claim made is on medical matters, or require detailed knowledge of a major company’s accounts, or a fine point of law – it may be more suitable to check a claim by talking to a number of recognised experts. When doing this, the most important thing is to know and declare any interest the expert may have in the matter that may cause, or be seen to cause, a bias in their analysis

• The crowd – Again depending on the topic, the best source for information might be the knowledge to be found in the wider community; crowdsourcing as it is known. If an official claims on election morning that all polling stations received their ballot papers on time, or an environmental group claims a factory is polluting a neighbourhood, the best placed people to confirm or undermine what they say may be people in the wider community. You need to know who your sources are and whether the information they supply is reliable.

The importance of checking evidence

(from Africa Check, https://africacheck.org/how-to-fact-check/tips-and-advice/)
• Where is the evidence? – Whenever anyone in public life makes a claim, big or

small, the first question you should ask – once you have got past whether the claim

is plausible and worth investigating – is ‘Where’s the evidence?’
• There is often a good reason for an official to refuse to reveal the evidence behind a claim they make. They may need, as journalists do, to protect their source. But if sources need protection, we still need evidence. And often the reason officials

refuse to provide it is that their evidence is weak or partial or contradictory
• If no evidence is forthcoming you know there is, or may be, a problem with the

claim
• Is the evidence verifiable? – The next step, if evidence is provided, is to see

whether it can be verified. One of the key tests made before the results of any new trial are accepted by the scientific community is to see whether the trial can be repeated by other researchers with the same or similar findings

• It should be the same in public debate. When a public figure, in any field, makes a claim they want believed, they should be asked to provide verifiable evidence. If they can’t, can you take what they say on trust?
• Is the evidence sound?
• Ø If the evidence is based on an eye-witness account, could the person know what they claim to know? Were they there? Is it credible to believe they would have access to this sort of information? Is the information first- or second-hand, something they had heard and believed? Is it something that could be known?
• Ø If there is data, when was it gathered? It is a favoured trick of public figures to present information collected many years previously, as if it were from today, and make no mention of the dates. But data ages and, unlike most wines, this is not good. To understand the data, you need to know when it was gathered and what the picture looked like before and after
• Ø Was the sample large enough? Was it comprehensive? An opinion poll that samples the views of a few dozen – or even a few hundred – people is unlikely to be representative of the views of a population of millions. Most polling organisations suggest that a well-chosen sample of around 1,000 people is the minimum required to produce accurate results. But it is surprising how often surveys of a few hundred, or few dozen, people are quoted by public figures and reported in the media as representing wider views. And remember, even large scale surveys – that do not look in all the right places – can give an inaccurate picture
• Ø How was the data collected? Is the sample representative, e.g. of all relevant social groups/genders/ages? How was the study done? Similar surveys done door-to- door can produce different results from those done on the telephone because of how the interviewee responds to someone face-to-face and on the phone. And studies that rely on the respondents filling in forms tend to show more errors, particularly among respondents with low literacy skills, than person-to-person interviews
• Ø How is the data presented? Did the person tell the truth, the whole truth and nothing but the truth? Public figures from all walks of life like to select what they tell you, and what they don’t, cherry-picking the juiciest evidence, favourable to their side in an argument, and leaving the less tasty morsels in the bowl. When a politician says, for example, that he or she put “record sums” into the public health system, and does not mention inflation, the claim may be true, in itself, but misleading if inflation means “real terms” spending is falling. So make sure to look at other factors that make up the wider picture. And always remember to keep numbers in proportion. Spending $50million on a health project may sound like a lot, for a small community. But divide it among a population, and note that the programme is set to run over 10 years and it seems a lot less generous than it seemed at first