Measuring news bias
News bias has expanded many avenues of scientific research in social as well as computational sciences. Methods used in social sciences often include a process called coding, where news articles are selected and analyzed manually, or theoretical frameworks such as discourse analysis and content analysis. Studies of bias in the computational field, on the other hand, concentrate on methods of pattern analysis (Ali et al., 2010) and annotate large volumes of text data to detect bias patterns. This shows the study of news bias is important to both the media and computer science (RENOIR Project 2017).
Ukraine-Russia issue reflected by different media outlets
The experiment was originally conducted by the XLike research team using Event Registry’s API. We tried to duplicate the experiment.
To identify the relevant articles, we used the following criteria:
Articles have to
- mention Russia and Ukraine
- have to be published in the time period between Jan 15th, 2014 and June 1st, 2014, and
- have to be categorized into the Business or Society category (or any of their subcategories). This criterion was used to remove the articles related to the Olympic games and other non-related topics.
By inserting the minus sign (-) before the selected news source it is coloured in grey. In the example below this means that we will get the news articles from all the other news sources except The Moscow Times. (The minus sign works, in the same way, to define other search criteria as well).
As we can see from the tag clouds above, The Moscow Time used more often words like West, Obama and foreigners when reporting about Ukraine and Russia, in comparison with the other news source which used more often words like protesters, Crimea, sanctions…
In the research project conducted by the XLike team, the aim was to understand the differences in reporting by The Moscow Times and other non-Russian news publishers. They extracted some top keywords that separate the Russian publisher from the other publishers which you can see in the table below.
Research methods for detecting news bias by using Event Registry’s API
We showcased just one of the examples how to detect news reporting bias by using Event Registry. By downloading news articles using our API and doing the analysis offline, there are many other options how to measure media bias. You can, for instance, detect agency citation, topic coverage, geographical bias and others. Some of these are described more in details in the XLike and RENOIR papers in the references. As an example, we show the sieve diagram displaying the bias that publishers have in citing the news agencies. For this image, the researchers analyzed 1.3 million news articles and identified which agencies were being cited in them. The graph shows that the US-based news sources (Washington Post, FOX News and Time) are significantly more frequently citing AP as the source, while Reuters is commonly cited by non-US based news sources (and also the Time). Interesting trends can also be seen by other pairs of publishers and news wires.
- Ali, O., Flaounas, I., De Bie, T., et al. (2010). “Automating News Content Analysis: An Application to Gender Bias and Readability” JMLR: Workshop and Conference.
- (2017). Deliverable 2.1: Cross-lingual news reporting bias. Project: Reverse Engineering of social information processing (RENOIR).
- Leban, G., et al. (2014). “D5.1.1.News reporting bias detection prototype “. XLike–Cross-Lingual Knowledge Extraction.
* Research papers which are not available online can be obtained from their authors.