MADRID – On April 28, Spanish Prime Minister Pedro Sánchez was alone on the stage of a bright but empty briefing room. As a CNN reporter asked a video question, the Prime Minister looked deep into concentration, scribbling notes and stopping to watch the monitor only once. As he launched into his response, he looked straight into the camera to brag about the Covid-19 test volume in Spain.
“We are one of the countries with the highest number of tests performed,” said Sánchez.
Initially, the Prime Minister cited data from a recent ranking by the Organization for Economic Co-operation and Development (OECD) which ranked Spain eighth among the Covid-19 tests among its members.
“Today,” he added, “we discovered another study, from Johns Hopkins University, which […] ranks us fifth in the world for the total number of tests performed. “
There were only two problems: the OECD data were incorrect. And although some sources ranked Spain fifth in total testing volume, Johns Hopkins was not one of them; the study cited by Sánchez does not exist.
However, two weeks later, the Spanish government maintains the merits of its Prime Minister’s claim. Instead of quoting Johns Hopkins, Spanish authorities are now pointing to placement tests from a data aggregation website called Worldometer – one of the sources for the university’s widely cited coronavirus dashboard – and prompting questions about why some respected governments and institutions have chosen to trust a source of which little is known.
What is the Worldometer?
Before the pandemic, Worldometer was best known for its “counters”, which provided live estimates of figures such as the world’s population or the number of cars produced this year. Its website indicates that revenues come from advertising and licensing its meters. The Covid-19 crisis has undoubtedly increased the popularity of the website. This is one of the first Google search results for coronavirus statistics. Worldometer pages have been shared about 2.5 million times in the past six months, compared to just 65 shares in the first six months of 2019, according to statistics provided by BuzzSumo, a company that tracks social media engagement and provides insight into content.
Question the reliability of this coronavirus statistics site 03:36
The website claims to be “run by an international team of developers, researchers and volunteers” and “published by a small, independent digital media company based in the United States.”
But public records show little evidence of a company that employs a multilingual team of analysts and researchers. It is unclear whether the company paid staff to verify the accuracy of its data or whether it relies solely on automation and crowdsourcing. The site has at least one job offer, from October, looking for a volunteer web developer.
Formerly known as Worldometers, the website was originally created in 2004 by Andrey Alimetov, then a 20-year-old recent immigrant from Russia who had just gotten his first computer job in New York.
“It’s a super simple website, there’s nothing crazy about it,” he recently told CNN.
In about a year, Alimetov said, the site was receiving 20,000 or 30,000 visits a day, but costing it too much money in web hosting costs.
“There was no immediate quick way to withdraw money,” he said, so he registered the site on eBay and sold it for $ 2,000 in 2005 or 2006.
When the Reddit home page introduced its old site in 2013, Alimetov emailed the buyer, a man by the name of Dario, to congratulate him.
In his answer, Dario said he bought the site to drive traffic to his other websites.
As these companies “started to decline, I decided to invest in Worldometers, bringing in resources and people until it ended up going its own way,” wrote Dario.
Worldometer no longer carries its ending “s”, except in its URL. Beyond that, little has changed.
Today, the Worldometer website belongs to a company called Dadax LLC.
Worldometer and Dadax representatives did not respond to requests for CNN interviews, but state business records show that Dadax was first trained in Delaware in 2002. The record indicates a post office box as the address of the company. From 2003 to 2015, the files filed in Connecticut and New Jersey registered the president of Dadax as Dario Pasqualino. Addresses on the files linked the business and Pasqualino to homes in Princeton, New Jersey and Greenwich, Connecticut. The company is still actively registered in Delaware and has been in good standing since 2010.
The company shares the name Dadax with a software company based in Shanghai. In March, the two companies released statements denying a connection. Chinese Dadax said it published his statement after receiving “many calls and emails” from the statistics site. Worldometer, in a tweet, said he had never had “any type of affiliation with a Chinese-based entity.”
IDs from Worldometer source code and Dadax’s US websites link them to at least two dozen other websites that appear to share ownership. Some seem to have disappeared. Others, like usalivestats.com, italiaora.org and stopthehunger.com, share the same principle: live statistics counters. Most of the sites have a rudimentary aesthetic, which recalls an Internet of the 90s or the beginning of the 2000s. Some seem quite random. An Italian site presents Christmas poems and gift suggestions, such as a bonsai (for her), or a land on the moon (for him). Another site is dedicated to Sicilian puppet shows.
A person with the name and birthday of Pasqualino is also registered as the sole owner in Italy. This company manages and sells “advertising space”, according to an Italian registration document filed last year. Its address leads to a tidy three-story apartment building on a green street in an upscale neighborhood of Bologna.
CNN could not reach Pasqualino using the contact details on Worldometer and in the public archives.
According to the Worldometer website, its Covid-19 data comes from a multilingual team that “monitors live press briefings throughout the day” and through crowdsourcing.
Visitors can report new Covid-19 numbers and data sources to the website – no name or email address required. A “team of analysts and researchers” validates the data, says the website. It may at first sound like Wikipedia from the data world, but some Wikipedia editors have decided to avoid Worldometer as the source for Covid-19 data.
“Several updates lack source, do not correspond to their cited source or contain errors,” wrote an editor, under the username MarioGom, on a talk page for Wikipedia editors working on content related to Covid-19 last month. “Some errors are small and temporary, but some are relatively large and are never corrected.”
The publisher, whose real name is Mario Gómez, told CNN in an email: “Instead of trying to use consistent criteria, [Worldometer] seems to be aiming for the highest figure. They have a system for users to report higher numbers, but so far I haven’t been able to use it to report that a certain number is wrong and should be lower. “
Edouard Mathieu, data manager for Our world in data (OWID), an independent statistics website headquartered at the University of Oxford, has experienced a similar trend.
“Their main objective seems to be to have the latest issue wherever it comes from, whether it is reliable or not, whether it is well supplied or not,” he said. “We think people should be wary, especially the media, policy makers and decision makers. These data are not as accurate as they think. “
Virginia Pitzer, an epidemiologist at Yale University, who has focused on modeling the spread of Covid-19 in the United States, said that she had never heard of Worldometer. CNN asked him to rate the reliability of the website.
“I think the Worldometer site is legitimate,” she wrote by email, explaining that many of its sources appear to be credible government websites. But it also found faults, inconsistencies and an apparent lack of expert retention. “The interpretation of the data is lacking,” she writes, explaining that she found the data on active cases “particularly problematic” because the data on recoveries are not systematically reported.
Pitzer also found few detailed explanations for the data communication issues or discrepancies. For Spain, it’s only one sentence. For many other countries, there is no explanation at all.
She also found errors. In the Spanish data, for example, Worldometer reports more than 18,000 recoveries on April 24. The Spanish government reported 3,105 recoveries that day.
Apple-to-apple accuracy, fairness and comparisons
When the Spanish Prime Minister Pedro Sánchez boasted of the high ranking of Spain, he did not get his numbers out of thin air. On April 27, the OECD incorrectly ranked Spain eighth in per capita tests. The OECD originally used OWID data to compile his statistics. But he obtained the Spanish figures independently because the OWID data was incomplete. Mixed sourcing skewed Spain’s position in the rankings because it had a broader test category than other countries’s numbers. The organization corrected the next day, two hours before Sánchez’s press conference, propelling Spain to 17th place.
In his statement, the OECD said “we regret the confusion created on a sensitive issue by any debate on methodological issues” and stressed that it is more important to increase the availability of tests in general than to know where a country ranks in particular.
Sánchez’s subsequent reference to a Johns Hopkins study, in which he said that Spain ranked fifth in tests in the world, seems to have been a case of mixed attribution by the Prime Minister. JHU has not released international test figures. Jill Rosen, a school spokesperson, told CNN that the university could not identify a report that matched Sánchez’s description.
At a press conference on May 9, Sánchez evaded a question from CNN urging him on the existence of the JHU study and listed government figures on test totals instead of. In comments made to a Spanish journalist the next day, Health Minister Salvador Illa continued to insist that test data was published by JHU, although he cited Worldometer as an under-source. underlying. Since Johns Hopkins got his data from Worldometer, he argued, it’s been just as good.
“These are data provided by John Hopkins University […] taken from as a basic source of information, the Worldometer website, “said Illa. “You can check it out.”
It is true that on April 28, Worldometer data ranked Spain fifth in terms of total test volume. At the time, OWID data also ranked Spain fifth, but as more and more countries reported higher test volumes, it was clear that Worldometer data was flawed. . His Spanish figure includes both polymerase chain reaction (PCR) tests, which show whether patients are currently infected, and antibody tests, which indicate whether patients have already been infected. For most countries outside of Spain, Worldometer data appears to include only PCR tests.
Because so few countries report data on antibody testing and to ensure comparison between apples, OWID says it only tracks PCR tests. According to this measure, as of May 17, Spain ranked sixth, behind the United States, Russia, Germany, Italy and India. The Worldometer ranked Spain fourth.
But relying on the classification according to the gross number of tests carried out is always misleading because it does not take into account the differences in population between countries.
OWID data manager Edouard Mathieu says that a much fairer way to compare test data is to take into account the size of the population. On May 10, OWID ranked Spain 19th in the tests for 1,000 people. The Worldometer placed Spain 15th in a similar measure.
Story of two classifications
Worldometer data ranks Spain fifth in terms of total test volume. But relying on raw figures is misleading because it does not take into account the differences between countries. After adjusting for population, Spain’s ranking drops to 16th. Experts say the data from Worldometer is even more flawed as their numbers in Spain have a broader test category than most other countries.
Roberto Rodríguez Aramayo, research professor at the Institute of Philosophy of the Spanish National Research Council (CSIC) and former president of a Spanish ethics association, said that Spain communicates data on the most common types of tests reliable and less reliable.
“Unfortunately, there seems to be [political] interests in the readings that are given of that data, when it is shown, “he said.
What does Worldometer have to do with Johns Hopkins University?
Johns Hopkins has not published international data on the Covid-19 tests, but he lists Worldometer among several sources for its widely cited coronavirus dashboard.
The university declined to say for which specific data points it relies on Worldometer, but problems with the data on the meter site caused at least one notable error.
On April 8, the global count of JHU-confirmed cases of Covid-19 briefly exceeded 1.5 million before falling by more than 30,000. Johns Hopkins posted later an explanation for the incident on his GitHub page. At the time, JHU told CNN that the error appeared to be caused by double counting French nursing home cases. But French officials told CNN that there had been no revision, not even data on retirement homes. The Johns Hopkins data appeared to come directly from Worldometer. The website has not listed any source for its figure.
Wikipedia editor James Heilman, assistant professor of emergency medicine at the University of British Columbia, said Wikipedia volunteers noticed persistent errors with Worldometer, but also with “a better name with a longer history of accuracy, “referring to Johns. Hopkins. “We hope they will double-check the numbers as well.”
In an article published in February, JHU said it started manually tracking Covid-19 data for its dashboard in January. When this became unsustainable, the university began extracting data from primary sources and aggregation websites. Lauren Gardner, an associate professor of engineering who heads the university’s Covid-19 dashboard, told CNN in A declaration that the university uses a “two-step anomaly detection system” to detect potential data problems. “Moderate” changes are automatically added to the dashboard but marked so that staff can recheck them in real time. Changes above a certain threshold require “a human to manually review and approve the values before posting them to the dashboard,” said Gardner.
The university’s dependence on Worldometer has surprised some academics.
Phil Beaver, a data scientist at the University of Denver, seemed at a loss for words when asked what he thought of JHU citing Worldometer.
“I’m not sure, that’s a big question, I kind of feel like Worldometer was counting on [Johns] Hopkins, “he told CNN after a long break.
Mathieu also seemed taken aback.
“I think JHU has been under a lot of pressure to update its figures,” he said. “Because of this pressure, they have been coerced or pressured into obtaining data in places where they should not be, but in general, I would expect JHU to be a fairly reliable source.”
In the university’s response to CNN, Gardner said that Worldometer was one of “dozens” of sources and that “before incorporating a new source, we validate their data by comparing it to other references.”
“We try not to use a single source for any of our data,” added Gardner. “We use reports from public health agencies and aggregation sources to cross-validate the numbers.”
The British government cited the Worldometer’s data on the deaths of Covid-19 at its daily press conferences for much of April, before moving on to the Johns Hopkins data.
“Worldometer and John Hopkins provided complete and respected data. As the situation changed, we moved from Worldometer to John Hopkins because John Hopkins relies more on official sources, “read a statement from a spokesperson for the British government.
“Contamination of public opinion”
In Spain, Sánchez’s apparent misallocation to Johns Hopkins has become a major controversy. In the parliament on Wednesday, the center-right People’s Party deputy, Cayetana Alvarez de Toledo, called the government “a party of lies against CNN and the Spanish people”.
On May 10, a spokeswoman for the Spanish Embassy in London complained to CNN about her coverage of the case.
“In April, Mr. Sánchez mentioned the analysis of statistical data carried out by Johns Hopkins University which is based on data published by Worldometer,” wrote the spokesperson in an e-mail sent to the diplomatic editor of the network just after 4 a.m.
“Even though Mr. Sánchez did not mention Worldometer as the main source in his remarks, [CNN] might have known that most of the comparisons and analyzes on Covid-19 around the world use [Worldometer’s] the tables.”
In remarks to CNN, a spokesperson for the Prime Minister’s Office acknowledged that Worldometer was counting PCR tests and antibody tests together and dismissed the critics’ call to adjust the numbers of tests for the population, calling it of “trap that the OECD and the Spanish press […] fell into “and arguing that Spain should not be compared to small countries like Malta, Luxembourg or Bahrain.
However, it is unclear why the Spanish government continues to insist that test data published by Worldometer be published by Johns Hopkins University.
His refusal to admit his attribution error comes just a month after Spanish Justice Minister Juan Carlos Campo said the government was considering changes to the law, seeking to punish the false information.
“I think it is more than justified – with the calm and tranquility necessary for any legal change – that we review our legal instruments to prevent those who contaminate public opinion in a serious and unjustified manner,” said Mr Campo .
At the time, Human Rights Watch executive director Kenneth Roth told CNN in an email: “If the justice minister suggests a penalizing speech that infects public opinion, it would be very dangerous.”
“Turning the government into a censor would undermine this public responsibility precisely when it is most needed,” warned Roth.
In her letter to CNN, the embassy spokeswoman insisted that Spain was – and still is – fifth in the world in the Covid-19 tests, attaching screenshots of the tables from the Worldometer as proof.
“The numbers speak louder than the words,” she wrote. “And unwilling to recognize the truth of reality […] is very disturbing, to say the least. “