Folha: Who is more popular, Taylor or Selena?
Reproduction of Marcelo Viana's column in Folha de S. Paulo.
Of the 400 people interviewed after Taylor Swift 's concert, 290 really enjoyed it. Of the 400 people surveyed after Selena Gomez 's concert, only 261 were satisfied. The research mathematically proves it: Taylor is more popular than Selena (among my daughter's friends, I didn't even need to ask!). Right?
But things are more subtle. At Taylor's show, 300 respondents were adults, and of those, 240 liked it (80%). At Selena's show, 100 adults were interviewed, of whom 90 liked it (90%). Therefore, among adults, Selena wins. So far, so good.
On the other hand, at Taylor's show, 100 of the respondents were teenagers, and of those, 50 declared themselves satisfied (50%). Selena, however, pleased 171 of the 300 teenagers interviewed (57%). Therefore, she also wins among teenagers…
How is it possible that Selena wins in each of the two audience categories and yet loses to Taylor in the overall audience?
In fact, this phenomenon is quite common in data analysis and even has a name: Simpson's paradox , in honor of the British statistician Edward Simpson (1922–2019) who published a paper on this topic in 1951. But the paradox had been discovered half a century earlier by his compatriots Karl Pearson (1857–1936) and Udny Yule (1871–1951).
They found that a trend that manifests itself in several data groups can disappear, or even be reversed, when these groups are combined. For this reason, the phenomenon is also called the reversal paradox, or the amalgamation paradox.
A famous case occurred at the University of Berkeley's graduate school in 1973. That year, the success rate for male applicants (45%) was significantly higher than that for women (only 33%). Fearing a discrimination lawsuit, the university asked the statistics department to study the issue.
What they noticed was that, when analyzing the numbers for each department separately, the situation was completely different. In fact, in two departments the numbers were balanced between the two genders, while in the other four there was a significant bias in favor of women!