full transcript

From the Ted Talk by Mark Liddell: How statistics can be misleading


Unscramble the Blue Letters


Statistics are persuasive. So much so that people, organizations, and whole countries base some of their most important decisions on organized data. But there's a pleobrm with that. Any set of statistics might have something lurking inside it, something that can turn the results cepeotlmly upside down. For example, imagine you need to choose between two hospitals for an elderly relative's surgery. Out of each hospital's last 1000 patient's, 900 survived at haiopstl A, while only 800 survived at Hospital B. So it looks like Hospital A is the better choice. But before you make your deioscin, remember that not all patients arrive at the hospital with the same lveel of health. And if we divide each hospital's last 1000 patients into those who arrived in good health and those who arrived in poor hleath, the picture starts to look very different. Hospital A had only 100 patients who arrived in poor health, of which 30 srvieuvd. But Hospital B had 400, and they were able to save 210. So Hospital B is the better choice for patients who arvrie at hospital in poor health, with a sauvvirl rate of 52.5%. And what if your relative's health is good when she arrives at the hospital? Strangely enough, Hospital B is still the better chcoie, with a survival rate of over 98%. So how can Hospital A have a better overall survival rate if Hospital B has better survival rates for patients in each of the two groups? What we've stumbled upon is a case of Simpson's paradox, where the same set of data can appear to show opposite trends depending on how it's grouped. This often occurs when aggregated data hides a conditional variable, sometimes known as a lurking variable, which is a hidden additional factor that sinialgtfciny influences results. Here, the hidden factor is the relative proportion of paeittns who arrive in good or poor health. Simpson's padorax isn't just a hypothetical scenario. It pops up from time to time in the real world, sometimes in iamtnport contexts. One study in the UK appeared to show that smokers had a higher survival rate than nonsmokers over a twenty-year time period. That is, until dividing the participants by age group swhoed that the nonsmokers were significantly older on agverae, and thus, more likely to die during the trial period, precisely because they were living longer in gaenerl. Here, the age groups are the lurking variable, and are vital to correctly interpret the data. In another example, an analysis of Florida's death penalty cases seemed to revael no raaicl disparity in stenienncg between black and white defendants convicted of murder. But dividing the cseas by the race of the victim told a different story. In either situation, black dneftdanes were more likely to be sentenced to death. The slightly higher overall sentencing rate for white defendants was due to the fact that cases with white vimctis were more likely to elicit a death sentence than cases where the vitcim was black, and most murders occurred between people of the same race. So how do we avoid fllniag for the paradox? Unfortunately, there's no one-size-fits-all answer. Data can be grouped and divided in any number of ways, and overall numbers may sometimes give a more accurate pcturie than data divided into misleading or arbitrary categories. All we can do is celrflauy study the ataucl saitonuits the scatiistts describe and consider whether lurking virlbaaes may be present. Otherwise, we leave ourselves vulnerable to those who would use data to manipulate others and promote their own agendas.

Open Cloze


Statistics are persuasive. So much so that people, organizations, and whole countries base some of their most important decisions on organized data. But there's a _______ with that. Any set of statistics might have something lurking inside it, something that can turn the results __________ upside down. For example, imagine you need to choose between two hospitals for an elderly relative's surgery. Out of each hospital's last 1000 patient's, 900 survived at ________ A, while only 800 survived at Hospital B. So it looks like Hospital A is the better choice. But before you make your ________, remember that not all patients arrive at the hospital with the same _____ of health. And if we divide each hospital's last 1000 patients into those who arrived in good health and those who arrived in poor ______, the picture starts to look very different. Hospital A had only 100 patients who arrived in poor health, of which 30 ________. But Hospital B had 400, and they were able to save 210. So Hospital B is the better choice for patients who ______ at hospital in poor health, with a ________ rate of 52.5%. And what if your relative's health is good when she arrives at the hospital? Strangely enough, Hospital B is still the better ______, with a survival rate of over 98%. So how can Hospital A have a better overall survival rate if Hospital B has better survival rates for patients in each of the two groups? What we've stumbled upon is a case of Simpson's paradox, where the same set of data can appear to show opposite trends depending on how it's grouped. This often occurs when aggregated data hides a conditional variable, sometimes known as a lurking variable, which is a hidden additional factor that _____________ influences results. Here, the hidden factor is the relative proportion of ________ who arrive in good or poor health. Simpson's _______ isn't just a hypothetical scenario. It pops up from time to time in the real world, sometimes in _________ contexts. One study in the UK appeared to show that smokers had a higher survival rate than nonsmokers over a twenty-year time period. That is, until dividing the participants by age group ______ that the nonsmokers were significantly older on _______, and thus, more likely to die during the trial period, precisely because they were living longer in _______. Here, the age groups are the lurking variable, and are vital to correctly interpret the data. In another example, an analysis of Florida's death penalty cases seemed to ______ no ______ disparity in __________ between black and white defendants convicted of murder. But dividing the _____ by the race of the victim told a different story. In either situation, black __________ were more likely to be sentenced to death. The slightly higher overall sentencing rate for white defendants was due to the fact that cases with white _______ were more likely to elicit a death sentence than cases where the ______ was black, and most murders occurred between people of the same race. So how do we avoid _______ for the paradox? Unfortunately, there's no one-size-fits-all answer. Data can be grouped and divided in any number of ways, and overall numbers may sometimes give a more accurate _______ than data divided into misleading or arbitrary categories. All we can do is _________ study the ______ __________ the __________ describe and consider whether lurking _________ may be present. Otherwise, we leave ourselves vulnerable to those who would use data to manipulate others and promote their own agendas.

Solution


  1. average
  2. showed
  3. survival
  4. paradox
  5. hospital
  6. completely
  7. victims
  8. situations
  9. problem
  10. survived
  11. actual
  12. health
  13. choice
  14. variables
  15. important
  16. reveal
  17. statistics
  18. significantly
  19. falling
  20. picture
  21. victim
  22. racial
  23. defendants
  24. level
  25. cases
  26. sentencing
  27. carefully
  28. decision
  29. arrive
  30. general
  31. patients

Original Text


Statistics are persuasive. So much so that people, organizations, and whole countries base some of their most important decisions on organized data. But there's a problem with that. Any set of statistics might have something lurking inside it, something that can turn the results completely upside down. For example, imagine you need to choose between two hospitals for an elderly relative's surgery. Out of each hospital's last 1000 patient's, 900 survived at Hospital A, while only 800 survived at Hospital B. So it looks like Hospital A is the better choice. But before you make your decision, remember that not all patients arrive at the hospital with the same level of health. And if we divide each hospital's last 1000 patients into those who arrived in good health and those who arrived in poor health, the picture starts to look very different. Hospital A had only 100 patients who arrived in poor health, of which 30 survived. But Hospital B had 400, and they were able to save 210. So Hospital B is the better choice for patients who arrive at hospital in poor health, with a survival rate of 52.5%. And what if your relative's health is good when she arrives at the hospital? Strangely enough, Hospital B is still the better choice, with a survival rate of over 98%. So how can Hospital A have a better overall survival rate if Hospital B has better survival rates for patients in each of the two groups? What we've stumbled upon is a case of Simpson's paradox, where the same set of data can appear to show opposite trends depending on how it's grouped. This often occurs when aggregated data hides a conditional variable, sometimes known as a lurking variable, which is a hidden additional factor that significantly influences results. Here, the hidden factor is the relative proportion of patients who arrive in good or poor health. Simpson's paradox isn't just a hypothetical scenario. It pops up from time to time in the real world, sometimes in important contexts. One study in the UK appeared to show that smokers had a higher survival rate than nonsmokers over a twenty-year time period. That is, until dividing the participants by age group showed that the nonsmokers were significantly older on average, and thus, more likely to die during the trial period, precisely because they were living longer in general. Here, the age groups are the lurking variable, and are vital to correctly interpret the data. In another example, an analysis of Florida's death penalty cases seemed to reveal no racial disparity in sentencing between black and white defendants convicted of murder. But dividing the cases by the race of the victim told a different story. In either situation, black defendants were more likely to be sentenced to death. The slightly higher overall sentencing rate for white defendants was due to the fact that cases with white victims were more likely to elicit a death sentence than cases where the victim was black, and most murders occurred between people of the same race. So how do we avoid falling for the paradox? Unfortunately, there's no one-size-fits-all answer. Data can be grouped and divided in any number of ways, and overall numbers may sometimes give a more accurate picture than data divided into misleading or arbitrary categories. All we can do is carefully study the actual situations the statistics describe and consider whether lurking variables may be present. Otherwise, we leave ourselves vulnerable to those who would use data to manipulate others and promote their own agendas.

Frequently Occurring Word Combinations


ngrams of length 2

collocation frequency
survival rate 4
white defendants 2



Important Words


  1. accurate
  2. actual
  3. additional
  4. age
  5. agendas
  6. aggregated
  7. analysis
  8. answer
  9. appeared
  10. arbitrary
  11. arrive
  12. arrived
  13. arrives
  14. average
  15. avoid
  16. base
  17. black
  18. carefully
  19. case
  20. cases
  21. categories
  22. choice
  23. choose
  24. completely
  25. conditional
  26. contexts
  27. convicted
  28. correctly
  29. countries
  30. data
  31. death
  32. decision
  33. decisions
  34. defendants
  35. depending
  36. describe
  37. die
  38. disparity
  39. divide
  40. divided
  41. dividing
  42. due
  43. elderly
  44. elicit
  45. fact
  46. factor
  47. falling
  48. general
  49. give
  50. good
  51. group
  52. grouped
  53. groups
  54. health
  55. hidden
  56. hides
  57. higher
  58. hospital
  59. hospitals
  60. hypothetical
  61. imagine
  62. important
  63. influences
  64. interpret
  65. leave
  66. level
  67. living
  68. longer
  69. lurking
  70. manipulate
  71. misleading
  72. murder
  73. murders
  74. nonsmokers
  75. number
  76. numbers
  77. occurred
  78. occurs
  79. older
  80. organizations
  81. organized
  82. paradox
  83. participants
  84. patients
  85. penalty
  86. people
  87. period
  88. persuasive
  89. picture
  90. poor
  91. pops
  92. precisely
  93. present
  94. problem
  95. promote
  96. proportion
  97. race
  98. racial
  99. rate
  100. rates
  101. real
  102. relative
  103. remember
  104. results
  105. reveal
  106. save
  107. scenario
  108. sentence
  109. sentenced
  110. sentencing
  111. set
  112. show
  113. showed
  114. significantly
  115. situation
  116. situations
  117. slightly
  118. smokers
  119. starts
  120. statistics
  121. story
  122. strangely
  123. study
  124. stumbled
  125. surgery
  126. survival
  127. survived
  128. time
  129. told
  130. trends
  131. trial
  132. turn
  133. uk
  134. upside
  135. variable
  136. variables
  137. victim
  138. victims
  139. vital
  140. vulnerable
  141. ways
  142. white
  143. world