Discussion

Stay connected with your statistics peers with this quarter-long book discussion!

This quarter WIST will be sponsoring a quarter-long book discussion. Participants will read one or two chapters each week and then have a discussion. Discussion sections will take place once a week on Friday’s at 1:00pm CT using Zoom. For spring quarter 2020, WIST will be reading Weapons of Math Destruction by Cathy O’Neil.

Meeting Information

Schedule

Discussion Topic Date Time
Introduction and Chapter 1 Friday, April 10th 1:00 - 1:30pm CT
Chapter 2 Friday, April 17th 1:00 - 1:30pm CT
Chapter 3 Friday, April 24th 1:00 - 1:30pm CT
Chapter 4 Friday, May 1st 1:00 - 1:30pm CT
Chapter 5 Friday, May 8th 1:00 - 1:30pm CT
Chapter 6 Friday, May 15th 1:00 - 1:30pm CT
Chapter 7 Friday, May 29th 1:00 - 1:30pm CT
Chapter 8 Friday, June 19th 1:00 - 1:30pm CT
Chapter 9 Friday, June 26th 1:00 - 1:30pm CT
Chapter 10 Friday, July 3rd 1:00 - 1:30pm CT
Conclusion Friday, July 10th 1:00 - 1:30pm CT

Link

The Zoom link for the meeting will be emailed out to the WIST listserv the day before a given discussion.

Add yourself to the WIST listserv (WISTNU) to recieve updates about this event and future WIST updates.
Directions for subscribing to the WIST listserv

Please email Martha, mareichler@u.northwestern.edu, if you have questions or need more information.

Book


Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Cathy O’Neil

Description from the Cover

A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life and threaten to rip apart our social fabric.

We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated.

But as Cathy O’Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data.

Tracing the arc of a person’s life, O’Neil exposes the black box models that shape our future, both as individuals and as a society. These “weapons of math destruction” score teachers and students, sort resumes, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health.

O’Neil calls on modelers to take more responsibility for their algorithms and on policy makers to regulate their use. But in the end, it’s up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change.

Get the Book

Get at Bookends and Beginnings, located in Evanston
Paperback
Hardcover

Other Options
Buy at thriftbooks
Buy at Barnes & Nobel
Borrow from the Chicago Library

Discussion Questions

If you have other questions or topics you would like to discuss for a given secession, please email Abby Smith, als1@u.northwestern.edu, by the Wednesday before the discussion.

Introduction and Chapter 1

  1. O’Neil defines WMD’s by opacity, scale, and damage. Do you think these are appropriate qualifications? Should there be any others?
  2. O’Neil writes “Like gods, these mathematical models were opaque, their workings invisible to all but the highest priest in their domain: mathematicians and computer scientists” (Introduction, p 3). Do you think the comparison to religion is appropriate? How as the role of mathematicians and computer scientists in culture and society changed over the past decade?
  3. Is there a way to change the incentive structure such that companies want to use data fairly? WMDs survive because they offer a black-box scapegoat tool for boosting profits. The profits are a visible, measurable incentive. So how do we advertise and measure the advantages of frequently-tested, feedback-driven, fair algorithms? How will this affect statistical consulting firms (like Mathematica in O’Neil’s example)?

Chapter 2

  1. Assumptions are essential to building a good model. O’Neil says that finance models she saw were based on the past on the assumption that patterns will repeat. Other common assumptions in finance are that the market is general efficient and that investors act rationally. Discuss the strengths and weaknesses of these assumptions.
    Learn more about the rational investors assumption in this Nova episode: Mind over Money.1
  2. After going through the 2008 financial recession, O’Neil says “I was forced to confront the ugly truth: people had deliberately wielded formulas to impress rather than clarify” (Ch 2, p 44). Do you think this is true in industry? What about in academia?

  3. O’Neil points out that jobs in finance and Big Tech recruit from top schools and pay employees large salaries. “This leads to the fallacious conclusion that whatever they’re doing to bring in more money is good. It ‘adds value.’ Otherwise, why would the market reward it?” (Ch 2, p 47). She refers to this as “economic Darwinism.” Does society appropriately value statistical work?

Chapter 3

  1. A huge issue with the US Today Rankings WMD is the use of proxies, especially those relating to money. Are their “better” proxies that could be used in a quantifiable way?
  2. Given the new climate (bad economy, increased inequity), how do you think schools will respond to demands for affordability and diversity at their school? How are students responding to the pressure to perform in a quantifiable way?
  3. How do we encourage colleges to keep programs - Liberal arts, teaching, social work, division three sports – that are valuable but not quantitatively valuable in high salaries or large donations?
  4. O’Neil writes that “All of them, from the rich to the working class, are simply being trained to fit into an enormous machine – to satisfy a WMD” (Ch 3, p 65). How can why resist being put into WMD’s? Is it possible on an individual level?

Chapter 4

  1. Advertising today is driven by the mass amounts of personal data collected from various sources. Do you feel comfortable using “free services” (i.e. Facebook, Google) where payment is your data? If not, are you willing to pay for services instead?
  2. Do you think personalized advertising is useful in online retail? Do you feel comfortable giving away personal data to receive a personalized experienced? (i.e. Amazon)
  3. An algorithm itself is neither good nor bad – it is how it is written and implemented. Is there a way to monetize public health marketing for companies? (i.e. Amazon using its algorithm to find those vulnerable and market them appropriate services such as mental health care, unemployment benefits, and other resources).
  4. Are there any benefits from for-profit colleges? Should the federal government stop allowing student loads for these institutions?

Interested in learning more about for-profit colleges? Check out Lower Ed by Tressie McMillan Cottom2 , recommended by Abby Smith. Also check out Frontline’s Subprime Education from 2010.

Chapter 5

  1. O’Neil talks about PredPol, a crime prediction software, and even though the founder stressed that “the model is blind to race and ethnicity” (Ch2, p 86), it’s use focused policing efforts on poor people of color. Often, we want to encourage people and organizations to take a data-based approach to policies, but sometimes taking a ‘data-based’ approach can create a WMD. How do we inform the public about what is and is not appropriate when it comes to creating models? How do we encourage using data as a resource while also having valid quality checks for negative feedback loops?
  2. Discuss the trade-off between fairness and efficacy/efficiency.
    Discuss the trade-off between safety and privacy.

Chapter 6

  1. Should personality tests be used in any capacity in a work environment? Why or why not?
    • “… if we define the goal as a happier worker, personality tests might end up being a useful tool.” (Ch 6, p 109)
    • “The primary purpose of the test is not to find the best employee. It’s to exclude as many people as possible as cheaply as possible” (Ch 6, p 109)
    • “It’s doing its job – even if it misses out on potential stars” (Ch 6, p 111)
  2. In finance and consulting, interviewees are asks a variety of questions to gauge their ability for critical thinking on the spot as well as their knowledge of the field. Many programing and data science jobs require some sort of take-home coding assessment or project. These ‘tests’ are not done blindly – they are done after a resume is picked from the pile and the individual is known. Is it possible to implement some form of a ‘blind audition’ for these positions? What about for the majority of jobs?

Chapter 7

  1. People in jobs that use scheduling software may go to a for-profit school as the courses may be easier for them to work into their chaotic, last-minute schedule. Ideally, traditional schools would offer more flexible, possible online, courses AND employers would release scheduling earlier. If we had to focus on making one of the changes, which would it be? Which one is better for long-term success? Which one is more likely to change?
  2. How can we place trust in government trust when huge errors are made, like not recognizing the Simpson’s Paradox in Nation at Risk? Last week an analyst at Florida’s Department of Health alleges she was fired after expressing concerns about altering covid19 to support reopening the state. How can we try to rebuild trust between the general population and statistics? Should we work on building trust in private organizations or trying to rebuild trust in governmental organizations.

Chapter 8

  1. When people are the product rather than the consumer, there is little incentive for them to fix errors (like on credit reports). How can we change the situation so they are incentivized to correct errors? Does the solution require new laws or regulations? What about putting more funding into research regarding record linkage to ensure greater accuracy?
  2. In Chapter 4, O’Neil talks about how many online services take your data and use it for advertising purposes. In this chapter, she discusses on this data is also used for e-scores. With the knowledge that your data is not only being used for advertising but also e-scores that could have a greater impact of your life do you still feel comfortable using “free services” where your data is the payment? If not, are you willing to pay for these services instead?

Chapter 9

  1. A common issue with insurance is confusing causation with correlation (i.e. people of color have lower life expectancy and saying that is due to race rather than a product of poverty and injustice). This is a basic statistical topic that is (or at least should be) taught in intro-level classes. Why do you think it is so difficult for people to understand the difference between causation and correlation? How can we improve this in science and society?
  2. As more data are available, insurers are able to put people into smaller and smaller groups with very targeted pricing. Discuss the following issues and what you would do to alleviate the negative impact:
    • Privacy: use of personal data (driving data, BMI, cholesterol and other health data)
    • Transparency: people often don’t know what ‘group’ they are in and how that affect pricing
    • Choice: requirement to participate in programs (tracking driving data or wellness programs)

Chapter 10

  1. Many social media companies want to appear as a neutral platform that everyone can use (e.g. Facebook, YouTube, Twitter, Reddit). What responsibility do social media companies have to regulate content? Does having and enforcing rules about content end their political neutrality?
  2. The way the electoral college is set up, political campaigns have an incentive to only focus on a small amount of swing voters in specific districts in swing states. Do you think this is a good enough reason to have reform? How would you change the presidential election so candidates are encouraged to campaign to all people?

Conclusion

  1. Did you learn anything new while reading the book? What was your reaction to this new information?
  2. When someone says ‘Weapons of Math Destruction’ what is the first thing that comes to your mind?
  3. Are there any actions you have taken or will take in response to what you have learned?

Thank you so much for everyone who has participated in the book club!

  1. This episode is available with PBS Passport, available to PBS donors in the U.S. that contribute $60 annually or more or $5 monthly.↩︎

  2. Available for purchase at Bookends and Beginnings↩︎