This Interview is with a data scientist named Cathy O'Neil and she is discussing her new book, Weapons of Math Destruction. In the interview she discusses the drawbacks of utilizing big data analytics and if we are putting too much faith in these systems.
I was very skeptical at first as to how much of a bias Cathy had due to rhetoric that she used in the interview, like how society is now living in a “techno-utopia” and that this threatens democracy. After listening further, however, I concluded that she was actually very fair about approaching the positive impact that big data analytics has had and will continue to have on our daily lives. I agree with the notion that data analysis is simply a tool that can have a wide range of consequences based on how it is utilized. A tool by definition is an implement which assists in carrying out a specific function. By its very nature, big data analytics is an extension of a human’s ability to solve a given problem. The issue comes when people utilize these tools without “understanding what’s under the hood” as Cathy puts it. What I believe she is getting at is that the issue is not that there is a belief that these systems can do no harm, that concept is irrational as a man can do both harm and good with any tool even one as simple as a shovel; it is more a function of if there is a complete understanding of the problem that is being approached and more importantly what the unintended consequences are to the solution.
In my experience, when solving any given problem, people often look for a solution through a narrow scope. It can be extremely difficult to sort through all of the implications that come with any given solution. I can therefore understand how easy it is to put trust in these systems as they are able to process information at a volume and scale that is simply impossible for a human brain to fathom. It can become easy to utilize these systems to an extent where you are no longer reflecting on the process itself and the pitfalls of the approach. Cathy illustrates this point with a hypothetical engineering firm which decides to build a new hiring process for engineers. They utilize historical data of how successful new hires have been in the past. This system concludes that the ideal candidate is male, as women have not achieved as much as male counterparts at this firm. While this is a very specific example, it points to a much larger problem that comes from human complacency in the utilization of big data analytics. This firm will continue to hire men based on this system instead of reflecting on why women have not been as successful in the firm in the first place. To avoid this, these systems must be implemented as part of a much larger holistic solution.