Tuesday, November 22, 2016

How Big Data got it wrong this Election

In my previous post, I discussed how big data analytics has become a new frontier in the world of political elections. Data analytics firms hired by the Hillary Clinton for America campaign predicted almost unanimously that she and the democratic party would have a landslide victory over Donald Trump. Yet here we are. Despite everyone's expectations and predictions, we now have one of the most conservative congresses we’ve seen in decades and a Donald Trump president-elect. So what happened? A question that many shocked liberals have been asking themselves since the results came in on Tuesday, November 8, 2016. How did big data get it so wrong? The article opens with what we knew going into this election. Hillary Clinton’s campaign was arguably the most data-driven campaign in history, having hired dozens of data scientists who ran nearly six times as many simulations a day as the Obama administration which during its time was also the most data-driven campaign to date. After the results of the election, many have turned to these firms and have begun speculation on how they messed up along the way. One theory is based on the notion that when utilizing big data, all of the statistical models you are running are based on certain assumptions that you make about the population and the data that you are analyzing. Complacency and misinformation during this process can result in very biased results which may have played a part in mitigating the edge that big data gave Obama’s campaign. Trump on the other hand only hired one  data consulting firm relatively late in his campaign choosing instead to focus his efforts on his Twitter account. This decision ended up giving him billions of dollars of free marketing when large media outlets aired whatever shocking thing Trump said on his Twitter handle. Trump's campaign also focused on other social media platforms, Facebook in particular. It has been estimated that nearly half of adults now receive their news exclusively from Facebook. Facebooks algorithms are designed to show users content that they have already been shown to like in order to maximize engagement on the site. Trump's campaign essentially tailored the content they were putting out in order to maximize this engagement factor. “Trump and his camp essentially hacked Facebook’s algorithm,” says Matthew Hindma of George Washington University. So it seems as though big data shouldn’t be seen as the silver bullet as it has been played up to be leading up to this election. At the end of the day, there are factors and considerations that are put into these algorithms that leave room for biases. I am curious to see how the impact of these “false positives” will be felt in the political realm of big data analytics. While still unarguably a very powerful tool to utilize, I believe people will begin backing away from the na├»ve notion that it is an end all be all solution, as this kind of mindset breeds complacency and ignorance.

Source: http://www.economist.com/news/united-states/21710614-fake-news-big-data-post-mortem-under-way-role-technology


  1. This comment has been removed by the author.

  2. Ryan has picked a topic that many people rarely questions. The matter of fact whether if big data can truly determine such trivial things, like the 2016 Election, can be proven by how false the actual results turned out to be. Big data predicted Hillary Clinton would have an overwhelming victory over Donald Trump. However, the election results proves big data cannot be that reliable.

    Like Ryan said, the analysts were too complacent, so they couldn’t anticipate this upset. Personally, I would ideally like to view big data as the most reliable source we have because we don’t have anything else that is more important to companies, governments, researchers, and business analysts. Here are some suggestions to improve the use of big data.

    First off, you should improve data collection. Someone might think, “That’s going to be a ton of data, right?” Yes, but to improve data collection also means categorizing various types of data that may be relevant to firms. For instance, DSW should observe and analyze the behaviors of shoppers. If analysts weren’t as complacent, they could have dedicated more time to observing voter behavior more closely instead of dismissing so brashly. Secondly, improving data collection can also assist the validity of big data. Obviously, you need an effective and efficient method to store and manage data. This goes on hand to hand with data collection in that it will improve a firm’s and specialist efficiency to analyze.

    Third and fourth, you should clean and normalize data whenever you can. Cleaning data is very important because dirty data can obscure your analysis. Cleaning up your data can ensure the data will be high quality. An effective firm should collect data from multiple sources. However, these data sources sometimes contain errors and inconsistencies, which isn’t the firm’s fault, but the source of it. Nonetheless, creating a set standard for the data collected should help eliminate the inconsistencies and errors.

    Next, big data should be integrated across departments. Having multiple platforms may be more convenient. However, there is a problem. It will be harder to integrate multiple platforms and things can get hairy. To counter this dilemma, you should have a single platform linked by departmental data, so you have better and more accurate data. Finally, you can segment the data for analysis. Dealing chunks of data can be overwhelming. Breaking down these chunks can not only enhance the accuracy of the data, but also focus on aspects, such as certain behaviors (purchasing and voting). You can also classify the data into more specific categorizations.

    Big data is important to everyone. The whole point of this lesson… Make sure your data is accurate and easy to handle. Brushing it off will only hurt you in the end. New York Times Upshot, one of the vote forecasters, didn’t attention to the details needed to analyze the trends and behaviors of voters. I won’t be surprised if their integrity will be questioned for the next elections



Note: Only a member of this blog may post a comment.