Thursday, September 15, 2016

Wall Street Rolls Up It's Sleeves and Gets Dirty: Data Mining in the Investment Industry

The Wall Street Journal published an article titled “Wall Street’s Insatiable Lust: Data, Data, Data”.  This article focused on the data mining and pattern identification practices that financial services and investment management firms utilize in order to make effective investment decisions.

Presently, the “quest for yield” is a common expedition for asset managers of all sizes—large enterprises and boutique firms—and of all investment strategies—growth, value, and long/short hedging practices; client demand for beating the markets, particularly when speaking of institutional clients, is strong, and managers are in need of any sort of edge they can get in order to maximize returns.  Bloomberg and FactSet, while requisite, are no longer enough for research analysts to gain an edge on the competition.

“Data hunters” search for, and utilization of, alternative and indirect sources of data can be time-consuming and is not a practice of instant gratification—combing through large sets of abstract data is laborious and can leave a “hunter” with only a small amount of telling information.  But when taken into an aggregate of all data analyzed, these small ( and dare I say, nonmaterial) conclusions can add up to something very significant and rewarding.  For those folks who are versed in equity research and valuation, the CFA Institute calls this concept the “Mosaic Theory”.

But while investors are demanding yield, and while crunching abstract data sounds like a (more) surefire way to achieve strong returns, these practices are not feasible, nor wise, for all investors.  In fact, I would argue that the practices described in the article are best reserved only for firms like the hedge funds it mentions. 

My rational behind this sentiment is twofold; for one, “data hunting” or data mining is a costly venture, both in terms of time and money.  Firms must attract employees who have great experience and knowledge in handling complex data sets and applying advanced statistical concepts to them in order to extract useful information (if any exists).  These types of people—“quants”, mathematicians, or data scientists—don’t necessarily come cheaply when it comes to salary negotiation.  Secondly, some of the data that is utilized in these practices is highly subjective, and the application of the extracted information in investment decisions can just as easily lead to great capital loss as it can lead to capital appreciation.  Hedge funds are inherently risk-loving operations with a preexisting affinity for quantitative analysis, already possess the talent to engage in these practices, and are more accustomed and familiar with tremendous capital loss.

So in closing, I think the greatest part about data mining’s presence within the investment world is more of what it signals about the evolving thought processing of investors; analysts are becoming more abstract in reasoning, and they are looking beyond the income statement and balance sheet for answers.  But just as its taught in microeconomic principles, the opportunity cost for pursuing those answers is just too high for too many people. 

Article Source:


  1. This comment has been removed by the author.

  2. This comment has been removed by the author.

  3. I think Taylor’s article is extremely well written and gives great insight into the way data is bringing seemingly different industries together. For someone who hopes to work in tech after graduation, this was an interesting read for me to understand how bankers and analysts are increasingly looking to the troves of data they have access to for investment advice. When I think of Wall Street, I think of bankers in fancy suits and energetic traders running around trading floors taking risks in the hopes of a big payoff, but I never imagined there was a place for “Data Hunters” and high level mathematicians who could have a major impact on the gains or losses the aforementioned group has. The other aspect of this particular part of the industry I was unaware of is how much these analysts cost to employ. I ran a quick google search for “Quantitative Analyst salaries” and found that they can make anywhere between $100,000-$160,000 (link below) a year before bonuses. These figures were shocking initially, but then I thought about how much of an impact they can have on the decisions the firm makes and I began to see the picture more clearly.

    After considering the impact and cost it carries, I understand where Taylor is coming form when he argues that only hedge funds, and firms that manage billions of dollars can afford to implement these practices. He alludes to the immense skill that is needed when working in these high stress, high risk environments, and it is clear that there is little room, and little sense to utilize assets like this in an average investors position.

    As I said before, I think it’s exciting that tech (data) and finance are becoming so intertwined on the research and execution side, and that investors are moving away from the ridged way they appear to have been operating in the past. I can see data mining becoming a massive part of the financial systems infrastructure, especially as we innovate and develop new ways to understand and analyze data.

    Link to salary data:,20.htm

  4. This comment has been removed by the author.

  5. After reading Taylor’s argument, I never realized data mining (data hunting) had some side effects for investment firms. Collecting data is one thing, but hiring data scientists to analyze it is a whole different breed. For instance, he argues successfully that hiring data scientists is first off expensive and may not be as reliable. When they analyze the information they have gathered, the data scientists’ work may be subjective. Whenever you gather information, you should definitely be objective as possible. So do we need data scientists at all? The answer is still yes. Although it is relatively expensive for employers to take the time to interview and hire potential candidates, data hunting allows the ideas to be exchanged. For instance, one data scientist may have an assumption while the 2nd data scientist may have a different one.

    After browsing through Google, I found an interesting video about an investment hedge fund firm called GovBrain. In short, the video sums up that global hedge funds have limited knowledge on government and politics. Here’s where GovBrain comes in… GovBrain can help hedge fund firms by linking political events and government information to individual stocks, commodities, and bonds. In addition, GovBrain can give hedge fund firms the competitive advantage over other firms by using information that increases security prices. Perhaps, these firms from Wall Street should consider in using GovBrain because it may save time and resources in training and hiring data scientists.

    Youtube Link to GovBrain


Note: Only a member of this blog may post a comment.