FLASH FRIDAY: The Evolution of Data Management in Trading

FLASH FRIDAY is a weekly content series looking at the past, present and future of capital markets trading and technology. FLASH FRIDAY is sponsored by Instinet, a Nomura Company.

It is said that data is the lifeblood of a trading desk. Sell-side firms in particular need to effectively manage the torrents of data flowing across the desk, as the brokers who extract the most insight out of data will be best positioned to meet the needs of their buy-side customers.  

In step with the evolution of markets to mostly (or at least more) electronic, with rapid-fire trading and high volume, data management has evolved to a level of sophistication that years ago was probably hard to imagine. Computers are high-horsepower, data-collation processes are automated, and emerging technologies such as artificial intelligence are increasingly deployed. 

Data has always been a valuable commodity in trading, dating back to old black and white photos of people studying stock ticker tapes. Two decades ago, it wasn’t quite that archaic, but a concern was about which manual processes should be used to collect data from telephone trades.  

From a January 1999 Traders Magazine feature article, “Making More Money With Perishable Data”:

“Information sent by the buy side, usually via the old-fashioned telephone, will perish soon after it arrives at a sell-side firm. That’s unless the same data can be captured electronically and transmitted internally by sell-side desks.

The data includes the ubiquitous “bids wanted” and the natural orders that are the foundation of indications of interest (IOIs) — data that could sometimes be used more efficiently given today’s surging trade volume and the stock market’s bouts of volatility. 

Facing the prospect that potential buy and sell orders could escape the attention of sales traders handling the other side of this business, many desks are scrambling to develop systems that capture, store and disseminate critical pre-trade data. 

Hambrecht & Quist’s 40 equity-trading professionals in New York, San Francisco, and Boston, for instance, enter the natural orders from their buy-side clients into an in-house system. That system in turn sends the data over the firm’s wide area network — supported by dedicated T-1 communication lines — allowing individual traders to find the other side of potential trades, the sending traders’ names and pertinent comments. All three locations are live on the system.

In hindsight, it all seems logical.

“Without a good system,” said Steven Smith, head of application development at San Francisco-based Hambrecht & Quist, “you can get bit [for poor] customer service, and on the revenues side [you can also be hurt] by failing to let a customer know of a trading opportunity.” 

…Hambrecht & Quist is one of several trading desks that are building their own applications rather than using an off-the-shelf package. Smith opted for a proprietary system, claiming that vendor systems don’t provide enough flexibility in tailoring software to users’ needs.”


Data science in trading has come a long way since then, and no doubt has a lot farther to advance.