The Pragmatic Engineer newsletter is back from their summer break. They resume with a detailed deepdive about the trading industry, and interesting engineering challenges that come when working at a company that has no external customers, but where a single, unfortunate enough software bug could wipe out the whole company.
In tech recruitment, proprietary trading companies have a particularly high bar and typically offer compensation on a par with, or even exceeding, Big Tech ; right at the top of the market. That’s because for these market makers, success is all about gaining a competitive edge over rivals. Such competitive advantages today includes software that is superior to that at their competitors.
Software engineers tend to know little about trading companies – and this piece aims to change that. Trading companies build bespoke hardware stacks and have larger platform engineering teams than most workplaces. For software engineers, it’s a lucrative niche in terms of compensation, full-stack (hardware to software) work and for engineering challenges, and so the Pragmatic Engineer newsletter decided to go deeper in this interesting area.
In order to find out more, The Pragmatic Engineer sat down with a leading proprietary trading firm, Optiver. Headquartered in Amsterdam, they also have a large engineering presence in the US and globally. They met engineers and engineering leaders to learn in depth how engineering works in a modern trading business, with contributions from :
- Alex Itkin : CTO, Optiver US
- Pat Cooney : Head of Global Platform Engineering
- David Gross : Technology Lead, Options
Thanks to everyone at Optiver for taking part in this report which lifts the lid on how software engineering is done when even nanoseconds can count. In this article, the Pragmatic Engineer newsletter look into a software engineering environment that’s distinct from what you expect at most startups and Big Tech. For example :
A) No external customers. Usually, companies have consumer customers (B2C), business customers (B2B), or both. But not trading houses like Optiver, where their own business is the customer. This is a different reality : there’s no external deadlines and related pressures, but personal motivation to improve is highly valued.
B) Latency : “enemy number one”. Nearly every major engineering decision at Optiver is made in the interest of minimizing latency – the amount of time between a request and response. This approach is present across the software stack and in kernel-level work. It’s why Optiver manufactures its own hardware.
Today, latency is the floor, and AI models are becoming a differentiator. Gone are the days of having lower latency than the competition allowing for arbitrage opportunities to make risk-free profits. Instead, information models are becoming a differentiator : slow models with a fast trigger sending signals to execute trades, and fast models running at the edge of the network making trade decisions realtime.
C) Haunted by a bug that nearly killed a business. Among trading houses, there’s a cautionary tale of when a peer company, Knight Capital, nearly went bankrupt after a single bug in a high-frequency trading system triggered a $440M loss.
D) Different incentives. The business is incentivized to move very fast, but with a high premium on caution in order to avert potential financial disasters on the market. This cautious attitude to risk in concert with chasing speed feels pretty distinct in tech.
So according to the Pragmatic Engineer newsletter here are some daily activities of a trading company like Optiver for example :
- Overview of trading & hedge funds. Categories of trading companies, high-frequency trading (HFT), plenty of ML & math, and AI labs poaching HFT talent
- Engineering organization. How trading-specific roles work together, platform engineering, the “build and own” culture, and more.
- Software tech stack. The three-layer tech stack, languages and tools, CI/CD stack and the data layer.
- Hardware engineering, FPGAs and Silicon. Latency progression, custom FPGAs, custom hardware, AMD hardware partnership, and more.
- Network & physical infrastructure. Physical infrastructure, dedicated fiber & wavelength leasing, optical cable, radio, data centers & co-locations, and why AI models matter more than ever before.
- Engineering practices. Risk vs speed, knowledge-sharing culture, testing culture, monitoring & incident detection, risk management.
- AI at Optiver. AI tooling stack, future of agentic coding, details about adoption, and how it all looks in practice.
- Hiring, career development & culture. Engineering levels at Optiver, going from hiring mostly juniors to hiring experienced engineers today, competition during hiring, and the onboarding feedback loop.
Overview of trading & hedge funds
Here’s a summary of the world of ‘prop shops’ ; another name for firms like Optiver that invest their own funds in trading financial assets. Below are some useful mental models for understanding the sector.
How trading operates
Buy side/sell side
Buy side : companies invest money and earn returns. Examples : hedge funds, asset managers, pension funds.
Sell side : firms sell services or products such as advice, underwriting, research, execution, etc. These are usually investment banks and broker-dealers.
Optiver is on the “buy side”, as a prop shop.
Based on whose money is being traded, there are three main capital sources :
Investment banks serve corporate and institutional clients by raising capital, advising on deals, and executing trades on their behalf. Examples : Goldman Sachs, JPMorgan, Morgan Stanley.
Hedge funds raise money from external investors and trade it on their behalf, charging management & performance fees. Examples : Citadel, Millennium, Two Sigma, Bridgewater.
Proprietary trading firms trade only their own capital, with no clients or external funding. Examples : Optiver, Jane Street, Jump Trading, DRW, Hudson River Trading.
How roles work together
At Optiver, there are three main areas for tech roles :
Engineering : build and own the full trading-platform stack
Research : quantitative scientists who build models and predictive signals to create and improve trading algorithms. Typically, their background is in math, physics, economics, and statistics
Trading : quantitative traders who watch live markets, adjust trading system parameters in response to conditions, and build tools to automate decisions
To conclude, In reality the boundaries between these areas are porous. Yes, people do the job they were hired for, but it’s common to also see researchers roll up their sleeves and take part in implementing a trading strategy, or software engineers conducting research.
