10 June 2017

StockML Dev Notes #1 - Rookie Mistakes

A little while ago I decided to take a more serious look at algorithmic trading. I had little to no trading experience or training, but I thought to myself that if I am to make it in trading, it would have to be leveraging my strength. So I dwelled myself in technical analysis immediately, only skimming theories on fundamental analysis.

Here’s what happens when you understand so little of a subject that you think it’s simple, you make elementary mistakes about skipping to the endgame immediately. Instead of trying to just trade algorithmically, or just trade, I wanted to tackle the pinnacle of algorithmic trading. To me, those were:

  • HFT, leveraging algorithms to outpace other players
  • ML, finding the optimal weight of technical indicators, and adapt faster than market

Mistake: trying to tackle HFT (high-frequency trading)

I think the true mistake here is trying to do high-frequency trading without knowing how to trade yet, but in pursuing HFT I decided to start writing things in Rust. It’s very seductive to do a side project in a language like Rust, and I fell for it. It had compounded impact on trying to do anything towards the project, since a lot of times I had to write the tools myself.

Seriously, what’s the point to think about scaling at this stage.

  • Where’s the vertical slice that confirms a trade can be performed?
  • Where’s the data source? Getting it at < 10ms intervals, really?

Mistake: trying to jump right in and “just apply machine learning”

On paper is sounds good right? Get some daily data for 10-20 years, use TA-Lib to compute indicators, shove them all in Tensorflow and call it a day. I can picture generating heuristics and back-testing them already, although I haven’t quite figured out how back-testing is done yet.

Honestly it’s still tempting today.

My mistakes were more trivial though, since I had to build TA libraries for Rust to even start. And I got to deal with all these nice things:

  • Floating point storage in database
  • Butterfly effect on computed indicators
  • Caching what values to computer different indicators
  • Separating indicators and cached values to faster fetch
  • If the math were correct

Basically I got way ahead of myself. The good thing is I had to read up so much on technical indicators and the math behinds them.

Comments