I built my own crypto trading bot from scratch in python and it’s running in production for over a year now. In this story I will give you some insights on the structures involved, how they evolved and why I am using them.
A trading bot is something, that reads data from (possibly multiple) sources and makes trading decisions based on that data. Those trading decisions result in buy and sell orders which are send to an exchange for execution. In this story I will focus on the framework and architecture of such a bot (aka “this is not about the tradinglogic itself”). If you build the framework decoupled from the strategy itself, it’s relatively easy to replace the strategy without having to change anything else. Since trading strategies might change over time and you want to keep developing and testing new strategies continuously, this will help you to iterate much faster in your process.
I will present the structure that I developed in my bot, but I will build it from the simplest version up to the full production version, adding parts and layers with each step to help understand the big picture and reasons for each decision. Let’s start with the most simple case
Starting small and simple
In the simplest case, a trading bot reads market data from the exchange, executes the tradinglogic and then sends the resulting orders and updates back to the exchange.
It also needs to keep track of existing positions. Your open and executed orders are usually stored on the exchange, so you can query them. For the position however, the exchange only stores your net position. So if you have multiple positions open at once, or even longs and shorts open at the same time, there is no way around storing this information locally yourself.
Interlude: API connection
The most common crypto-exchange APIs have two parts: classic endpoints which return you data on request, and realtime websocket connections where you get new information pushed as soon as it’s available.
In a production-ready tradingbot you definitely want to use both. Orders and order updates are usually send to the exchange via endpoints. You could then query the endpoint periodically to get updates about those orders (mainly executions), but this will always be delayed compared to getting the events realtime as they happen.
Same for marketdata: On initial startup of the bot you probably want to get the chart-history from the exchange to work on. During runtime you then want to be informed on any tick that happens in the market (aka any pricechange or orderbook update) which again requires a realtime connection.
All the big exchanges (Binance, Bybit, Coinbase, Bitstamp, Kraken…) provide such APIs, but they all have different datastructures, namings and their own little quirks. Luckily they usually provide a good documentation and sample implementations for the most common languages.
Introducing a real architecture
To make the bot productionready I highly recommend to separate the housekeeping from the trading logic. This means introducing a framework around the trading logic that keeps track of the open positions, does sanity checks and general housekeeping. This also makes sure that the open positions are persisted in some way and the bot can “survive” a restart.
That’s actually a really important part:
Make the bot restart-resistant
In any production environment, your bot will see regular restarts. Due to server topics, updates, settings-changes etc. So it must handle restarts as if it ran without interruption. Of course you miss a few ticks and updates, but you must not loose track of the open positions, the state of your algorithms etc. And this can all be done in a generic way as part of the housekeeping.
Also sanity checks and global business rules can be implemented on a generic level. For example I defined for my bot that every position must have a SL order. So the bot can generically check if this is the case for every position, and if not, try to add one (based on the state), send a warning message (to monitoring) or close the position (as a last resort).
You better also plan for trouble, coding errors or whatnot. So if your expected state and net-position don’t match the data from the exchange, you need a plan how to handle that. In my case I decided to never increase my exposure on the exchange (don’t want to have a bot going crazy and increasing the position until margin runs out and it gets liquidated) but rather close positions in case of problems. Of course after a cooldown period to prevent race conditions etc. You see: lots of logic to make it safe for 24/7 use, which has nothing to do with the tradinglogic itself.
Since you separated the API-connection from the bot itself now, I would recommend to also introduce internal datastructures for everything you need. That includes representation of orders, positions, bars etc. That way you don’t rely on any specific exchange and it gets much easier to replace them. The only part where you should have exchange-specific code is the API-connector.
Ability to scale horizontally
Another big benefit of this split is the ability to easily add or replace the tradinglogic itself and also the exchange.
If you want to move the same strategy to another exchange, all you need to do is implement the API connector for that exchange and you are all set.
The same goes for new strategies and logics: Just implement the new logic within the defined structure of the bot, and you can easily replace/add them.
one more level for backtesting
For the last level of abstraction I would strongly recommend to move the connection handling to another level outside of the actual tradingbot. This makes sense because running the bot in a live environment needs some special handling regarding realtimedata. The amount of data and frequency of updates on the market itself varies a lot and, depending on the strategy, you probably don’t need to forward every single tick to the strategy.
For example my strategies usually run on H4. So I don’t want to forward every tick every few milliseconds, as it’s not necessary but would spam the strategy and waste resources. So I aggregate the data and trigger an update every 30 seconds.
On the other hand I want to forward every finished bar and any change in position or orderexecutions to the strategy without delay. But this leads to another problem: An order execution leads to multiple changes in the data (the execution itself, an update to the order and an update to the current net position on the exchange). And those updates might not arrive on your server at the same time, but with a delay (due to network etc. sometimes a “big” delay). You don’t want to trigger your bot-loop (with all the sanity checks) on inconsistent data due to network delays.
So basically we have an additional layer, which handles the data transfer (receiving updates and sending orders) between exchange and bot. In the simplest form the tradingbot provides a method “onTick” which is called by the LiveTrading with the updated data whenever the LiveTrading decides that this makes sense.
Now it’s also pretty easy to add a backtesting engine. This engine loads historic data and triggers exactly this “onTick” for every simulated tick. It also needs to handle the orders and updates sent by the tradingbot and simulate executions (which then again triggers the onTick) of course.
Truth be told: Simulating executions on historic data is a science on its own. You definitely need to invest time into this, trying to get it as perfect as possible (for your usecase). Because in the end, this determines the quality and reliability of your backtest results. And they will be the main thing that make you run this bot with those parameters on real money or not. So better get this right. Trust me, it gets expensive if you spot errors in the execution-simulation because live trading behaves differently than backtests. Been there, done that, lost money on it.
So here we are, this is the framework of my realworld crypto trading bot. I hope this helped you in building your own. As always: if you have any question, feel free to leave a comment.
