Continuous Integration/Deployment

TLDR - Automate early

Everytime a developer changes some code, he commits it and pushes it upstream to the repository. From there, they can create a PR for other developers to review the code. After the PR is approved, the code can be merged into develop or master branches. Without continous integration and deployment, the new code has to be manually verified for integration and manually deployed. This is generally a tedious, repeated task. [Read More]

Notifications

TLDR - Why don't you have monitoring yet?

Notifications make monitoring useful. When looking at it from a passive monitoring perspective using Prometheus, Alertmanager is a great service to leverage as it can hook into Slack, PagerDuty, or even SMS. Other times you may want to know when a client registers on your SaaS product, or when a client is attempting to leave. This is known as Application Performance Monitoring. One example in my trading bot is that whenever the bot executes a trade or a stoploss is invoked, send me a slack message. [Read More]

Passive Monitoring

TLDR - Why don't you have passive monitoring yet?

Monitoring is a big topic. I can attempt to simplify it into a few categories. - Passive Infrastructure Monitoring: Track CPU, Memory, Load, Uptime, Network, etc - Log shipping: This is where your services stream their logs and you set alerts or metrics on certain patterns - Active Monitoring: Ping your servers on a timer for aliveness/health, or a full verification of an API endpoint and its response - Application Performance Monitoring: Metrics through code injection/addition to view application metrics [Read More]

Millions of Inserts | The ideal database?

TLDR - When Mongo isn't webscale

Disclaimer: I haven’t ran any benchmarks or ran any of these databases in HA mode for proof of concepts but this was my experience with trying to work with lots of data for my HFT Trading Bot. The Data: (all indexed on time) - 1 minute candle: {timestamp, instrument, high, low, open close, volume, trades}, 1 inserts per minute - Individual trade: {timestamp, instrument, price, side, type}, 0-2000 inserts per minute [Read More]

Ansible, why didn't I use this sooner

TLDR - I wish I could get back days of my life

I’ve been working with Linux since 2008. Back then, I mostly used a spare machine to run FreeNAS, Apache, MySQL, Python to host a static or Wordpress website. Then as I got more invested with projects running on Linux, I started to set up VPSes to start hosting online in 2011. Everytime, it was the same process. Login as root, set up new user, disable root, set up PKA, enable firewall… etcetc. [Read More]