Let me introduce you to this amazing service called zeit.co AKA ZEIT NOW. I currently use it with another free online service called MicroBadger, a service I use to trigger custom Dockerbuilds after a base image such as alpine has been updated. This allows all my personal projects to be on the latest base image potentially improving performance and security. Microservice sends a POST request to a function hosted on ZEIT NOW, and that function runs a bunch of tasks. [Read More]

Everytime a developer changes some code, he commits it and pushes it upstream to the repository. From there, they can create a PR for other developers to review the code. After the PR is approved, the code can be merged into develop or master branches. Without continous integration and deployment, the new code has to be manually verified for integration and manually deployed. This is generally a tedious, repeated task. [Read More]

Notifications make monitoring useful. When looking at it from a passive monitoring perspective using Prometheus, Alertmanager is a great service to leverage as it can hook into Slack, PagerDuty, or even SMS. Other times you may want to know when a client registers on your SaaS product, or when a client is attempting to leave. This is known as Application Performance Monitoring. One example in my trading bot is that whenever the bot executes a trade or a stoploss is invoked, send me a slack message. [Read More]

Monitoring is a big topic. I can attempt to simplify it into a few categories. Passive Infrastructure Monitoring: Track CPU, Memory, Load, Uptime, Network, etc Log shipping: This is where your services stream their logs and you set alerts or metrics on certain patterns Active Monitoring: Ping your servers on a timer for aliveness/health, or a full verification of an API endpoint and its response Application Performance Monitoring: Metrics through code injection/addition to view application metrics The one I want to discuss today is passive monitoring. [Read More]

Disclaimer: I haven’t ran any benchmarks or ran any of these databases in HA mode for proof of concepts but this was my experience with trying to work with lots of data for my HFT Trading Bot. The Data: (all indexed on time) 1 minute candle: {timestamp, instrument, high, low, open close, volume, trades}, 1 inserts per minute Individual trade: {timestamp, instrument, price, side, type}, 0-2000 inserts per minute Individual order: {timestamp, instrument, price, side, type, status}, 0-5000 inserts per minute Note: There will exist one collection/database per exchange as certain exchanges provide more valuable information than others. [Read More]

I’ve been working with Linux since 2008. Back then, I mostly used a spare machine to run FreeNAS, Apache, MySQL, Python to host a static or Wordpress website. Then as I got more invested with projects running on Linux, I started to set up VPSes to start hosting online in 2011. Everytime, it was the same process. Login as root, set up new user, disable root, set up PKA, enable firewall… etcetc. [Read More]