The official blog from the team at Runnable.

9 Common Dockerfile Mistakes

We work with Dockerfiles on a daily basis; all the code we run for ourselves and for our customers, we run from a set of Dockerfiles. In this article, we’ll talk about what mistakes people commonly make, and how to write them better. For those of you who are Docker experts, a lot of the tips in this article will probably be pretty obvious and will just provoke a lot of head-nodding. But for beginner to intermediate developers, this will be a useful guide that will hopefully help clean and speed up your workflow.

Keep reading
Distributing Docker Cache across Hosts

Building and compiling code can take a huge hit on our time and resources. If you have dockerized your application, you may have noticed how much of a time-saver Docker cache is. Lengthy build commands can be cached and not have to be run at all! This works great when you’re building on a single host; however, once you start to scale up your Docker hosts, you start to lose that caching goodness.

Keep reading
External API Caching with Varnish & Nginx

We use a number of external services to handle data retrieval, monitoring, and reporting. The importance of each service depends on its role in our sandboxes product. For instance, it’s generally fine if we miss a few reports sent to Datadog but it is not okay if we are unable to reach the GitHub API. In this post, we’ll discuss the issues we’ve faced when interfacing with external services and dive into how we use Varnish Cache to handle them.

Keep reading
Blueprint Architecture for Managing Static Sites

I see Runnable as a very versatile, and therefore powerful, tool. It enables teams to deploy and run code on every commit. The concept sounds simple, but it lights up a vast number of not-so-obvious use cases.

For example, here’s how Runnable can be added to an existing revision workflow for static sites / blogs to improve how updates can be made.

Keep reading
Introducing Ponos: A RabbitMQ-based Worker Server

Our problem started out benign enough: we were dealing with third party endpoints that started dropping requests and we could no longer count on to be reliable. The immediate solution was to write retry logic in-place within our API server that instigated these calls, adding a layer of protection around the library that did not have its own retry logic. However, when put under load, this retry logic started becoming a bottleneck, holding onto unnecessary resources and creating race conditions we had never before seen. Debugging it started becoming nightmare-esque and maintaining it became nigh impossible.

Keep reading
Stringify: The Silent Killer

How much do you log (in your application)? Do you use tools like Bunyan to make your logs easier to read? Do you Stringify everything and print it straight to console.log()? If so, logging may be the reason why you’re having trouble reducing your app’s average route response time to under 200ms.

When we first started implementing a unified logging system throughout our API, we didn’t anticipate how something as simple as logging to console could possibly slow down our system. Our data models were in their infancy; they barely had actual data on them, so it became common to just Stringify the whole object into the log. It sounded like such a great idea! Investigating bugs was easier since we had a record of the entire data object during each step of our flow.

But then things started to change…

Keep reading
The Mysterious Case of the Leading Space

We had just performed an overhaul of how we handled logging across our services at Runnable. It appeared to work as intended — log files were being rotated and sent over to Loggly. I later learned we had a bug that was causing logs to be improperly formatted.

“Logs aren’t formatting correctly for me.” My fellow engineer showed me what looked like JSON, which is what our various services produce as log output.

“Looks OK to me,” I offered trepidatiously.

“You would think so, but when I pipe it into Bunyan, nothing happens.”

Sure enough, Bunyan wasn’t re-formatting the JSON as expected.

Keep reading
Preserving web terminals during connection hiccups

Sockets can be hard. Scaling out websockets can be even harder. At Runnable, we make use of websockets heavily — for notifying users that their containers are running, for implementing deployment messages, and for powering our terminals in the browser. Terminals are a tricky beast to tackle. I learned that a few weeks ago, when our users’ terminals were getting lost and reset based on uncontrollable network issues.

Keep reading