A very simple server that can be used to catch all incoming HTTP requests and just echo them back + log their contents. I needed it to test what a webhook actually returned to me, but I’m sure that there are a number of other things it could be dropped in for.

It will take in any GET/POST/PATCH/DELETE HTTP request with any path/params/data (optionally JSON), pack that data into a JSON object, and both log that to a file (with a UUID1 based name) plus return this object to the request.

Warning: Off hand, there is already a potential security problem in this regarding DoS. It will happily try to log anything you throw at it, no matter how big and will store those in memory first. So long running requests / large requests / many requests will quickly eat up your RAM/disk. So… don’t leave this running unattended? At least not without additional configuration.

That’s it! Hope it’s helpful.

read more...

Another thing that came up recently: we have many (many) sidekiq queues. Each has their own admin interface, but sometimes you just want all of the information in one place. Of course, you could bookmark all of the pages. Or make a single page with a lot of frames (remember HTML frames?). Or use their API. But where’s the fun in that? Instead, let’s dig straight into the redis backend and see what we can see!

read more...

For a number of years now, I’ve been writing down my ‘observations’. Essentially, it’s a semi-structured set of text files that I keep in Dropbox. One for each day, in a folder by month. I record interesting people I see, things I did worth doing, and things my children did which were adorable.

After a while, I started wanting to look back, so first, I wrote a relatively simple script that would go back through my archives and send me everything I did 1/2/3/4/etc years ago. That worked well enough, but it ended up generating a lot of emails to go through some days. So the second generation is a server that can format those pages and display them as a nice webpage.

The most interesting part perhaps was dealing with the tarballs that I keep the archives in (they’re plain text, so they compress very well). I wanted to keep them compressed, so I had to decompress them in memory on the fly.

read more...

I’ve recently started to listen to audiobooks again (The Aeronaut’s Windlass). If you buy books through Audible or some other setup that has their own app, it’s a straight forward enough process. On the other hand, if you have them on CD and want to play them on a mobile device… It’s a little more interesting.

I tried a few different apps that purport to do exactly what I wanted: import an audiobook as a folder full of MP3s and play them, but none that quite meet what I wanted. Since I also listen to a lot of podcasts and have more than one podcast app that I really like (I’ve used and liked both Downcast and Pocket Casts), I decided to see if I couldn’t use one of those as an audiobook player.

read more...

Recently at work, we were trying to get an application that uses websockets working on an AWS instance behind an ELB (load balancer) and nginx on the instance.

If you’re either not using a secure connection or handling the cryptography on the instance (either in nginx or Flask), it works right out of the box. But if you want the ELB to handle TLS termination it doesn’t work nearly as well… Luckily, after a bit of fiddling, I got it working.

Update 2018-05-31: A much easier solution, [https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/](just use an ALB):

WebSocket allows you to set up long-standing TCP connections between your client and your server. This is a more efficient alternative to the old-school method which involved HTTP connections that were held open with a “heartbeat” for very long periods of time. WebSocket is great for mobile devices and can be used to deliver stock quotes, sports scores, and other dynamic data while minimizing power consumption. ALB provides native support for WebSocket via the ws:// and wss:// protocols.

read more...

I had an interesting problem recently on a project I was working on. It’s a simple Flask-based webapp, designed to be deployed to AWS using Docker. The application worked just fine when I was running it locally, but as soon as I pushed the docker container…

Latency spikes. Bad enough that the application was failing AWS’s healthy host checks, cycling in and out of existence¹:

read more...