OS X Cleanup

Developer computers often get a lot of cruft built up in non-standard places because of compiled binaries, assets, packages, and other tools that we install over time then forget about as we move onto other projects. In general, I like to reinstall my OS and wipe my disk every year or so to prevent crud from accumulating. As an interemediate step, this post compiles several maintenance caommands that I run fairly routinely....

November 24, 2020 · 3 min · 480 words · Benjamin Bengfort

Managing Multi-Errors in Go

This post is a response to Go: Multiple Errors Management. I’ve dealt with a multiple error contexts in a few places in my Go code but never created a subpackage for it in github.com/bbengfort/x and so I thought this post was a good motivation to explore it in slightly more detail. I’d also like to make error contexts for routine cancellation a part of my standard programming practice, so this post also investigates multiple error handling in a single routine or multiple routines like the original post....

October 22, 2020 · 4 min · 790 words · Benjamin Bengfort

Writing JSON into a Zip file with Python

For scientific reproducibility, it has become common for me to output experimental results as zip files that contain both configurations and inputs as well as one or more output results files. This is similar to .epub or .docx formats which are just specialized zip files - and allows me to easily rerun experiments for comparison purposes. Recently I tried to dump some json data into a zip file using Python 3....

August 20, 2020 · 4 min · 700 words · Benjamin Bengfort

Read mprofile Output into Pandas

When benchmarking Python programs, it is very common for me to use memory_profiler from the command line - e.g. mprof run python myscript.py. This creates a .dat file in the current working directory which you can view with mprof show. More often than not, though I want to compare two different runs for their memory profiles or do things like annotate the graphs with different timing benchmarks. This requires generating my own figures, which requires loading the memory profiler data myself....

July 27, 2020 · 2 min · 253 words · Benjamin Bengfort

Basic Python Profiling

I’m getting started on some projects that will make use of extensive Python performance profiling, unfortunately Python doesn’t focus on performance and so doesn’t have benchmark tools like I might find in Go. I’ve noticed that the two most important usages I’m looking at when profiling are speed and memory usage. For the latter, I simply use memory_profiler from the command line - which is pretty straight forward. However for speed usage, I did find a snippet that I thought would be useful to include and update depending on how my usage changes....

July 14, 2020 · 2 min · 370 words · Benjamin Bengfort

Launching a JupyterHub Instance

In this post I walk through the steps of creating a multi-user JupyterHub sever running on an AWS Ubuntu 18.04 instance. There are many ways of setting up JupyterHub including using Docker and Kubernetes - but this is a pretty staight forward mechanism that doesn’t have too many moving parts such as TLS termination proxies etc. I think of this as the baseline setup. Note that this setup has a few pros or cons depending on how you look at them....

October 9, 2019 · 8 min · 1644 words · Benjamin Bengfort

Mount an EBS volume

Once the EBS volume has been created and attached to the instance, ssh into the instance and list the available disks: $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 86.9M 1 loop /snap/core/4917 loop1 7:1 0 12.6M 1 loop /snap/amazon-ssm-agent/295 loop2 7:2 0 91M 1 loop /snap/core/6350 loop3 7:3 0 18M 1 loop /snap/amazon-ssm-agent/930 nvme0n1 259:0 0 300G 0 disk nvme1n1 259:1 0 8G 0 disk └─nvme1n1p1 259:2 0 8G 0 part / In the above case we want to attach nvme0n1 - a 300GB gp2 EBS volume....

February 5, 2019 · 2 min · 323 words · Benjamin Bengfort