Posts

I have a minor issue with freezing requirements, and so I put together a very complex solution. One that is documented here. Not 100% sure why this week is all about packaging, but there you go. First up, what is a requirement file? Basically they are a list of items that can be installed with pip using the following command: $ pip install -r requirements.txt The file therefore mostly serves as a list of arguments to the pip install command. The requirements file itself has a very specific format and can be created by hand, but generally the pip freeze command is used to dump out the requirements as follows: ...

Package deployment is something that is so completely necessary, but such a pain in the butt that I avoid it a little bit. However to reuse code in Python and to do awesome things like pip install mycode, you need to package it up and stick it on to PyPI (pronounced /pīˈpēˈī/ according to one site I read, though I still prefer /pīˈpī/). This process should be easy, but it’s detail oriented and there are only two good walk throughs (see links below). ...

The topic of the day is a simple one: JSON serialization. Here is my question, if you have a data structure like this: import json import datetime data = { "now": datetime.datetime.now(), "range": xrange(42), } Why can’t you do something as simple as: print json.dumps(data)? These are simple Python datetypes from the standard library. Granted serializing a datetime might have some complications, but JSON does have a datetime specification. Moreover, a generator is just an iterable, which can be put into memory as a list, which is exactly the kind of thing that JSON likes to serialize. It feels like this should just work. Luckily, there is a solution to the problem as shown in the Gist below: ...

Programming with databases is a fact of life for any seasoned programmer (read, “worth their salt”). From embedded databases like SQLite and LevelDB to server databases like PostgreSQL, data management is a fundamental part of any significant project. The first thing I should say here is skip the ORM and learn SQL. SQL is such a powerful tool to query and manage a database, and is far more performant thanks to 40 years of research and development. ...

If you’ve pair programmed with me, you might have seen me type something to the following effect on my terminal, particularly if I have just created a new file: $ codetime Then somehow I can magically paste a formatted timestamp into the file! Well it’s not a mystery, in fact, it’s just a simple alias: alias codetime="clock.py code | pbcopy" Oh, well that’s easy — why the blog post? Hey, what’s clock.py? A great question! This Python script is the dumbest thing that I have ever written, that has become the most useful tool that I use on a daily basis. Whenever there is a dumb to useful ratio like that, it’s blogging time. Here is clock.py: ...

The standard library logging module is excellent. It is also quite tedious if you want to use it in a production system. In particular you have to figure out the following: configuration of the formatters, handlers, and loggers object management throughout the script (e.g. the logging.getLogger function) adding extra context to log messages for more complex formatters handling and logging warnings (and to a lesser extent, exceptions) The logging module actually does all of these things. The problem is that it doesn’t do them all at once for you, or with one single API. Therefore we typically go the route that we want to wrap the logging module so that we can provide extra context on demand, as well as handle warnings with ease. Moreover, once we have a wrapped logger, we can do fun things like create mixins to put together classes that have loggers inside of them. ...

Let’s face it, most of the Python programs we write are going to be used from the command line. There are tons of command line interface helper libraries out there. My preferred CLI method is the style of Django’s management utility. More on this later, when we hopefully publish a library that gives us that out of the box (we use it in many of our projects already). Sometimes though, you just want a simple CLI script. These days we use the standard library argparse module to parse commands off the command line. Here is my basic script that I use for most of my projects: ...