I use watchdog to run continuous integration on my projects: every time a file changes (e.g. is saved, deleted, or created) in the directory I am watching, a script will automatically run unit tests, compile libraries, build docs, and run other tests, as necessary.

Below is a sample script using watchdog that monitors the current directory for changes to .py files and .rst files. When a .py file changes, it runs unit tests; when a .rst file changes, it builds the documentation. Here, I assume that the documentation is built using Sphinx, and is located in the `docs` subdirectory. `python -m unittest discover -b` is used to run the unit tests.

Of course, you could perform any actions you wanted, using any triggers you choose.

The first thing that you need to do is define an event handler based on watchdog.FileSystemEventHandler:

Here, I'm catching all file events, and reacting to changes to .py and .rst (reStructured text) files.

The next thing is to create a watchdog.Observer, and register your event handler.

Finally, define your actions. Here are mine for running unit tests and building my docs.

Here is the full code listing:

You can download the code here.

Doing so is quite simple:

IEHtmlWindow has a ctrl member that you can use to manipulate a page's DOM. One of the cool things that this allows you to do is edit a specific element on a page.

To do this, you first need to get the desired element:

And then set contentEditable to "true".

However, getElementByID returns an IHTMLElement interface, and contentEditable is only supported by the IHTMLElement3 interface and higher (IE has been around for a while, and there are now something like 6 iterations of this interface…). We therefore need to call QueryInterface on our element in order to get an IHTMLElement3.

The QueryInterface method call takes the class of the interface we want, which lives in the MSHTML module generated by comtypes (iewin ensures it's generated), so we need to import that.

Then, we get the IHTMLElement3 interface, and set contentEditable to "true" like this:

Since we can't manipulate the DOM until the document is loaded, it's a good idea to put this code in the DocumentComplete event handler (which we get called by setting ourselves as the event sink).

So putting it all together:

That's it. Not bad for 26 lines of code, including docstrings.

py2exe is amazingly good at sussing out your program's dependencies, but it's a complex task, and inevitably there are some gotchas.

One of them is the DLL file "POWRPROF.DLL".

This happens when building your executable on Windows Vista/7, and then deploying it on Windows XP. Since this has bitten me twice now, I thought I'd describe it here.

Windows Vista starting shipping a version of POWRPROF.DLL that is incompatible with XP and earlier. Because of the rules that XP uses for DLL lookup, if py2exe includes this DLL, then it won't find the correct one in your system directory.

Typically, the error you'll get when running your compiled program on Windows 2000/XP is "ImportError: DLL load failed: The specified procedure could not be found.", raised by win32api.

The fix is to add "POWRPROF.DLL" to the list of "dll_excludes" passed into py2exe. That will make your executable use whichever version is in the Windows system directory (and will save you a few KB on your bundled app size).

Here's how I do this with the setup script of one of my programs, Align Assist:

I was happy to take on this project, partly because Murasaki is something of a legend in the Japanese rock scene (and especially here in their home prefecture of Okinawa), and partly because it would be a good chance to test out some ideas I've been having about getting simple, clean websites up quickly and efficiently.

There was a firm deadline for getting the site up: a major Japanese heavy-metal magazine (Burrn!) was going to feature Murasaki, and they were going to list the band's website. My strategy was therefore to get something simple and working up by the deadline, and then add to the site gradually.

The Tools

django

I'm partial to cherrypy/jinja2/SQLAlchemy as my web stack, but for this project I chose django for the website backend, because django is known for making bog-standard stuff very easy.

The built-in admin interface was a big win for django, because it allowed the band to get in and start adding information right away, as I worked on the site design and structure.

A major annoyance, however, was the inability to smoothly evolve my database. django won't modify database tables for you if you add a field to one of your models, and since I'm growing the site organically, I'm faced with either mucking with the SQL by hand to modify my tables, or violating YAGNI by adding all the fields up front, even if they're not used now (and might never be used).

A good example was lyrics for song tracks. At first, I didn't add a lyrics field, because we weren't sure if we were going to publish them. But then the band's manager decided that it would be OK, so I had to modify the database to add the field. The next issue is samples for tracks; the band thinks that it would be a cool idea, but hasn't worked out if they want them. So that will be another modification to the database if they're added.

I think that django's insistence on making database evolution as hard as possible is wrong-headed and violates the Pythonic principle that we're all consenting adults. Modify the damn table for me, and let me deal with the consequences!

I'll note that although there are a couple of django evolution apps, the ones I checked didn't support sqlite databases (which I'm using because the database is currently around 100 KB, and I don't expect it to ever pass 1 MB).

jQuery UI

I chose jQuery UI for the site layout. The Theme Roller application for designing a custom theme was a huge win; I was able to get exactly the look and colors I wanted in minutes.

One small issue I had was that the UI is a bit too coupled to the JavaScript end. For example, I decided to use the tabs widget for the navigation menu, but out of the box, this widget makes you load all the content into the page, and switch using JavaScript. I had to muck about in the CSS file in order to have the tabs link to actual URLs.

I also benefited from a couple of other jQuery components, such as jQuery galleria for the photo gallery (although I'll probably move to something else as more photos are added, and I add a tagging feature).

WordPress

I used WordPress to create two blogs for the site (one English and one Japanese). Yes, there are blog solutions for django, but WordPress is ready to go out of the box, and in this case (a tight deadline and budget), practicality definitely beat purity.

I created themes integrating the blogs into the site design using Themes Press — that was $10 very well spent! I doubt that even an experienced wordpress hacker could create an integrated theme for less than $10 of their time.

Overall Impressions

I think that the combination of django and jQuery UI can make designing a simple site very quick and painless, although django has some issues with organic site growth, and jQuery UI couples the CSS and JavaScript a bit too tightly for me.

There were a couple of other minor issues (such as when my approach to internationalization differed from django's), but they were probably due more to my lack of familiarity with the frameworks than inherent limitations, and I found workarounds for all of them, so no big deal.

It was also a good experience for me to step away from my more familiar combination of cherrypy/jinja2/SQLAlchemy to use django. I think that in cases like this one, where you have non-programmers contributing a fair amount of the content, the advantages of the free admin interface outweigh the loss of flexibility from using an all-in-one stack like django.

Here's the project URL: http://bitbucket.org/ginstrom/mailer

Apparently you can't open write access to everyone, so if you'd like write privileges, then please let me know your account name.

About mailer

mailer is a wrapper around the Python email and mimetypes libraries.

Ajax stands for "asynchronous JavaScript and XML," but these days json is often used as a lighter-weight alternative to XML. In this tutorial, I'm therefore going to show how to use the python simplejson library to enable communication between python and JavaScript using json.

What you will need for this tutorial:

• cherrpy
• simplejson
• jQuery
• The sample project

You can get cherrypy and simplejson from pypi via easy_install; there's a copy of jQuery in my sample project. The sample project consists of three files, in the following structure:

ajax_app.py
media/
  + jquery-1.4.2.min.js
  | index.html

First, I'll show the HTML file:

Let's see what this is doing. In the body, you've got a title tag, and bog standard HTML form, which asks for a name.

In the head, you've got an include to the jQuery library, and the following JavaScript code:

The initial $(function() statement is a jQuery construct that says, "When the page loads, execute this anonymous function." That function then creates a callback for the submit event of the element with id "testform" (the form). When the form is submitted, the jQuery post function is executed, which is where the magic happens.

jQuery.post is an Ajax method. The first argument is the URL to post to ("/submit"); the second argument is the data to send in the post (the "name" value from the form); and the third argument is the callback to call with the data we receive from the post.

The function then takes this data, and uses its "title" element to update the title of the page.

Next, here's the ajax_app.py file:

First, the AjaxApp class is our application. It's pretty simple, having only two methods: index and submit.

When the index method is called, the app opens the index file and returns it.

The submit method is our Ajax method; when it is called with the form data, we use the information to create a new title, and pass that back as json data using simplejson.dumps. The JavaScript function that called this method can then use the data to update the page content.

This little piece of code is a convenience for development; when you run the cherrpy app, it loads the index page in your web browser.

Finally, the last two lines mount the app, and start up the server. The "config" dict tells cherrpy to serve up static files from the "media" dir in the script's home directory.

If you download the sample project, unzip it, and run ajax_app.py, you should get the following screen (shown here with the name filled in):

Ajax app -- step 1

Here it is after filling in the name and clicking Set:

Ajax app -- step 2

When to use Ajax

Ajax is useful when you want to update the page content without reloading the entire page. This is useful for CRUD apps when you want to do things like edit items in place, delete items, or add items.

When not to use Ajax

Since Ajax actions don't correspond to URIs, you can't bookmark Ajax actions, send links to friends, or use the back and forward buttons to navigate like in a normal website.

For this reason, avoid Ajax if you want certain application states to be navigable as URIs. For example, if you have a calendar application, and use Ajax to show all the data, the user won't have any way to bookmark an event on December 12th, 2011. Likewise, if you use an Ajax data grid to show tables of data, you won't be able to email your data views to your colleagues.

One middle way is to offer a "permalink" for each Ajax view, which allows for bookmarking, sending links, etc. This still breaks the back-button functionality, but may be worth the trade-off.

In this post, I want to share a small helper module called sftp (zip file) (code in post below) that wraps paramiko.SFTPClient and makes uploading/downloading via SFTP even simpler.

First, some usage

Upload or download a file

with statement also supported:

Finally, a demo recipe for uploading all the png files from a specified local directory to a specified directory on the server:

Now, here's the code for my sftp module:

I've been reading Richard Feynmann's fascinating reminiscences in Surely You're Joking, Mr. Feynmann. The entire text is available online.

One passage that really struck me was where he described the cyclotrons at Princeton, MIT, and Cornell:

It reminded me of my lab at home. Nothing at MIT had ever reminded me of my lab at home. I suddenly realized why Princeton was getting results. They were working with the instrument. They built the instrument; they knew where everything was, they knew how everything worked, there was no engineer involved, except maybe he was working there too. It was much smaller than the cyclotron at MIT, and "gold-plated"? — it was the exact opposite. When they wanted to fix a vacuum, they'd drip glyptal on it, so there were drops of glyptal on the floor. It was wonderful! Because they worked with it. They didn't have to sit in another room and push buttons! (Incidentally, they had a fire in that room, because of all the chaotic mess that they had — too many wires — and it destroyed the cyclotron. But I'd better not tell about that!)

(When I got to Cornell I went to look at the cyclotron there. This
cyclotron hardly required a room: It was about a yard across — the diameter of the whole thing. It was the world's smallest cyclotron, but they had got fantastic results. They had all kinds of special techniques and tricks. If they wanted to change something in the "D's" — the D-shaped half circles that the particles go around — they'd take a screwdriver, and remove the D's by hand, fix them, and put them back. At Princeton it was a lot harder, and at MIT you had to take a crane that came rolling across the ceiling, lower the hooks, and it was a hellllll of a job.)

Princeton and Cornell had inferior equipment, but they got superior results partly because of the quick iteration loop, eliminating the step of waiting on the engineers to make changes. Scientists at these schools could very quickly try something, make adjustments, and try again, where at MIT the same adjustments took a lot of time.

Shuji Nakamura, inventor of the blue LED, said something similar about his work:

I had a very, very small budget and had to make everything I needed myself. I even made my own reactors—the furnaces needed to do the crystal work.

Nakamura succeeded in creating a blue LED, working by himself, when huge multinationals were pouring hundreds of millions of dollars into R&D without results. Elsewhere, he commented that the ability to make adjustments to his equipment quickly, without getting engineers involved, was one of the secrets to his success.

It struck me that this is one of the reasons why I get better results with Python than with compiled languages like C and C++. With Python, there's no compiler in the loop, so the iteration cycle is very fast — rarely more than 10 seconds to run all the unit tests.

This affects every part of the development cycle. Development itself is a lot faster, so you get more done. With a faster cycle, refactoring code in small steps is also quick and painless, so it's easier to evolve code. The cost of trying out new ideas is low, so you're encouraged to prototype lots of designs to get the best ones.

I think that in just about any kind of creative endeavor, the ability to rapidly iterate between creation and feedback gives a tremendous boost to productivity. This is one of the reasons why I'm so much more productive with Python.

• Project homepage
• pypi page

The result is my kanjinums module, with a function kanji2num that will convert a string containing a kanji num to a Python integer.

Download the source distribution (kanjinums-0.1.zip)
Download the Windows installer (kanjinums-0.1.win32.exe)
Download the unit tests (test_kanjinums.zip)

Examples:

Here's the full code: