Don't Be Stupid

Time for a bit of self-flagellation, I think. Nothing like public humiliation to remind you not to do bad things. Feel free to skip this if you're not into programming. The takeaway message is that I'm a bumbling dolt - not that most you aren't aware of it already.

Here's the thing: I'm creating some simulations in Python, a common programming language. Well, the simulation itself is run in highly optimized parallel C++ code (we use Nest; it's good, take a look), but we use Python to create the model, set up and run the simulation and collect and save the resulting data.

We run quite a lot of simulations, so I want to collect the data from one simulation run into one catalog. So, towards the end of the simulation we create a new catalog:

os.makedirs(catalog)

Now, the problem is that if the catalog already exists this will fail. Python will complain and stop. We need to check that the catalog doesn't already exist first:

if not os.path.exists(catalog):
    os.makedirs(catalog)

Great! Problem solved. Job well done, beers for everyone, pat yourself on the back. Except...

These simulations are pretty heavy. They take a long time to run. To make things go faster we use many CPU's in parallel. On my desk I have an 8-cpu computer which helps a bit, and I have access to 256 cpu's in a cluster in Tokyo if I need it. Eventually, of course, this project (where this model is just one part) aims to use the new super-computer in Kobe when it comes online next year, and that one has more cpu's than you can shake a very long stick at.

 
This means that the python code above isn't run just once for a simulation, but once for each cpu we use (some of you probably realize where this is going already). Normally this is not a problem: the first process to finish will create the new directory. The other processes will find the directory is already there and skip the creation.

But here's the problem: there's a tiny bit of time in between checking if the catalog exists, and creating it if it doesn't. What happens if two processes try to do this at the same time? Both find that the catalog does not exist, one of them creates the catalog, then the other tries to create it too - it wasn't there a moment ago after all - and the whole simulation fails. That's what happens.

This is called a "race condition", because you have two (or more) processes 'racing' each other, and you get different results depending on which one happens to get there first.

The chance of this actually happening - that two processes manage to get the timing so exactly wrong - seems really slim of course. It may indeed be a really rare and unlucky event if you just run a simulation once and get hit by this. But if you're running lots of simulations, and using lots of cpu's, then you're bound to get hit by this bug sooner or later.

And I did, this weekend. I had started a long series of simulations to run over the weekend. As I came to work this morning I found that the series had failed about halfway through because of this bug. Bad programmer. Bad, bad programmer. No coffee for you.

So, instead of almost three days worth of simulation data to analyze, I now have to spend another day generating the missing stuff - a day that I can ill afford with deadlines looming like thunderclouds over my head.

What should I have done? What should you do? Something like this:

try:
    os.makedirs(catalog)
except OSError, e:
    if e.errno == errno.EEXIST:
        pass
    else:
        raise

This is Python's way of dealing with exceptions - run-time errors such as failing to create a catalog. We don't check beforehand if the catalog exists, but simply try to create it. If we fail, we don't just stop. Instead we catch the error (that's the "except" bit). If the error is that the catalog already exists we just ignore it and continue the simulation (the "pass" thing). If it was some other kind of error we send it on for the system to take care of ("raise", as in raise a flag to alert that something is wrong).

I did not do this. Which made me lose a day's worth of data analysis. And makes me a blockhead. Don't be a blockhead like me. Take care of race conditions when you program. Do proper error checking and recovery. Think of the future. Think of the children.

EndNote sues Zotero

Citations power
your research

When researchers write a paper, we need to give references to any other paper from which we've used an idea, or any paper that is related to your own work in some way. Those references are cited in the text, then collected at the end of the paper, always in some very tightly specified format. This reference list is not some afterthought; the references are almost as important as the paper itself. So we tend to spend an inordinate amount of time collecting, sorting and generally dealing with references in some way or another.

There's several pieces of software that can help you handle this task. I use BibTeX and some related tools myself; another popular software is EndNote. It'll help you keep a database of references with links to the actual paper, your notes and so on, and can let you share your reference lists with others. Like BibTeX, EndNote also has a large collection of style files for various journals and conferences that will format your reference list in the appropriate way. It's worth noting that both reference lists and style files are generally created by the users - us - not by the software developers.

 
Enter Zotero. It's a paper and citation organization tool, as a Firefox plugin. It's completely free and open-source, and is more at home on the web than any of the earlier tools. I've tried it previously and while it hasn't displaced my current tools yet (old habits are hard to break) it does look very, very promising, and it's steadily becoming more and more popular among academics.

So promising, in fact, that Thomson Reuters (owners of EndNote and a lot of other academic-related property) is suing the makers of Zotero for allowing its users to convert EndNote style files and citation files into Zotero format. Files, worth noting, that were contributed by their users in the first place. More from Crooked Timber

This is exceedingly stupid. Or desperate - EndNote may be losing market share to open tools much faster than people realize. The stupid part is, they are antagonizing their own customer base - they're suing part of it. People whose livelihood depends in part on their citation databases are naturally a little wary about keeping them in a closed format like EndNote's, and now Thomson have given them ample reason to be anxious. They are stating, in effect, that they will do anything they can to prevent people from moving their own data away from the EndNote system. EndNote is an intellectual roach motel - your data enters but it can't leave.

The legal case itself seems to be on very shaky ground even in the US and it certainly has no merit at all in much of Europe (where reverse-engineering for interoperability is expressly legal). But that is beside the point. There is possibly not one single thing they could have done to damage their own product and promote Zotero further than by suing them in court.

This is the same Thomson Reuters that is one of the giants in scientific publishing that are already seen with some suspicion and hostility by the community itself due to their antagonistic stance on open access. If you ever needed a reason not to consider EndNote for your own work, this is it. And take a good look at Zotero; it may be everything you need and more already.

This just in: Researchers also Stupid

In further proof that a) scientists are no smarter than anyone else outside their own field; and b) nobody really understands all the research funding rules any longer, Tokyo university researchers in engineering and medicine have been caught falsifying their accounting in order to keep unused research funds to use the next year. The sad thing is, the funding rules would have allowed them to keep the unused funds without resorting to false accounting had anyone bothered to check....