Astronomy bottlenecks…

[tweetmeme only_single=false source=allinthegutter]

Guess what’s the largest hurdle impeding scientific progress in astronomy? Lack of money? Governmental disinterest? Nope, according to a paper published yesterday it’s our bad programming skills.

Modern astronomers are much more likely to be found in front of a computer these days than behind a telescope. We spend our time analysing our data and to do so we need to write computer programs. Some of these are pretty simple (`read table, make graph, join dots’), but often they’re much more sophisticated (`read table, decipher Universe’ etc). However, whilst we generally received some basic programming training during our degrees, we tend not to be proper software developers. As the authors of the paper point out this can mean that our programs:

…often contain the goto statement every 10–20 lines; names of variables do not
follow any conventions, i.e. a1, a2, aa1; the code is unreadable: no or bad indentations,
very long function bodies and/or source files. There is a lot of hard-coding of file and
device names, file system paths.


When the author is
returning to the same program after several months or years, he/she often finds that the
existing procedure/function calls do not satisfy his/her needs, however is not willing to
modify them to keep the backward compatibility. Then, a wrapper routine is created
which is calling some underlying procedures/functions in a slightly different way.

These aren’t the only flaws they highlight but they’re the ones, I’m embarrassed to admit, I might, perhaps, maybe inflict on my own code (though not the GOTO statement – I don’t do that)!

(GOTO as seen by Futurama from this page of programmer’s jokes)

So, is hiring software developers to code `properly’ for us the answer? Nope: our programs might be inefficient and unwieldy but

…at the end the program does what it is supposed to, because the author
knows exactly what it should do. Even though it may sometimes crash during run-time
or have very poor performance,

whereas a programming professional could write pretty code but might not spot an obvious flaw in the output. It’s not all bleak though, as they also give some examples of well-written pieces of astronomy software such as the excellent TOPCAT which Niall has guest-blogged about before over at AstroBetter.

The authors’ solution to the problem is to introduce mandatory undergraduate courses in these skills and to wean us all off the obsolete programming languages (in their opinion) that we insist on using. This sounds like a good idea, especially as this problem will only increase in future as the amount of data we have to handle rises. This only leaves the question of what to teach everyone which, as xkcd points out, is another kettle of fish entirely.

ResearchBlogging.orgIgor Chilingarian, & Ivan Zolotukhin (2010). The True Bottleneck of Modern Scientific Computing in Astronomy Astronomical Societ of the Pacific arXiv: 1012.4119v1