Astronomy bottlenecks…

[tweetmeme only_single=false service=wp.me source=allinthegutter]

Guess what’s the largest hurdle impeding scientific progress in astronomy? Lack of money? Governmental disinterest? Nope, according to a paper published yesterday it’s our bad programming skills.

Modern astronomers are much more likely to be found in front of a computer these days than behind a telescope. We spend our time analysing our data and to do so we need to write computer programs. Some of these are pretty simple (`read table, make graph, join dots’), but often they’re much more sophisticated (`read table, decipher Universe’ etc). However, whilst we generally received some basic programming training during our degrees, we tend not to be proper software developers. As the authors of the paper point out this can mean that our programs:

…often contain the goto statement every 10–20 lines; names of variables do not

follow any conventions, i.e. a1, a2, aa1; the code is unreadable: no or bad indentations,

very long function bodies and/or source ﬁles. There is a lot of hard-coding of ﬁle and

device names, ﬁle system paths.

and

When the author is

returning to the same program after several months or years, he/she often ﬁnds that the

existing procedure/function calls do not satisfy his/her needs, however is not willing to

modify them to keep the backward compatibility. Then, a wrapper routine is created

which is calling some underlying procedures/functions in a slightly diﬀerent way.

These aren’t the only flaws they highlight but they’re the ones, I’m embarrassed to admit, I might, perhaps, maybe inflict on my own code (though not the GOTO statement – I don’t do that)!



(GOTO as seen by Futurama from this page of programmer’s jokes)

So, is hiring software developers to code `properly’ for us the answer? Nope: our programs might be inefficient and unwieldy but

…at the end the program does what it is supposed to, because the author

knows exactly what it should do. Even though it may sometimes crash during run-time

or have very poor performance,

whereas a programming professional could write pretty code but might not spot an obvious flaw in the output. It’s not all bleak though, as they also give some examples of well-written pieces of astronomy software such as the excellent TOPCAT which Niall has guest-blogged about before over at AstroBetter.

The authors’ solution to the problem is to introduce mandatory undergraduate courses in these skills and to wean us all off the obsolete programming languages (in their opinion) that we insist on using. This sounds like a good idea, especially as this problem will only increase in future as the amount of data we have to handle rises. This only leaves the question of what to teach everyone which, as xkcd points out, is another kettle of fish entirely.

Igor Chilingarian, & Ivan Zolotukhin (2010). The True Bottleneck of Modern Scientific Computing in Astronomy Astronomical Societ of the Pacific arXiv: 1012.4119v1