Monthly Archives: June 2014

Eureka! Erm… Ureka!

What is Ureka?

Ureka serves as an all-purpose science solution, that allows users to test different python modules and environment configurations separate from their standard shell environment. The software distribution also includes many common science tools, such as, IRAF 2.16, SAO DS9, S(ource)Extractor, Python (astropy, numpy, scipy, etc). Ureka’s subset of GNU libraries is quite extensive for a relatively small package (~3GB). A bulk of its total size is IRAF, which is an unavoidably huge science package, but Ureka has outdone itself by taming the beast into a self-contained structure (unlike installing it from scratch), so you can spend more time focusing on functional work, rather than troubleshooting missing trailing slashes.

Ureka’s core relies heavily on virtualenv and makes creating “variants” of your current installation extremely easy. As a developer, if you require ten different Python installations this distribution should make your life much easier.

Installation

Refer to this page for more detailed installation information.

The shortest quickstart guide in the history of science software, ever:

$ wget http://ssb.stsci.edu/ureka/1.4/install_ureka_1.4
$ chmod +x install_ureka_1.4
$ ./install_ureka_1.4
...
$ source ~/.bashrc
$ ur_setup

Performance

Compared to other software distributions in the same league, such as Anaconda, Ureka is on-par. The added benefit of Ureka’s built-in complement of science tools makes everything a bit more palatable. Would you rather compile IRAF and link it against Anaconda’s libraries all by your lonesome? How about SAODS9? I seriously doubt it. Ureka has the upper hand in this regard but there is one area where performance falls unavoidably short: ATLAS (Automatically Tuned Linear Algebra Software).

Anyone I have ever spoken with about ATLAS, even mentioning it in passing, starts out the same way: The person involuntarily rolls their eyes and lets out a huge painful sigh as if they are reliving a recurring nightmare from their childhood. Every single person complains it was one of the most annoying pieces of software to compile, ever.

ATLAS is “automatically tuned” (i.e. it benchmarks itself against your physical system hardware at compile-time) and therefore makes releasing an even marginally functional pre-compiled shared library impossible.

Ureka’s linear algebra performance on Linux is not very impressive. All hope is not lost! If you are willing to give up an afternoon… Manually compiling ATLAS+LAPACK(Tuned) then re-linking SciPy and NumPy via pip works fantastically. I’ve written a script to do this automatically, so when I’ve ironed out all the kinks I’ll post a how-to article.

The Science Software Branch at STScI has gone through a lot of effort to make this distribution easy to use on a day-to-day basis. So even if you’re not receiving the best number crunching capabilities out of the box, you’re already making up lost time not building/installing extremely difficult to understand, often poorly documented and barely maintained scientific tools that everybody relies on.

Note

ATLAS performance degradation is limited to Linux only.
OSX provides VecLib (Apple’s in-house build of ATLAS/BLAS/LAPACK) which Ureka’s linear algebra dependent software is linked against by default. So if you own an iThing you’re OK for the time being.

Potential Caveats

  • Upgrading the Ureka distribution destroys changes made by the user.
  • Wrapping of system-provided utilities could lead to unwanted effects.

I understand the need for some of these hacks from a development standpoint… I am not sure I agree with them being implemented in a production build.

Environment Variables

IRAF’s nightmare of a FORTRAN compiler wrapper is unfortunately here to stay, so if you decide to compile code while Ureka is active, due to library linkage issues you may need to redefine the following variables (i.e., F77=gfortran, F2C=gfortran) for normal compilation to work again.

$ which $F77
/opt/Ureka/iraf/unix/hlib/f77.sh

$ which $F2C
/opt/Ureka/iraf/unix/bin.linux/f2c.e

GCC

The wrapping of GCC can be undesirable if you rely on a tool-chain other than the system’s default.

$ which `gcc`
/opt/Ureka/python/bin/gcc

The following error is benign but may be confusing to a newcomer:

$ gcc
Undefined symbols for architecture x86_64: "_main", referenced from: 
    implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Because gcc returns 1 instead of 0, this behavior may interfere with scripts that check for the existence and/or functionality of GCC.

PKG-CONFIG

If you rely on pkg-config (most GNU software does) consider removing this wrapper script, or be prepared to set -L and -I arguments manually in CFLAGS/LDFLAGS, etc.

$ which pkg-config
/opt/Ureka/python/bin/pkg-config

$ cat `which pkg-config`
#!/bin/sh

case "$*"
in
    *freetype2*|*cfitsio*)
        /usr/bin/pkg-config $*
        ;;
    *)
        exit 126
        ;;
esac

OTOOL (OSX)

If otool doesn’t exist on your Apple iThing there is a substantially high probability Ureka isn’t the only piece of software no longer able to check library linkage. However, it does appear to be supplied within the distribution.

$ which otool
/opt/Ureka/python/bin/otool

$ diff /usr/bin/otool `which otool`
Binary files /usr/bin/otool and /opt/Ureka/bin/otool differ

INSTALL_NAME_TOOL (OSX)

This Apple iThing system utility for changing RPATHS is included in the distribution as well.

$ which install_name_tool
/opt/Ureka/bin/install_name_tool

$ diff /usr/bin/install_name_tool `which install_name_tool`
Binary files /usr/bin/install_name_tool and /opt/Ureka/install_name_tool differ