Category Archives: Python

Installing OpenCV via brew

Install Brew (Quickstart)

$ cd /usr/local
$ sudo ruby -e "$(curl -fsSL"
$ sudo ln -s /usr/local/homebrew/bin/brew /usr/local/bin/brew

Install opencv

$ sudo brew tap homebrew/science
$ sudo brew install -v opencv

Adjust PYTHONPATH to find opencv

$ export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH

Importing opencv

$ python
import cv


Disabling MacPorts and/or Fink

Brew’s “doctor” suggests we get rid of /sw and /opt/local from our environment if they exist, however this can prove to be cumbersome, especially when you have many different paths defined (most of which you’re not willing to lose). Below is a simple Python script that takes N paths and strips them from PATH. The resulting string is PATH excluding any matched patterns.

#!/usr/bin/env python
import os
import sys

if __name__ in '__main__':
    ARGS = sys.argv[1:]
    PATH = os.environ['PATH']
    PATH_NEW = []

    for arg in ARGS:
        for path in PATH.split(':'):
            if arg in path:



Assume your standard PATH variable is defined as follows:

$ echo $PATH

In order to remove the offending tree (/sw) from PATH, we execute path_purge.

$ purge_path /sw

Notice that /sw is indeed missing in the output.

Putting purge_path to work

$ export PATH=`eval purge_path /sw`
$ echo $PATH
$ setenv PATH `purge_path /sw`
$ echo $PATH

Using purge_path with brew

Create the following wrapper script(s):

exclusion="/sw /opt/local"
export PATH=`eval /usr/local/bin/purge_path ${exclusion}`
export PYTHONPATH=/usr/local/lib/python2.7/site-packages

[Tcsh title=”~/bin/percolate.csh”]
set exclusion="/sw /opt/local"
setenv PATH `/usr/local/bin/purge_path ${exclusion}`
setenv PYTHONPATH /usr/local/lib/python2.7/site-packages

If you intend to run brew packages with Fink or Macports installed in parallel, issue (depending on your default shell) source ~/bin/ or source ~/bin/percolate.csh after opening a fresh terminal.

Ureka, Eclipse+PyDev Quickstart

The Eclipse interface differs slightly between operating systems (particularly OSX). If you are regular user of Eclipse, you should not have much trouble following along. Otherwise, I suggest flipping through the Eclispe documentation before diving in.

Install Eclipse

Install PyDev

  1. Click Help
  2. Select Eclipse Marketplace...
  3. Under the Search tab select Yaxos Marketplace (the orange icon)
  4. In the Find: textbox enter “pydev”, and click Go or press the [Enter] key
  5. Select “PyDev for Eclipse”
  6. Click Install
  7. Proceed through the plugin installation dialog(s).
  8. Restart Eclipse when prompted
  9. Select the PyDev perspective (upper-right-hand corner of the Eclipse interface)

PyDev Interpreter Setup

Important note: Step 7 grants you the ability to import modules installed via pip within the Ureka environment.

  1. Click Window
  2. Select Preferences
  3. Expand the PyDev settings tree
  4. Select Python Interpreter
  5. Click New...
    • Interpreter Name: Ureka
    • Interpreter Executable: /path/to/Ureka/python/bin/python2.7
  6. Click Ok
  7. With the Ureka interpreter highlighted, click New Folder
    • Location: /path/to/Ureka/variants/common/lib/python2.7/site-packages
  8. Click Ok to close the New Folder dialog box
  9. Click Ok to apply the new interpreter settings

PyDev Editor Settings (Personal Touch)

I seriously recommend enabling line trimming if you intend to share your code with others.

  1. Repeat steps 1, 2, and 3
  2. Expand the Editor settings tree
  3. Expand the Code Style settings tree
  4. Select Code Formatter ( ^ = CHECKED, _ = UNCHECKED )
    • [^] Auto-format only files in the workspace?
    • [_] Use for code formatting?
    • [_] On save, only apply formatting in changed lines?
    • [^] Use spaces after commas?
    • [_] Use space before and after parenthesis?
    • [_] Use space before and after assign for keyword arguments?
    • [^] Use space before and after operators? (+, -, /, *, //, **, etc.)
    • [^] Right trim lines?
    • [^] Right trim multi-line string literals?
    • [^] Add new line at end of file?
    • [2 spaces] Spaces before comment?
    • [At least 1 space] Spaces in comment start?

Optimizing Ureka

As promised in my previous post, I have decided to release the Ureka optimization script I’ve been working on. Head over to the ur_optimize project site for more details: click here


Why do I want to use ur_optimize?

The version of ATLAS bundled with Ureka ( is designed to work on a large number of CPUs, therefore certain operations requiring linear algebra may perform inadequately to your needs.

ur_optimize rebuilds the underlying ATLAS/LaPACK/BLAS stack to be tailored for your CPU’s architecture. After that, the NumPy/SciPy stack is recompiled and linked against the new ATLAS implementation in order to take advantage of these optimizations.




RHEL 6.0+ / Fedora 10+

sudo yum install gcc gfortran

Ubuntu 12.04+

sudo apt-get install build-essential

Testing ur_optimize

For this test I used the latest Ureka build (as of 09/04/2014):



Manufacturer: Dell Inc.
Product Name: OptiPlex 790


Vendor: Dell Inc.
Version: A07
Release Date: 09/10/2011


vendor_id   : GenuineIntel
cpu family  : 6
model       : 42
model name  : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping    : 7
microcode   : 0x29
cpu MHz     : 3399.867
cache size  : 8192 KB
physical id : 0
siblings    : 8
core id     : 3
cpu cores   : 4


size            : 16GB
type            : DDR3
Mhz             : 1333 module

#!/usr/bin/env python
import numpy
import scipy.interpolate

def interpolate(user_methods=None, points=1000):
    x = numpy.linspace(1.0, 1000.0, num=points)
    y = numpy.linspace(1.0, 1000.0, num=points)

    methods = ['linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic']
    if user_methods is not None:
        methods = user_methods

    for method in methods:
            scipy.interpolate.interp1d(x, y, kind=method, bounds_error=False, fill_value=0.0)
        except MemoryError:
            # Common ocurrance with quadratic

if __name__ == "__main__":

Performance Comparison

Before ur_optimize

$ ipython
Python 2.7.5 (default, Jun 19 2014, 11:22:38) 
Type "copyright", "credits" or "license" for more information.

IPython 2.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import test_linalg

In [2]: %timeit test_linalg.interpolate(['quadratic'])
quadratic... ok
quadratic... ok
quadratic... ok
quadratic... ok
1 loops, best of 3: 39 s per loop

39 seconds per operation.

After ur_optimize

$ ipython
Python 2.7.5 (default, Jun 19 2014, 11:22:38) 
Type "copyright", "credits" or "license" for more information.

IPython 2.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import test_linalg

In [2]: %timeit test_linalg.interpolate(['quadratic'])
quadratic... ok
quadratic... ok
quadratic... ok
quadratic... ok
1 loops, best of 3: 1.17 s per loop

1.17 seconds per operation.

Found a bug?

If ATLAS fails to compile for your architecture, please contact the developers of ATLAS, not me:

Otherwise, feel free to submit a bug report on this project’s issue tracker:

Eureka! Erm… Ureka!

What is Ureka?

Ureka serves as an all-purpose science solution, that allows users to test different python modules and environment configurations separate from their standard shell environment. The software distribution also includes many common science tools, such as, IRAF 2.16, SAO DS9, S(ource)Extractor, Python (astropy, numpy, scipy, etc). Ureka’s subset of GNU libraries is quite extensive for a relatively small package (~3GB). A bulk of its total size is IRAF, which is an unavoidably huge science package, but Ureka has outdone itself by taming the beast into a self-contained structure (unlike installing it from scratch), so you can spend more time focusing on functional work, rather than troubleshooting missing trailing slashes.

Ureka’s core relies heavily on virtualenv and makes creating “variants” of your current installation extremely easy. As a developer, if you require ten different Python installations this distribution should make your life much easier.


Refer to this page for more detailed installation information.

The shortest quickstart guide in the history of science software, ever:

$ wget
$ chmod +x install_ureka_1.4
$ ./install_ureka_1.4
$ source ~/.bashrc
$ ur_setup


Compared to other software distributions in the same league, such as Anaconda, Ureka is on-par. The added benefit of Ureka’s built-in complement of science tools makes everything a bit more palatable. Would you rather compile IRAF and link it against Anaconda’s libraries all by your lonesome? How about SAODS9? I seriously doubt it. Ureka has the upper hand in this regard but there is one area where performance falls unavoidably short: ATLAS (Automatically Tuned Linear Algebra Software).

Anyone I have ever spoken with about ATLAS, even mentioning it in passing, starts out the same way: The person involuntarily rolls their eyes and lets out a huge painful sigh as if they are reliving a recurring nightmare from their childhood. Every single person complains it was one of the most annoying pieces of software to compile, ever.

ATLAS is “automatically tuned” (i.e. it benchmarks itself against your physical system hardware at compile-time) and therefore makes releasing an even marginally functional pre-compiled shared library impossible.

Ureka’s linear algebra performance on Linux is not very impressive. All hope is not lost! If you are willing to give up an afternoon… Manually compiling ATLAS+LAPACK(Tuned) then re-linking SciPy and NumPy via pip works fantastically. I’ve written a script to do this automatically, so when I’ve ironed out all the kinks I’ll post a how-to article.

The Science Software Branch at STScI has gone through a lot of effort to make this distribution easy to use on a day-to-day basis. So even if you’re not receiving the best number crunching capabilities out of the box, you’re already making up lost time not building/installing extremely difficult to understand, often poorly documented and barely maintained scientific tools that everybody relies on.


ATLAS performance degradation is limited to Linux only.
OSX provides VecLib (Apple’s in-house build of ATLAS/BLAS/LAPACK) which Ureka’s linear algebra dependent software is linked against by default. So if you own an iThing you’re OK for the time being.

Potential Caveats

  • Upgrading the Ureka distribution destroys changes made by the user.
  • Wrapping of system-provided utilities could lead to unwanted effects.

I understand the need for some of these hacks from a development standpoint… I am not sure I agree with them being implemented in a production build.

Environment Variables

IRAF’s nightmare of a FORTRAN compiler wrapper is unfortunately here to stay, so if you decide to compile code while Ureka is active, due to library linkage issues you may need to redefine the following variables (i.e., F77=gfortran, F2C=gfortran) for normal compilation to work again.

$ which $F77

$ which $F2C


The wrapping of GCC can be undesirable if you rely on a tool-chain other than the system’s default.

$ which `gcc`

The following error is benign but may be confusing to a newcomer:

$ gcc
Undefined symbols for architecture x86_64: "_main", referenced from: 
    implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Because gcc returns 1 instead of 0, this behavior may interfere with scripts that check for the existence and/or functionality of GCC.


If you rely on pkg-config (most GNU software does) consider removing this wrapper script, or be prepared to set -L and -I arguments manually in CFLAGS/LDFLAGS, etc.

$ which pkg-config

$ cat `which pkg-config`

case "$*"
        /usr/bin/pkg-config $*
        exit 126


If otool doesn’t exist on your Apple iThing there is a substantially high probability Ureka isn’t the only piece of software no longer able to check library linkage. However, it does appear to be supplied within the distribution.

$ which otool

$ diff /usr/bin/otool `which otool`
Binary files /usr/bin/otool and /opt/Ureka/bin/otool differ


This Apple iThing system utility for changing RPATHS is included in the distribution as well.

$ which install_name_tool

$ diff /usr/bin/install_name_tool `which install_name_tool`
Binary files /usr/bin/install_name_tool and /opt/Ureka/install_name_tool differ