Monthly Archives: September 2014

IDL 8.2+ Segmentation Fault / Abort on OSX with Dual-Thunderbolt Displays

The Symptoms

IDL 8.2+ immediately crashes at run-time if LM_LICENSE_FILE or /Applications/exelis/license/license.dat is populated. If neither the environment variable or the license file are present, IDL enters demo mode successfully.

IDL 8.2 Crash Message

$ idl
IDL Version 8.2.3, Mac OS X (darwin x86_64 m64). (c) 2013, Exelis Visual Information Solutions, Inc.
Segmentation fault: 11

IDL 8.3 Crash Message

$ idl
IDL Version 8.3, Mac OS X (darwin x86_64 m64). (c) 2013, Exelis Visual Information Solutions, Inc.
idl(3438,0x7fff736a7310) malloc: *** error for object 0xf07: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6 

The Cause

At run-time, if licensing data is provided by the end-user, IDL attempts to gather information about the local system’s network interfaces (specifically the MAC address). In our case IDL crashes because we have two external Thunderbolt monitors connected to the computer. If we remove one of the external displays IDL is able to continue on and retrieve a license without incident.

The system experiencing the crash has eleven network interfaces defined, consisting of Bluetooth (1), FireWire (2), WiFi (1), Thunderbolt Ethernet (2), On-board Ethernet (2), Bridge Adapters (2), and an IPSec VPN (1). Whether these interfaces are active or not means nothing, as IDL will try to poll each one for a valid MAC address.

IDL 8.2 Backtrace

The data left behind in /Library/Logs/DiagnosticReports was mildly useful with IDL 8.2. The “l_getid_hw” tried to read the hardware address of a given (unknown) ethernet device, but failed miserably in the process:

VM Regions Near 0x2540be570:
    mapped file            0000000106483000-0000000107af6000 [ 22.4M] r--/r-x SM=ALI  /usr/share/icu/icudt51l.dat
--> 
    STACK GUARD            00007fff5bc00000-00007fff5f400000 [ 56.0M] ---/rwx SM=NUL  stack guard for thread 0

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libidl.8.2.dylib                    0x000000010079d6df l_getid_hw + 34 <<<< HERE
1   libidl.8.2.dylib                    0x000000010079d534 l_getid_type + 2064
2   libidl.8.2.dylib                    0x000000010079cc65 l_gethostid + 26
3   libidl.8.2.dylib                    0x00000001007866f6 checkout_from_server + 278
4   libidl.8.2.dylib                    0x000000010078400a lm_start_real + 1048
5   libidl.8.2.dylib                    0x000000010078343b l_checkout + 530
6   libidl.8.2.dylib                    0x0000000100783157 lc_checkout + 191
7   libidl.8.2.dylib                    0x00000001000ec7c2 IDL_lm_checkout2 + 184
8   libidl.8.2.dylib                    0x00000001000eddc4 d_line + 103
9   libidl.8.2.dylib                    0x00000001000eef7b init_system_routines + 699
10  libidl.8.2.dylib                    0x000000010000c306 IDL_Initialize + 3591
11  libidl.8.2.dylib                    0x000000010000c4b8 IDL_Main + 64
12  idl                                 0x0000000100000f11 main + 227
13  idl                                 0x0000000100000e0c start + 52

IDL 8.3 Backtrace

Unfortunately, IDL 8.3’s crash report is essentially useless. No relevant information was given to help trace the error.

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000

Application Specific Information:
abort() called
*** error for object 0xf07: pointer being freed was not allocated 

Conclusion

Until Exelis releases a patch to address this issue I recommend unplugging your second monitor if you need to get your work done.

Note: We did not delete the unused Thunderbolt interfaces from System Preferences -> Network to test whether this cleared things up or not. Maybe it does, maybe it doesn’t. Give it a try.

Update (09/25/2014)

Chris Torrence on the comp.lang.idl-pvwave Google group responded to my post (here):

Hi Joe,

I’ll echo what David said – excellent bug report! I believe that this bug (IDL-69092) has been fixed in IDL 8.4, due out in just a few weeks. If you’re curious, this was a bug in the FlexLM driver. Here is the bug information from Flexera:

“lmhostid instability in the presence of multiple ethernet adapters. Previously, in the presence of several ethernet adapters, lmhostid and the callers of the lc_hostid API may experience a segmentation fault on OS X. This is now resolved. (FNP-9885)”

Apparently, Matlab and other products ran into the same issue.

Joe, once you’ve upgraded to IDL 8.4, could you post here and let us know if it’s fixed? If it’s still an issue, definitely contact support@exelisvis.com.

Cheers,

Chris

ExelisVIS

(Thanks again Chris!)

Update (10/24/2014)

Great news!

Now that we finally got our hands on IDL 8.4 I wanted to confirm this update did fix the segmentation fault issue on the machines in question.

Ureka, Eclipse+PyDev Quickstart

The Eclipse interface differs slightly between operating systems (particularly OSX). If you are regular user of Eclipse, you should not have much trouble following along. Otherwise, I suggest flipping through the Eclispe documentation before diving in.

Install Eclipse

http://wiki.eclipse.org/Eclipse/Installation

Install PyDev

  1. Click Help
  2. Select Eclipse Marketplace...
  3. Under the Search tab select Yaxos Marketplace (the orange icon)
  4. In the Find: textbox enter “pydev”, and click Go or press the [Enter] key
  5. Select “PyDev for Eclipse”
  6. Click Install
  7. Proceed through the plugin installation dialog(s).
  8. Restart Eclipse when prompted
  9. Select the PyDev perspective (upper-right-hand corner of the Eclipse interface)

PyDev Interpreter Setup

Important note: Step 7 grants you the ability to import modules installed via pip within the Ureka environment.

  1. Click Window
  2. Select Preferences
  3. Expand the PyDev settings tree
  4. Select Python Interpreter
  5. Click New...
    • Interpreter Name: Ureka
    • Interpreter Executable: /path/to/Ureka/python/bin/python2.7
  6. Click Ok
  7. With the Ureka interpreter highlighted, click New Folder
    • Location: /path/to/Ureka/variants/common/lib/python2.7/site-packages
  8. Click Ok to close the New Folder dialog box
  9. Click Ok to apply the new interpreter settings

PyDev Editor Settings (Personal Touch)

I seriously recommend enabling line trimming if you intend to share your code with others.

  1. Repeat steps 1, 2, and 3
  2. Expand the Editor settings tree
  3. Expand the Code Style settings tree
  4. Select Code Formatter ( ^ = CHECKED, _ = UNCHECKED )
    • [^] Auto-format only files in the workspace?
    • [_] Use autopep8.py for code formatting?
    • [_] On save, only apply formatting in changed lines?
    • [^] Use spaces after commas?
    • [_] Use space before and after parenthesis?
    • [_] Use space before and after assign for keyword arguments?
    • [^] Use space before and after operators? (+, -, /, *, //, **, etc.)
    • [^] Right trim lines?
    • [^] Right trim multi-line string literals?
    • [^] Add new line at end of file?
    • [2 spaces] Spaces before comment?
    • [At least 1 space] Spaces in comment start?

Optimizing Ureka

As promised in my previous post, I have decided to release the Ureka optimization script I’ve been working on. Head over to the ur_optimize project site for more details: click here

Enjoy!


Why do I want to use ur_optimize?

The version of ATLAS bundled with Ureka (http://ssb.stsci.edu/ureka) is designed to work on a large number of CPUs, therefore certain operations requiring linear algebra may perform inadequately to your needs.

ur_optimize rebuilds the underlying ATLAS/LaPACK/BLAS stack to be tailored for your CPU’s architecture. After that, the NumPy/SciPy stack is recompiled and linked against the new ATLAS implementation in order to take advantage of these optimizations.

License

BSD

Requirements

RHEL 6.0+ / Fedora 10+

sudo yum install gcc gfortran

Ubuntu 12.04+

sudo apt-get install build-essential

Testing ur_optimize

For this test I used the latest Ureka build (as of 09/04/2014):

Ureka_linux-rhe5_64_1.4.1.tar.gz

SYSTEM

Manufacturer: Dell Inc.
Product Name: OptiPlex 790

BIOS

Vendor: Dell Inc.
Version: A07
Release Date: 09/10/2011

CPU

vendor_id   : GenuineIntel
cpu family  : 6
model       : 42
model name  : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
stepping    : 7
microcode   : 0x29
cpu MHz     : 3399.867
cache size  : 8192 KB
physical id : 0
siblings    : 8
core id     : 3
cpu cores   : 4

MEMORY INFORMATION

size            : 16GB
type            : DDR3
Mhz             : 1333

test_linalg.py module

#!/usr/bin/env python
import numpy
import scipy.interpolate

def interpolate(user_methods=None, points=1000):
    x = numpy.linspace(1.0, 1000.0, num=points)
    y = numpy.linspace(1.0, 1000.0, num=points)

    methods = ['linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic']
    if user_methods is not None:
        methods = user_methods

    for method in methods:
        print("{}...".format(method)),
        try:
            scipy.interpolate.interp1d(x, y, kind=method, bounds_error=False, fill_value=0.0)
        except MemoryError:
            # Common ocurrance with quadratic
            print("failed")
        
        print("ok")

if __name__ == "__main__":
    interpolate()

Performance Comparison

Before ur_optimize

$ ipython
Python 2.7.5 (default, Jun 19 2014, 11:22:38) 
Type "copyright", "credits" or "license" for more information.

IPython 2.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import test_linalg

In [2]: %timeit test_linalg.interpolate(['quadratic'])
quadratic... ok
quadratic... ok
quadratic... ok
quadratic... ok
1 loops, best of 3: 39 s per loop

39 seconds per operation.

After ur_optimize

$ ipython
Python 2.7.5 (default, Jun 19 2014, 11:22:38) 
Type "copyright", "credits" or "license" for more information.

IPython 2.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import test_linalg

In [2]: %timeit test_linalg.interpolate(['quadratic'])
quadratic... ok
quadratic... ok
quadratic... ok
quadratic... ok
1 loops, best of 3: 1.17 s per loop

1.17 seconds per operation.

Found a bug?

If ATLAS fails to compile for your architecture, please contact the developers of ATLAS, not me: http://math-atlas.sourceforge.net/faq.html#help

Otherwise, feel free to submit a bug report on this project’s issue tracker: http://bitbucket.org/jhunkeler/ur_optimize/issues