Senior Software Engineer
Science Software Branch / Operations & Engineering Division
Space Telescope Science Institute (Association of Universities for Research in Astronomy)
June 2015 – Present
Senior Systems Administrator
Core Mission Services
Space Telescope Science Institute (Association of Universities for Research in Astronomy)
October 2014 – June 2015
Systems Administrator II
Space Telescope Science Institute (Haymaker Technologies, Inc.)
April 2009 – September 2014
PC/Printer Hardware Technician
Space Telescope Science Institute (Symmetry Services Group, LLC.)
July 2008 – March 2009
- CompTIA A+ Certification, 2009
- Red Hat Certified Systems Administrator, RHEL 5, 2009
- Space Telescope Science Institute Achievement Award, IT Excellence, 2014
- Space Telescope Science Institute Achievement Award, For the creation of the Russell B. Makidon Optics Laboratory, 2014
- GED, 2003
- ANSI C
- PC gaming
- This blog does not represent the opinion of STScI, my employer, or any associated entities.
- The opinions expressed on this site are my own.
Going back as far back as I can remember, having been a child born in the mid-eighties, computers were far less useful but hundreds of orders of magnitude more intriguing than they are today. Back then, circa 1984, both of my parents worked for DEC (Digital Equipment Corporation) as field technicians out of their D.C. headquarters. I grew up surrounded by DECMate computers and their phosphor monitors. I was terrified of the basement in our house, not because monsters or ghosts lived down there, but due to the fact, it always seemed to be glowing an eerie shade green or amber. My father constructed more or less a clean room environment down there to do system repairs (computer hardware was extremely susceptible to damage back in those days). Dad would disappear for hours on end and eventually the smell of rosin core solder and Winston cigarettes would waft into the living areas.
In 1989, my father, still working for DEC, landed a contract at STScI, one year before the greatest space observatory in human history (HST) launched. Back then, when NASA had multitudes more funding, STScI used to take “bring your son or daughter to work” to an all-new level. They threw all day parties, with a moon bounce, clowns, balloons, face painting, free food, drinks, and a slew of hands-on activities. The free posters and lithographs were my favorite part of going to this event every year. My bedroom at home was plastered floor to ceiling with Hubble Space Telescope information and imagery.
However, on less kid-oriented days, when my father took me to work due to school closings, or babysitters not panning out, I would accompany him in the server room to observe him repairing VAX and SUN systems. Alternatively, I would sit around drawing bizarre shapes or scenes on his whiteboard while inadvertently getting high off the dry erase fumes. If I did not come home without black and red marker dust on my hands, you would know I was not at STScI for the day. I loved looking at pictures on the lightning fast bonded T1 line of old Apollo mission pictures (yeah, I was obsessed with the moon landing) all rendered beautifully in 800x600x16 resolution on a now ancient SparcStation.
My first “personal” computer at home was a 286 running DOS. The hard drive was monstrous (maybe 20 megabytes) and it was enough to store a few ASCII text games and all of my parent’s WordPerfect documents. In all of their cassette tape glory — my father also owned a Commodore 64, VIC-20, and a Sinclair ZX Spectrum. The Commodore caught my attention, of course, being a gaming computer with BASIC as the operating system I spent countless hours transcribing code out of numerous “game books”. Do you want to play a game on your home computer? Code it yourself.
As computer hardware got increasingly more complex and games arrived on a far better mediums, it was not long before I was playing Wolfenstein 3D, DOOM, and Quake religiously. Mind you, my family also kept up with Nintendo. My grandmother gave me an NES in 1990 for my fifth birthday, and my mom bought the SNES in 1994. I feel like I invested more hours playing DOOM than I did studying in all of my years at school. Gaming jump-started my interest in programming, because I remember being fascinated about the interaction of objects based on user input. If I left-clicked on an enemy it received damage and after enough left clicking the animated monster would drop to the floor in a pile of pixelated gore. There was obviously something cool going on behind the scenes and I wanted to know what it was.
My dad and his friend Patrick went to a computer show in Hunt Valley, MD one Saturday morning. They returned with a four CD-ROM set of this new operating system called “Slackware Linux”. I had seen articles about it in PC Magazine but the pictures were so dull. A black screen with white text, oh boy. This reminded me of DOS so I immediately stopped caring. After Patrick and my father installed Slackware on a Pentium Pro upstairs, neither of them really had any idea what to do with it. Though I am certain their futile attempts to configure XFree86 without an instruction manual were the reason they could go no further. Windows 95 was back on the system by the end of the evening.
In 1996, I brushed the dust off my yellowing BASIC booklets. Instead of copying the code straight from the book, I decided to study it. I wanted to understand how a monkey could throw an exploding banana across the screen and kill its opponent. Amidst playing DOOM (which never stopped by the way) I was turning bananas into kiloton blasts and making “Nibbles the Worm” into something far more treacherous than originally intended. Eventually however, the idea of producing a video game from scratch subsided and left me wondering what I should do next.
During a modest week-long vacation to Florida with my mother in 1999, I purchased a copy of Red Hat Linux 5.2 for $34.99 at Wal-Mart; complete with the Gnome/Sawfish window manager and a slew of useful GUI utilities. I studied the enclosed installation manual front to back many times before returning to Maryland. Unlike my father and his friend five years prior, I felt prepared for the battle ahead of me. Unfortunately, due to a video contrast bug in the Hercules monochromatic video adapter in my 486/DX/66 that dream was going to have to wait.
A few years later, armed with a relatively bad ass Duron 800MHz computer and a brand new NVidia TNT video card I set out on my next quest. Compared to the herculean pile of crap I was working with when I was fourteen this thing truly hauled ass. By my sixteenth birthday, my interest in Linux had come back like an itch I could not scratch. After many hours doing research I decided to try Slackware as my dedicated desktop system (with Windows 2000 on a separate hard drive as a fallback). I operated in this fashion for years. After school (if I decided to go) I’d boot into Linux, mess around, compile software, read the various man pages, do something stupid as root, break the system, and rebuild it. Strangely, needing to re-install the operating system after every horrifying mistake ingrained an appreciation for Linux in me that never went away.
At most, I would re-install Windows a couple times a year, primarily after unknowingly installing a virus (this was before any virus scanners were worth their salt) or if the Windows registry went to lunch and never came back, which it often did in those days. I was re-installing Linux on a near daily basis. The operating system afforded the ability to destroy your computer at-will. Linux never killed itself. No, it was completely up to me whether the build lasted a day, week, or a year. The total freedom to destroy the very thing you love.
Both my mother and father would get so angry with me for staying awake until three in the morning “finishing a build” on a weeknight (a.k.a. school night). The door to my room would open, the light obviously stinging their eyes, there would be a cigarette hanging from the corner of their mouth, flopping up and down as they’d make some statement about me never making it anywhere in life “playing with the computer”. The future was too important to sit around wasting away in my room. Ok fine, perhaps I was a bit anti-social in those days. I had a few close friends but for the most part, I found myself hard set on my goal and determined to continue learning.
I was never a troublemaker — I simply never did any of my written homework or classwork. I’d participate in group projects and always “read chapters A through B” at night from my text books, so I was still actively learning, but this ultimately lead me to drop out of school. I was bored with “Computer Topics I”, “Word Processing”, and “Introduction to Computers”.
For the following year, I spent every waking moment in front of a computer. My parents were still completely shell shocked with my decision to leave school they had no idea what to do with me. Telling me to get a job was like checking how much gas is in your car with a Zippo. Despite my teenage defiance, this gave me plenty of time to teach myself what I wanted to learn. My girlfriend at the time, although she shared the same views as my parents, bought me a book “C Primer Plus” by Stephen Pratta. This book changed everything.
By this time, my girlfriend’s father found me a job where he worked. Between building cubicles, chairs, desks, and other pieces of modular office furniture, I would read my book on C, and take notes. At night, instead of having fun, I would apply whatever I had learned by writing little programs like cash registers and parks full of zoo animals. Alternatively, examining existing programs on my computer and attempting to emulate their functionality turned out to be the best way to hone my skills.
None of this self-learning paid off at all for many years, so I was actually beginning to believe my parents various early morning lectures from long ago. Instead I did what I could to survive outside of the information technology niche (i.e. retail, manual labor, etc.), but in 2008, all of that changed. My dad called me up one evening with a proposition. A sub-contractor of my dad’s company was seeking to hire an on-site hardware technician to repair Apple products, Dell servers, and maintain the Hewlett-Packard printers. I was hesitant to apply for the position considering my blatant lack of functional experience in the field, but after a hefty dose of convincing, my resume was in mail.
The phone interview must have gone exceptionally well, because by the following week I found myself knee deep in printer toner and motherboards. Known only as “The Toner Guy” (I was not prestigious enough to have a name at this point) you would find me in the hallways with my extremely loud toner vacuum, vigorously extracting loose particles expelled from burst cartridges on worn drums. On occasion, I would be replacing parts or removing a printer to take it to another part of the building, but for the most part, I went rightfully unnoticed by my peers.
Although it was not required for me to attend branch meetings, I did anyway. I would listen to my co-workers, some of which I had grown up with in this building (see also; family day), and offer input if I felt like I knew the answer to a question.
Then it happened.
During a rather innocuous meeting with the datacenter folks, one of the managers said, “We have hundreds of hard drives that need to be wiped. Who wants to do it?”
I offered, because changing printer toner was not only mundane, it did not happen very often. After the meeting, this manager shuffled me into an extremely cold room with computers stacked to the ceiling.
“Here we are”, they said pointing at a piled assortment of decrepit hardware, “These are the systems we need wiped.”
Many of these were from the late-nineties, most of which were dead, dying, or incapacitated in one way or another.
The manager directed me to an Ultra 10 workstation, connected was a couple of Ultra SCSI II multipacks, and several instances of ‘dd’ still executing in the foreground.
This is going to take a while, I thought.
The process of hooking up random boxes and attempting to wipe them, especially antique OpenBoot PROM-based SPARC machines, is no easy task. Nine times out of ten, they did not even boot into the PROM. Moreover, this was a mixed bag of SCSI and PATA hard drives so it only added to the pain and suffering.
I told the manager I would come up with a solution to the problem. I did not tell them what it was, but I hurried back to my office and started doing research on hard drive annihilation. DBAN seems like the obvious choice (as the ultimate quick fix) but since they were not already using it, I figured this would be the perfect opportunity to write something in C. Therefore, I did, and holy crap was it awesome. The program (NetNuke) did not have the aesthetic polish of DBAN since there was no GUI to speak of, however it did have a certain level of charm to it. I went with FreeBSD 7 as my operating system of choice, because “camcontrol” far exceeded the disgusting /proc interface to the SCSI subsystem on Linux. I believe the resulting bootable NetNuke ISO was less than 50MB, and required a meager 32MBs of RAM to operate.
It all boils down to making a choice. Do I take the path of least resistance or charge headlong through unfamiliar (often-dangerous) territory to achieve my goal? Thinking back, if I had simply slapped DBAN on a couple of systems and walked away, then perhaps I would have continued to go unnoticed. They would refer to me as “The Toner Guy That Wipes Hard Drives” rather than just “The Toner Guy”.
Instead, shortly after implementing my bootable dystopian bit death program with great success, they offered me a full-time position as a Systems Administrator at STScI. I believe it is far more rewarding to gain recognition through action and perseverance than via oh-so-important gilded sheet of a paper with a signature on the dotted line.
At this point, my responsibilities changed drastically. No longer did I play “find the torx” with Apple laptops, or replace dead motherboards in PowerEdge servers, and changing toner had become an afterthought. Instead, I had entered a very different ballpark, in game where I had to interact with people, directly. Everyone has faked a smile or vocal inflection at some point in his or her life I suppose, but we are talking about me here. I am like a ray of sunshine penetrating the opaque clouds of a nuclear winter.
It came as a surprise to me that, even though I am somewhat anti-social, like many IT folks, it became unnecessary to fake anything at all. It turns out that doing something you enjoy might just make you smile anyway, even if it is for the wrong reasons. Some of the calls I would get, even if the problem happened to be a two-second fix, also incurred a fifteen-minute story tax. I truly had no idea my job was that of a pseudo-phycologist until working at a help desk. I found people are far friendlier when you have a real conversation with them, well after the assigned task is completed, and it seems the human condition, no matter how anti-social you are, predisposes you to talk about yourself relentlessly.
With that said, it took me a while to cope with the excess dialogue. Originally, when I started working in this field I found myself unable to multitask. A person talking in your ear as you attempt to concentrate is a skill learned, not a gift handed down through the generations. Because of this, my work performance waivered as I adjusted to this new world of human interaction, and that by itself is slightly terrifying. You begin to question whether you can do this, because you are still not off the master’s leash. You perform tasks as you receive them, when you receive them, and the omnipotent fear of failure looms over you wherever you go.
In this new world, you must also perform a ritual sacrifice of your sanity, in that you sit on the help desk for an entire month to “learn the ropes” in order to familiarize yourself with the client-base. My sentence upstairs began on June 26, 2009, two short weeks before STS-125 launched to perform Hubble’s fourth and final servicing mission. As the days pushed forward like a tsunami pulling back the ocean from the shore, getting closer and closer to the launch date, the volume of phone calls increased dramatically. People, mostly from Goddard Spaceflight Center, some in-house, were stopping by to get their VPN accounts setup. Standard accounts not used in ages were suddenly required to be refreshed and active. Computers locked in desk drawers since Servicing Mission 3 were cropping up all over the place. Hard drives were failing. Video cards in desktops were burning up. Dogs and cats living together – that sort of thing.
At the time of the launch, it was absolute mayhem, and this trend of anarchy continued well after the astronauts returned to Earth on May 24. In all honesty, I felt as if I had become a veteran overnight. By the time I returned to my new office in the depths of the building, I had a moment to reflect on having made it through something so crazy in one piece, which I decided nothing else here would ever reach beyond that now-predetermined threshold of stress.
In the years following the servicing mission, I have always found new and interesting ways to throw myself from the proverbial pan and into the fire. Perhaps I am a glutton for punishment. The downside of being an opinionated individual with a knack for finding the “wrong” a midst the “right”, is that I find myself doing projects all alone. Often the project involves programming something innovative to make life easier, but people lack either the time or the expertise to assist me. However, I enjoy a good challenge as you might expect, so I have no hard feelings about it.
For example, once upon a time, I built several dozen packages for RHEL 6 in preparation of incorporating Linux laptops at STScI. I built every scientific tool imaginable, most of which required some serious code patching to get working, to include a Python 2.7 tool-chain suite with as many astronomer-friendly modules as possible. If you can think of a program to package into an RPM, I probably did. Nevertheless, do you think this project ever got off its feet? Nope. Was that the only thing I had going on during the month it took me to complete the assigned task? Nope. Did I sleep during this time? Nope.
These setbacks teach you more about a particular topic than you imagine. I spent many hours buried up to my neck in Red Hat’s documentation, for so long in fact, I can probably recite every keyword permutation and macro of the file parsing engine from memory. In the end, despite my initial disappointment about having my work disappear into the bit-bucket, I sharpened a valuable skill along the way. As it turns out, I required this skill more than I thought. Nowadays I build RPMs on a nearly constant basis.
When it came time to transition from Solaris 10 to Solaris 11, I had no idea how to create an IPS software package. Oracle’s operating system documentation is as terrible as it is incomplete, so I suppose the developers found it difficult to use too, but out of necessity, I wrote a package building utility in Python to ease the pain. Had I not spent so much time learning about the inner workings of RPMs I would still likely be scratching my head.
Everything happens for a reason (in the non-religious kind of way, of course). My failure in the past ended up leading me straight into a future success through experience and knowledge. I love it when that happens.
I have also had my share of personal failures. I am not ashamed to admit that I have rebooted important servers in the middle of the day, accidentally deleted somewhat mission-critical data with a malformed argument, or even having caused an entire subnet to disappear from existence by issuing a poorly constructed “ifconfig”. However, as much it hurt at the time to benevolently destroy everything around me, I have always filed these mental lapses away in the “never to repeat” category of my brain.
For example, if the prompt of the server I am logged into does not display the hostname, I instinctually change PS1 to include ‘h’ to avoid accidental reboots. Now I read and re-read ‘rm’ calls to make sure every space character is shell-escaped. In addition, when in doubt while working with “ifconfig” or “ip”, I scour the manual page for more information before proceeding.
A piece of advice though, should anyone ever claim to be perfect, just smile, nod, and find a person who has destroyed the only remaining tape archive of something important. They know what they are doing.