I’m Old, Part LII: Nomenclature

Naming things is one of the hardest things in software engineering. You have to strike a balance between a name that conveys your abstraction but isn’t completely esoteric. For example, Mac OS 6.04 introduced the OS call ‘Gestalt’ which was to be used to find out specific information about the system on which an application was running. Gestalt is a horribly esoteric word, but it does convey what it does.

While I was working at Atalasoft, I did some code to handle digital signatures. The abstraction required an object to hold a set of signatures. I thought I would call it something like ‘philographist’ (someone who collected signatures or autographs), and while that describes precisely what this object does, I was certain that very, very few of our customers would have any idea what that meant. I ended up calling it PdfDocumentSignatureInformation or some such thing.

As a result of this, we end up putting trite names on things that we encounter. I want to celebrate some of the ones that my co-workers have come up with.

Drawer Dogs – Nathan Graham coined this term referring to hot dogs served at a movie theater. They always had the rollers loaded up with hot dogs at the concessions stand, but you never got one of those. No. When you asked for a hot dog, they opened a drawer and pulled one out.

Death Tart – I’m not putting a name on this one (you know who you are). It was in reference to the emerging goth trend in the 90’s and some of the stereotypical women styles.

Turd Fangs – Eric Zocher came up with this. There was a game for the Macintosh that was based on a 2D polygon engine. One of the first challenges you had was to get past a small swarm of poisonous creatures on the ground. The name was perfectly accurate for what it was.

Road Margarita – Jake Lauzier named this. It’s the combination of salt and slush that gets plowed into the end of your driveway.

Hard Candy Shell – Rick Minerich used this term to describe the result of an ice storm on your car.

Porknography – Me – in referring to catalogs we received at Atalasoft for Bacon and ham.

I’m Old, Part LI: Hacking Sleep

When I was working on the Acrobat team, the work was intense and copious. It didn’t help that I came onto the project late so for search tools, I was way behind. I found myself working 10-12 hours a day and frequent weekends. This is the pattern that you’d expect from a project that falls into the “land grab” scenario. At the time that Acrobat was being built, there were a couple of other competitors and we had to ensure that we dominated the space. The way to do that in land grab is to be there first and to be a dominant force that can’t be dislodged. That meant big goals, high standards, and long hours to meet them.

At the time, I decided that I was going to try to hack my sleep pattern and see what the minimum amount of sleep I could get and still be a functional engineer. I did a basic divide and conquer approach and found that I was good at 6 hours for months at a stretch. I wasn’t happy or healthy, but nobody else was either so I was in good company.

I would come home, take care of dinner (if it hadn’t been supplied at work), took care of my cats, watched some TV, read, and then went to sleep at midnight until the alarm went off at 6. I got up, took care of my cats, had breakfast, showered, and headed to work and got in around 8:30. Then I worked until 6:30 or later.

One hobby I fit in at that time was baking. There were wonderful farmer’s markets in the area with incredible produce. I would buy all kinds of fruit when it was in season and that which I didn’t eat or can, I would use for baking experiments. About once a week, when I got up in the morning, I would put together a quick bread, pie, coffee cake, or cobbler and put it in the oven while I was showering. When I got into the office, I put it out at Karin Jurcevich’s desk. Karin was the department admin and ray of sunshine. Her desk was a focal point for a lot of the group. I would set up the treat along with some plates, napkins, and utensils, cut myself a serving then walk the perimeter and give the people who were at their desks the simple message, “Karin’s desk” and watch the hordes descend.

At one point, I started doing more extreme baking experiments. I made two checkerboard cakes – one from box mixes and one from scratch using Joy of Cooking recipes (IMHO, the Joy of Cooking recipes tasted better, had better texture, and were closer in overall size. Also, the difference in time between making them from scratch or mix and cleaning up was negligible). At one point I made a cherry pie that included the Acrobat logo sculpted on the top in dough. It was fun. Somewhere there exists a picture of that pie. I thought I had scanned it, but I couldn’t find it.

Of course, I later found that the amount of sleep I needed in order to remain productive is far less my 6 hour experiment. That number changed when I had kids. I routinely had weeks on end where I had 4 hours a night on average. I looked like a wreck at times and there were days when I came in and announced that I’d had 8 hours of sleep. Over the past 3 days.

At one point, Rick Minerich and I went a conference in Las Vegas. Our flight out got canceled due to heavy thunderstorms and we got put up in a cruddy hotel and rescheduled for a flight early the next morning. We didn’t get into the hotel until late and we hadn’t had dinner, so we ordered from a nearby delivery pizza place and shot the shit until entirely too late. We had to set an alarm for around 4:30 in order for both up us to be able to shower and make the shuttle. I woke up before the alarm by 5 minutes and used that time to bring all systems on line. The alarm went off and Rick stirred and started trying to fiddle with the coffee maker. I got in the shower and was out, dry, and ready to go. Rick just got his first cup of coffee and was in a state of shock that not only was I up and, bright-eyed and bushy-tailed, but I had done all of that without coffee (hate the stuff). The answer was simple: my kids ruined my sleep patterns and getting four and a half hours of sleep was a luxury.

To be fair, Rick was decidedly not a morning person in general, so I don’t blame him for being out of it that early.

And while the sleep experiments were intellectually interesting to me, I can’t say that I recommend them. Over time, I’ve come to believe that land-grab mentality is dangerous for both the health of the employees and the precedent it sets for corporate culture. Since Acrobat, I have only ever done one all-nighter and have done my best to protect my teams from having to do the same. How do you do this? By planning better, making better estimates, and if pulling the extra hours looks tempting, quote the holy trinity of late software:

  1. Lose a feature
  2. Ship later
  3. Lose quality

Pick 1 – and if you don’t pick one, you’re going to get at least 1 and likely all of them.

I’m Old, Part L: Cultural Exposure

For better or for worse, most of software engineering in Silicon Valley in the 1990’s was very white and very male. A good thing, I guess, was that there was a fairly wide variety of restaurants outside of this prosaic culture readily available: Chinese, Japanese, Mexican, Spanish, Indian. I had found a nice Indian restaurant in Sunnyvale called The Empress of India, which I loved. They had a menu if you really wanted, but frequently Jeannie, the chef would come to your table and ask you if you were hungry and she would just cook for you. The beauty of this was that you didn’t get stuck in a rut and Jeannie always delivered wonderful food.

Lunch at Adobe was taken by most engineers in the company cafeteria, which was quite decent (certainly much better than the one at Bellcore), but wore thin soon enough. Sometimes you just needed to get out and have something else once in a while.

Once, Eswar Priyadarshan invited us to join him at an Indian restaurant on Castro Street that he said had very good home cooking. Eswar ordered for us, if I recall correctly, and insisted that we eat with our hand using thumb and the first two fingers to pick up food and/or using bread as a scoop. When I say hand, I mean right hand. He was very particular about that. It was interesting especially for Kevin Binkley who was a southpaw.  Eswar told him to eat with his right hand and Kevin asked him why and Eswar simply said, “sh-sh-sh.” waving at him in a gesture to put his left hand down. I was now curious. I asked, “but why…” and was also interrupted with “sh-sh-sh.” Fair enough. I learned later that eating with your left hand is considered disrespectful.

While this experience was not a huge thing – I firmly believe that teams are made better by having breadth in culture and gender. To learn and to respect another’s culture is to effect a change in yourself and how you look at the world and in turn how you look at problems.

In the Hitchhiker’s Guide to the Galaxy, Douglas Adams at one point uses the phrase, “fire missiles at right angles to reality.” It’s meant as an impossible task. Every job I’ve had has involved tasks that have been supremely difficult (albeit not impossible since I’ve succeeded at most). Sometimes to solve these problems it’s helpful to think beyond your own borders and prejudices. To be immersed in another culture is to give yourself a nudge in that direction. To remain homogeneous is to never grow.

Why Upgrades Aren’t Always Easy – The Horrors of Components

The software component market is great. If you’re trying to make an application, that needs a particular feature, you can often just buy a component that gives you that feature. In addition to buying a component you have other choices.

  1. Build it yourself
  2. Rely on the operating system or language support libraries
  3. Try and find an open source solution

#2 is OK if it’s there.

#3 is fine as long as it’s current and not abandonware.

#1 is usually only if you can afford the development or there is no other choice.

Buying or licensing a component is often the right solution and is often the cheapest thing short and long term because your time is spent integrating a technology not learning it, implementing it, debugging it and integrating it.

The problem that unfortunately comes up is the “can’t go forward, can’t go back” issue. I’ll give you an example of this. This did not actually happen – what actually happened is far worse, but it’s close enough.

At my last job, I created a general PDF library for manipulation and generation of PDF documents. One use of this was that I created an add-on to our OCR product line that let you create PDF documents with each page containing the original scanned image as well as the recognized text which was invisible so when it was highlighted, it made it look like the scanned text was selected. The quality of the output was typically better than that those bundled with OCR engines and it cost less and worked with all engines.

We had several customers who used it and were happy. The initial version had a page limit due to a bug, so I fixed that a few years later. By that time, the original OCR engine we licensed and sold was no longer in our product line and we weren’t allowed to sell it anymore. It had even lapsed from our nightly builds.

And a customer called up and needed more pages than we could offer. The problem was that the new PDF code, modular though it was, couldn’t just be put into the old version. It wasn’t compatible because of OS/language component changes. I couldn’t go backwards. We offered the customer a different engine running on our current version. Nope – because of other component dependencies, they couldn’t go forward. Great.

Because we were a customer-focused company, we ended up doing was figuring out how to get the new PDF code built in a way that it would work with their build, as is. It was a pain in the ass and fortunately, it was just a “one off” so the customer was happy.

Now consider all the machines running custom software with Windows XP that were recently affected by a worm that exploits a security flaw. In fact, I saw a system that runs image analysis on 3D dental x-ray pictures two weeks ago at my dentist’s office that still runs XP. Why? Almost assuredly it’s due to a “can’t go forward, can’t go back” problem similar to what I outlined above.

Why do these problems happen? Sometimes it’s because a component provider no longer supports an older product on a newer operating system. Sometimes it’s because a component provider has gone out of business. Sometimes it’s because a component depended on a bug in the operating system to function. Sometimes it’s because it costs too much to upgrade.

How can we minimize this kind of problem?

  1. Select component suppliers with solid support and a contract that spells out how long the component will be kept up to date and a good bug-fix policy.
  2. Select only components that are supported for the project lifespan of your product.
  3. When budgeting a project, require that there is a line item for operating system/software upgrades including field service.
  4. Consider a requirement that all source and build systems go into escrow.
  5. Include a system replacement clause in the event that a “can’t go forward, can’t go back” problem comes up before the end of the contract terms with training if need be.
  6. If you’re building the component, for the love of all things holy, don’t rely on OS bugs or undocumented features.

And if you are buying such a system, make sure that your vendor can do all these things. If you go with a low-ball contract that doesn’t include these things, then you are risking a much higher cost in the future – either for new hardware or for new software or both. If you can’t plan to pay for that from the beginning, how will you pay for it in a crisis?

I’m Old, Part XLIX: The Stanford Invasion

As I mentioned before, after Acrobat 1.0, there was a bit of shake up in the planning stages of Acrobat 2.0. There were plans afoot to get a UNIX version (but which one?), would DOS be a supported platform in the future, how would full-text search fit in, what about that whole OCR team anyway and so on.

I finally got to move onto the Search team full time. At this point, I was way behind the Windows side and had a huge amount of work to do to catch up. This meant that I was no longer working on the Mac viewer anymore. Mike Pell was doing several jobs (release engineering/installers, UX work, etc). So management brought in two young men from Stanford – Gordon Dow for the Mac Platform and Alan Padgett for the Windows platform. That they came from Stanford should not have been a surprise. There were a number of people on the team with a Stanford education (CS or otherwise), and there was always a lot of talk about Big Game. I neither went to Stanford not have ever had a particular interest in football, so that who thing was totally lost on me.

Both Gordon and Alan were monster coders with all the benefits of youthful energy and a fresh college education where C++ was the new hotness. When I left college a few years earlier, C++ was an idea that only existed in a bizarre compiler called cfront which turned C++ code (slowly) into C code. Microsoft had adopted C++, but there were still no full implementations on the Mac. We had used Think C for Acrobat 1.0, but it was fairly limited in its scope.

Gordon and Alan did a lot of the grunt work. For example, I had thunking mechanism that was friendly to C and Pascal calling conventions and hoisted it over to Gordon who refined it for use in Acrobat as well as later figuring out what to do when a C call into a C++ callback threw an exception and other mind-bending puzzles.

Gordon and Alan were an interesting pair – really kind of an odd couple. Gordon was very intellectual, introverted and reserved. Alan was smart, extroverted, and very outgoing. At various points, they shared work space. Gordon put on headphones because Alan programmed with sound effects. No seriously. It seemed like when he finished a particular phrase, it would be punctuated with something like “Shuh-keuh!”

Also brought onto the team was Steve Herskovitz as team lead, an experienced Mac engineer who had escaped from the ATM team. In the 2.0 product, a lot of the code was rewritten. My built-in search code, for example, got chucked. It was built for speed and light memory usage, but Moore’s law had eclipsed it entirely and instead Acrobat was going to have Daryoush Paknad’s Wordy algorithm (used for full-text indexing) instead. At one point, Steve called me into his office to go over some behavior that was in 1.0 and how it was in 2.0. He was showing me a highlight of some non-linear text and it looked awful to me compared to 1.0. I know why, too – I had sweated a lot to get the details of text highlighting correct and I took it as a personal affront to see what was happening to the product. I was being passive aggressive to Steve and he lost his temper – my fault entirely. He snapped me back to reality with one curt question: “are you going to talk or are you going to listen?” I reigned my behavior in and then tried to explain what I had done, how I did it and why and then left him with the task of emulating it as best he could. Not my problem anymore, but I learned a bit about my own pride in the process.

We had a small kitchen in our wing with a fridge stocked with soda, coffee and so on. Steve frequently brought in a grapefruit on which he wrote an abbreviation of his name: “Hersker”. One seeing one, I imagined that Steve was an agranthrope – a creature who turned into a fruit/vegetable during a full moon. I told him as much, but like many of my jokes, I think it fell on deaf ears.

It was hard seeing so much of my work getting thrown out wholesale, but most of it was for the best for the new architecture of 2.0. Besides, I was too busy trying get the Mac search engine done to spend a lot of time crying over those changes.

I’m Old, Part XLVIII: Leprosy

I spent a great deal of my youth hacking on the Apple II. I wrote games that used the low resolution graphics first and then later hires. Applesoft BASIC came with a brief vocabulary of commands for drawing in HIRES. You could plot points, draw lines, and there was also a very simple vector-based shapes that you could define. I had some trouble understanding the awful manuals, but my dad spent some time explaining it to me and then my brother Pat wrote a shape maker program that would let you draw the shapes out and get the data for you.

I wrote space games for the most part. Most of the games we had were space games and I took my inspiration from them (Jupiter Express by Gary Shannon, Star Cruiser by Nasir Gebelli, Rocket Pilot and Star Wars by Bob Bishop, Tranquility Base by Bill Budge). I found writing games in BASIC frustrating. The shape tables drawing routines were quick, but if you tried to do any kind of animation, the game flickered badly from the erasing and redrawing. Plus there were two shape drawing commands – DRAW which used paint drawing and XDRAW which used Exclusive-OR (XOR) drawing. For XDRAW you could draw and erase since two identical XDRAW commands are an identity operation. In this way, you could get cursors, for example, that moved over existing drawing on the screen without damaging it.

Interesting side note – the XOR drawing process was patented in 1980, but it was already in use on the Apple II at that time.

The problem with XOR drawing is that if you use XOR to move things around, you have have to be super careful if you’re using other drawing because you’ll end up with unexpected junk left behind. I figured this out early and gave up on it. I moved onto to using bit mapped shapes and custom blitters to draw them on the screen. You could use XOR here too, but for the most part I used a direct write – less to do per pixel and faster as a result.

One day after school, I biked to Summit to Stonehenge Computing to see what was new and to show off hacks. When I got there, there was another guy trying to get some help on his program. He had gotten a book on Apple graphics and was working his way through the examples and tried extending them. He had a program that made a stork walk across the screen from right to left while animating the legs. “I need some help with my code – my bird has leprosy.” I stepped in, “are you using XDRAW for the body and HPLOT for the legs?” “Yes” “OK – you can’t do that if they overlap, it will leave bits behind where they overlap. You need to use DRAW for the body and change the color to erase and draw.”

And this is how I met Jim Gittelson. Jim was a few years older than me and went to a private school. We started hacking together. We learned how to crack software protection on published games. For the most part, I dug though code to figure out where the game lived and what did what so we could end up with a minimal set of hunks that could be saved out and loaded back in again. Jim liked to figure out the track patterns to get Locksmith or Nibbles Away to copy them. Later, Jim started making dedicated hardware to help the tasks. For example, he built a peripheral card which had some ROM and ton of RAM on it. When you pushed a magic button, it took over the machine and copied chunks of system memory to his card. Then he reset the machine and wrote the memory out to disk. In essence, he was creating a snapshot of memory which could be loaded back. Clever.

At one point, we discovered RLE encoding and wrote some code the compress a ton of HIRES images onto disk (far more than could fit uncompressed) and then knocked together some code that would run in a loop loading each file in using RWTS (Read/Write Track/Sector) routines to pull in the data and decompress it live into the screen, along with code that was playing two voice music. It was a glorious hack.

Jim and I also were fans of the arcade game Defender. We made routine trips to local arcades to find the games and spend every quarter on them. Well, not quite. Jim collected Bicentennial quarters and always set those aside. The last time I contacted Jim, he had purchased a used Defender machine and had modified it so that it had a pause switch, that way he could start a game, pause it and come back to it later. Nice. I asked him if he still collected Bicentennial quarters. Yup. Still.

I still think it’s funny that a long friendship started because his bird had leprosy.

I’m Old, Part XLVII: Customer Support

When I started at Atalasoft, I was employee number 7. The company was engineer heavy for a tiny place, but that’s to be expected. We had Glenn, Dave, Seon-Young, Bill, and me all writing code. Bill was the CEO and routinely wore many hats. We had a goal of taking his compiler away from him and eventually succeeded.

One thing we didn’t have was support. Dave and Glenn did most of the support since they knew the products better than me. I mostly consulted.

Our sales grew and we were able to move into a new office. It was necessary; our office was packed to capacity – in the meantime, we had hired Sean and Lou into engineering and John into sales. It was also necessary that we had to take a more methodical approach to support. I wanted full-time support, but we weren’t flush with funds yet, so Lou put into place a process where we had engineers rotating into support. For each rotation, the engineer would answer the phones, take support cases from our Salesforce queue and also hit our forums.

Glenn had a rotation when he encountered a customer who not only he couldn’t please, the customer went as far as to royally piss of Glenn. This was a tremendously hard thing to do. Glenn was a quiet, reserved Texas transplant who was about as even keeled as you can imagine. I had never seen him pissed off before. He had put the customer on hold and explained to me just how awful this man was. I’m going to call him Jim Jones. Jim managed to rattle Glenn and Jim was still angry and demanded satisfaction. I had Glenn transfer Jim to me and I let him blow off some steam. I did what I could do to help him and he spent some time complaining about Glenn. After he was done, I promised him that he would never have to work with Glenn again. Ever.

After he hung up, I told Glenn that he would never have to talk to Jim ever again and that if he ever called in and showed up on caller ID, feel free to forward his call to me or to whoever else was available. I made sure that everyone else in the office knew the rule: if Jim Jones calls in, under no circumstances should Glenn have to speak with him.

We were talking about it a few weeks later and no sooner had we mentioned his name then Jim Jones called in again. It’s as if when we spoke his name, his ears started burning and he called in. We kept Glenn away from that call. I think the same thing happened again: we mentioned his name and he called in. It was like Voldemo…he who shall not be named. At that point, I started referring to him as Tim Tones or Slim Slones or Bim Bones or some other obvious alliterative variation on his name, just to avoid having him call in again.

From then on, whenever we got a new hire for any title, we made it clear that if Trim Trones called in, under no circumstances would Glenn speak to him.

Now, from Pim Pones’ point of view, I met his desire of not speaking to Glenn, but I couldn’t care less. I cared far more about Glenn. That I could meet both of their desires in one fell swoop was extra awesomesauce. The best part of it was that Thrim Thrones had absolutely no idea how much I was favoring Glenn here and as far as I know, he never will. Another satisfied customer.

I’m Old, Part XLVI: Acrobat 2.0

Acrobat 1.0 was an astounding achievement. The team, engineering, QA, marketing, documentation, and management put together a nice solid product that did what it said on the box. Upon release, Adobe took out a two page spread in the Wall Street Journal expressing how Acrobat would change the world. It was awesome to be a part of that and not long after we released, Adobe’s stock value doubled.

After the 1.0  release, people (finally) took vacations. Some people moved on. New people came on board. Management started to forge plans for version 2.0 and how it would shape the future.

Acrobat 1.0 cut a lot of corners in order to meet spec and ship on time. There was shared core code, but everything that the users saw was implemented uniquely on each platform. This was the most expedient way to build the product, but certainly not a long-term solution. In addition, there were so many things that we wanted to implement, but didn’t have the time to do without a huge effect on the overall product and its stability.

As we planned 2.0, Bill McCoy, a senior PostScript engineer, joined the team. He spent a lot of the time with pages and pages of source code in front of him shaking his head at the short cuts we took and the overall design.

Carl Orthlieb and I had put together a simple API that met our needs in terms of hooking a search engine into Acrobat and allowing us to present dialog boxes, create menus, automatically navigate to pages, highlight words, and go to particular views. It was compact and met our needs. We had hacked it into the existing framework and had prototypes up and running. At one point, Carl and I did a presentation to the 2.0 planning team. While Carl and I covered what we needed and how we got there, Bill was half listening and half going over pages of source code, trying to get all of Acrobat 1.0 into his head.

The meeting felt like we were being told, “oh, that’s so cute what you put together and thanks for your time, but we’re going to do something different.” It was demoralizing and delivered in a way that was ego-crushing, but so be it.

The problem was that what they had planned was way in the future and in the meantime, Carl and I had to build our product with a shifting, unstable platform. The big plan was a plug-in architecture that would make every feature within Acrobat accessible from plug-ins as well as allowing new features to be exposed to the product and other plug-ins. This was part of a design done by Bennett Leeds for Acrobat. Around that time, there was something like 4 or 5 different plug-in architectures being using in products in Adobe and nobody was willing to share. There were some issues in making the Acrobat work because of calling convention differences on each platform (and each compiler!).  At one point, when they were stuck on a problem of getting Pascal and C calling conventions to line up, I came up with a really nifty thunking mechanism that managed that problem. I liked it so much, I sent it off to one of my old college professors who was impressed that I had created a rarefied closure in portable C.

Eventually, the entire architecture was put into place. In addition to the plug-in model, it included a much more solid implementation of platform specific and platform neutral code, building on a decent MVC design with an OS abstraction layer that made sure that all the screwy things like file systems, menus, window handling, and so on were pushed way out into the fringe code. Even though it caused a great deal of pain an re-implementation of existing features, it was ultimately the right thing to do.

The process wasn’t without internal friction, but that’s a story for another day.

I’m Old, Part XLV: Caffeine Machine

In several ways, I do not conform to the typical programmer stereotype. One notable example is coffee: I can’t stand the stuff. It’s weird. I love the smell of coffee. I love the smell of fresh grounds. I adore coffee ice cream. I like beer with coffee overtones (in fact, as I write this I’m sipping a beer that I made which has just that flavor). For a while, I assumed that coffee was an acquired taste and that with enough repetition, I could acquire it. This had worked with other things like olives, broccoli, and cauliflower: I forced myself to each them again and again and eventually I learned to like the flavor.

 I tried this with coffee. Eventually, it got to the point where I started challenging people to supply me with the best tasting coffee. All of it tasty awful to me. One exception was a coffee made for me by a barrista at Cafe Borrone in Menlo Park. It was sweet and thickened with copious amounts of fresh whipped cream extruded from a piping bag. As I drank it, I realized that this was no different than melted coffee ice cream, so not really coffee then.
When I was working at Newfire, most of the other employees were coffee junkies. At one point, I think someone did the cost-benefit analysis of bringing in a coffee service and the numbers added up. One of my peers, Andy Hess, had a great deal of chemistry and could probably sketch out the caffeine molecule above at a moment’s notice. Heck, at one point, I asked him to draw out 1-1-1-3-3-3-hexafluoropropane diol and he could do that. But I digress.

The coffee service showed up at our office, installed an inline water filter, a bulk coffee maker and three bins of beans (maybe a fourth of decaf, but what’s point?) with integrated burr grinders. You shoved in the filter holder and it ground a bunch of beans into the filter. You pulled it out and put it in the maker and it filled the carafe with black glory. My peers went through coffee by the gallon. When my girlfriend came by, she would hit the coffee too.

Now, let me interject here that I’m the youngest of three boys. My brothers routinely beat on me and being the youngest, fighting back really didn’t do much good. So one of my survival skills was figuring out how to verbally stab at my brothers in a way that resulted in maximum irritation with minimum repercussion (or alternately such that even if I had repercussions from them, my brothers’ repercussions from my parents would be worse). This has stuck with me. For example, at one point, my oldest brother had gotten a Cooper Mini convertible. I teased him that it was a clown car, which I could tell was getting his goat. At one point, I taught my daughter (who was non-verbal at that point) how to sign CLOWN-CAR in ASL – and trust me, it’s really obvious. So he pulled up and she pointed at the car and signed it perfectly. Mike turned red and and blurted, “IT’S NOT A CLOWN CAR!” See what I mean?

When I observed my peers stumbling to the coffee maker at various points in the day, I knew that I could get an easy, friendly jab at my friends. Each coffee dispenser had a sticker on the front displaying the type of bean – you know “Columbian Marching Powder” or “Kama Sumatra” or something like that. So I put more accurate labels on them: “Bilge Water”, “Sludge”, “Watery Mud”. I figured that they would appreciate the accuracy of the labels.

I will say that I’m not immune to caffeine. For a while, I had a diet soda habit, but a couple years ago I did some reading on aspertame and other artificial sweeteners to the effect that they are pretty much just as bad as sugar. I gave up diet soda a couple years ago cold turkey. This turned out to be no big deal, which I also understand is also unusual. Many people experience very bad headaches after giving up caffeine. Me? Nothing. Once in a while, if I’m cold, I have some hot green tea to warm up.

I likely inherited all this from my mom. She also had no taste for coffee. My dad is a coffee junkie. In fact, when my mom was in labor, my dad decided that he was an old hand at the whole birth thing, went off to get some coffee. The coffee maker was a broken mess repaired with medical tape. My dad, an electrical engineer, went about repairing it and making coffee and ended up missing my birth. Not so much the other father-to-be in the waiting room whose last name was Holle and was apparently as used to his name being mispronounces as ours, so when I was about ready to be born, the nurse called in Mr. Hawley and he went in instead, much to the shock of my mom who was, well, on display.

But coffee for me? No thanks.

Swift: When Is a Protocol Not a Protocol?

Riddle me this, Batman: when is a Swift protocol not a protocol?

Answer: when it’s an Error.

I think the main thing about Swift types bugs me the most is that it’s type system seems to be a raft of special cases when there are perfectly good alternatives. Swift exceptions are a perfect example of this.

Let’s start with a quote from the Swift documentation from Apple:

In Swift, errors are represented by values of types that conform to the Error protocol. This empty protocol indicates that a type can be used for error handling.

Spoiler: this is an outright lie. Let’s prove it.
To start, let’s start with how swift represents protocols (in particular, this is the protocol list type)


It’s a block with 3 machine pointers set aside for the value represented by the Protocol (if the value does not fit in 3 machine pointers, there will be a pointer to allocated memory for the value), then there’s a pointer to the class Metadata for the value, and finally there is a list of pointers to protocol witness tables for each protocol that is represented. If Swift uses a protocol for the Error type, we should be able to test this side by side with another empty protocol and see what the compiler does with this.

public protocol EmptyProtocol {
public enum MyError : Error, EmptyProtocol {
                case itsAnError
                case itsAHorribleError
                case itsTheTotalCollapseOfCivilization
public func doNothing(_ a: Error) { }
public func alsoDoNothing(_ a: EmptyProtocol) { }
let x = MyError.itsAHorribleError

I compiled this and dumped out the resulting assembly and this is what I got:

; these 3 instructions set up the stack frame
0000000100000e40 push       rbp
0000000100000e41 mov        rbp, rsp
0000000100000e44 sub        rsp, 0x40
; rax is a pointer to stack memory for a protocol
0000000100000e48 lea        rax, qword [rbp+var_28]
; this is the protocol witness table - it goes into rcx
0000000100000e4c lea        rcx, qword [__TWPO5None17MyErrorS_13EmptyProtocolS_]
0000000100000e53 lea        rdx, qword [__TMfO5None17MyError]
; after this instruction rdx will point to the class metadata MyError
0000000100000e5a add        rdx, 0x8
; this is the value of itsAHorribleError, storing into x
0000000100000e5e mov        byte [__Tv5None11xOS_7MyError], 0x1
0000000100000e65 mov        qword [rbp+var_10], rdx
0000000100000e69 mov        qword [rbp+var_8], rcx
; this stores x into the stack memory
0000000100000e6d mov        r8b, byte [__Tv5None11xOS_7MyError]
0000000100000e74 mov        byte [rbp+var_28], r8b
0000000100000e78 mov        dword [rbp+var_2C], edi
0000000100000e7b mov        rdi, rax
; rax points to stack memory which contains this:
0x0000000000000001 -> value word 0 (itsAHorribleError)
0x0000000000000000 -> value word 1
0x0000000000000000 -> value word 2
0x00000001000043f8 -> Metadata for MyError
0x0000000100003710 -> Protocol witness table for MyError.EmptyError
; which is a protocol list type with 1 protocol

0000000100000e7e mov        qword [rbp+var_38], rsi
0000000100000e82 call       __TF5None113alsoDoNothingFPS_13EmptyProtocol_T_ ; None1.alsoDoNothing (None1.EmptyProtocol) -> ()
; now here's the set up for calling doNothing
0000000100000e87 lea        rax, qword [__TMfO5None17MyError]
; after this rax will have the Metadata for MyError in rax
0000000100000e8e add        rax, 0x8
; rsi will have the protocol witness table for MyError.Error
0000000100000e92 lea        rsi, qword [__TWPO5None17MyErrors5ErrorS_]
0000000100000e99 xor        r9d, r9d
0000000100000e9c mov        edx, r9d
0000000100000e9f xor        ecx, ecx
0000000100000ea1 mov        rdi, rax
; now this is different...
0000000100000ea4 call       imp___stubs__swift_allocError
0000000100000ea9 mov        r8b, byte [__Tv5None11xOS_7MyError]
0000000100000eb0 mov        byte [rdx], r8b
0000000100000eb3 mov        rdi, rax
// rax/rdi point to heap memory which contains:
0x00000001000043f8 -> Metadata for MyError
0x00000001000042a8 -> Protocol witness table for MyError.Error
0x0000000000000001 -> value of itsAHorribleError
0000000100000eb6         call       __TF5None19doNothingFPs5Error_T_            ; None1.doNothing (Swift.Error) -> ()

So here’s the dirty little secret about Swift.Error: it acts like a protocol, but in use it is not a protocol whatsoever. It’s a heap-allocated container that has information about the encapsulated type. In digging around, Apple appears to be claiming that this representation is so that an Objective C NSError and a Swift Error can co-exist in the same type. So if I call Objective C code which reports an NSError, the Swift code can treat it like a throw (and vice-versa).
I consider this to be a mistake on Apple’s part on a number of fronts. To explain why, we need to understand Swift exceptions a little better.
If I have a function in Swift that throws an exception, in theory there is a transformation available of the function to match Swift’s functional approach.

public func someFunc() throws -> SomeType { }
public enum ExceptionHolder<T> {
    case normalReturn(T)
    case exception(Error)
public func someFunc() -> ExceptionHolder<SomeType> { }

In this example, the second incarnation of someFunc is equivalent to the first incarnation except that instead of throwing, we’ve transformed it into a discriminated union of the either a return value or a thrown exception. For Swift, this makes complete sense because exceptions are not exceptions in the sense of typical OOP languages like C++/Java/C#. This is because Swift is a reference counted language and returning from a function abnormally wreaks absolute havoc on your reference counting. The solution to this is that an exception thrown is actually a normal return and that allows Swift to clean up references counts. The transformation above is more or less what Swift 2 used to do (at least I think so – I looked at it very briefly and put it aside and haven’t looked back). This is no longer the case. In Swift 3, Apple has deviated from the standard ABI yet again and has also broken from the functional approach of one argument, one return (see this previous blog). When a function can throw, it will return 2 values. One in standard registers (rax etc), and an exception in r12. If there is no exception, r12 is 0 and the standard registers have the return. If r12 is non-zero, it contains a pointer to a heap-allocated error object.
So why is Swift doing the wrong thing, in my oh-so-humble opinion? Well, first the calling conventions break the ABI. Second, Apple has decided to use one format for Swift.Error forcing all Swift code to suffer just to get interoperability with Objective C. Finally, it’s inefficient: transformations should instead happen at the point of interface, not for everyone. The reason being that in Swift error handling code, the typical pattern matching code ends up calling a library routine, swift_dynamicCast (bet you didn’t know that swift even had dynamic casting like this) for every single case and the way that swift_dynamicCast it written, this particular type gets forced through several cases that are guaranteed to fail and ends up eventually running through a recursive call. So much wasted time.
Now, the Swift ABI is not finalized yet and this may yet change, but jeez.
This doesn’t even touch on the problem that Swift exceptions are (still) fundamentally broken from a language point of view, but that’s a topic for another day.
So we see that Swift exceptions at first look appear to be a protocol that acts similarly to exceptions in other languages, but in implementation are a misfeature that breaks the language model and the ABI and hurts efficiency.