Author: Steve Hawley

I went to Oberlin College. It was an intense experience and one that I really enjoyed. I went there because a high school friend of mine, Kazumi Umeda, had gone to Oberlin a year ahead of me and on his fall break he came back to visit and told me directly that I needed to go there. I took it almost as a royal command.

Initially, I thought I would be a math major as there was just a fledgling CS department, but the major was announced while I was there and I was the 5th student to declare.

In the program, I met a number of people who had similar backgrounds to mine and wonderful senses of humor.

At various points, we played jokes on one of the CS/Math professors, Mike Henle, who was teaching an assembly language class.

I think it was a bit of a passive-aggressive response to some of the assignments. He assigned us the task of writing a program in VAX assembly that would maintain a family tree, which is a fine high-level language task, but a really crappy assembly language task. I wrote mine in Pascal first, then hand compiled it because while I probably could’ve done it from the ground up, I’m lazy.

At any rate, three of us: me, David King, and Tom McHugh had about enough of the assignment, so we walked into Mike’s office and sang a rendition of  a song called, “I Feel Tired” to the tune of “I Feel Pretty” from West Side Story.

At some point, we created our own ficticious computer with a complex instruction set that included such instructions as JEO (jump and execute operator) and the very important BVP (be very paranoid). Before we did anything to Mike, we usually told him “Be Very Paranoid”.

One day, we decided that we were going to throw him a surprise birthday party in class. We went to a bakery and got a cake that read “Get Well Soon Grandma” crossed out with “Happy Birthday Mikey” on the edges. We said we got a discount because Gramdma didn’t make it. We put 4 candles on it in a row and decided that this was his 6th birthday and lit 2, because binary. While we set up, the head of the math department walked in and sat down saying, “I heard something good was happening today.” Uh-oh. Meanwhile, Mike had run downtown to get a bag of lollipops, hoping to appease us.

He showed up, we sang “Happy Birthday” and he turned bright red and walked into a corner and stood while we finished. We cut the cake and served everyone. Class started on time.

Another time, around 11:00 one day, we decided to steal his door. We all had pocket knives so we undid the screw on the hinges and pulled the door and did a fantastic job of hiding it in plain sight. We waited for him, but he was late for office hours, so we decided to have a quick lunch and come back. When we came back, the head of the CS department and another professor were madly searching for the door. When the head spotted us, he said in a very stern voice, “OK – What did you do with it?” I pointed at the door, which was in plain sight. “Put it back.” “Couldn’t we just wait for Mi-” “Put it back now.” Fine. Whatever.

We put it back on and it didn’t close right. It was really sticky, so we took it off again and fixed it.

Right then Mike showed up. I said, “Mike, hypothetically, if you got here and your door was gone, how would you feel?” “Pretty angry – unless you guys were here.”

Good.

That means we read him right. “Mike – was your door sticky?” “Yes – it was.” We quickly wrote him a bill for fixing his door. He never paid it.

Over time, I’ve played a number of jokes on people I’ve worked with (and gotten my fair share in return). One of my recent co-workers told me that when I play a joke on someone, the joke is so thorough that the victim has no choice but to understand that s/he is loved.

I’ve been more or less consistent in my overall approach and when I worked at Atalasoft, I codified it for my peers: play jokes, but they should be in good fun, cause no permanent harm, and you should be able to clean them up in 15 minutes or less. In college, we made sure that nothing we did would prevent class from starting on time and that nothing was mean-spirited. After college, there was one April Fool’s joke that was an exception to that, but only one that I remember.

The printer division at Adobe had a structure wherein a manager was usually associate with a handful of manufacturers who wanted PostScript on their printers. The managers would have a single engineer assigned to each printer (or in some cases to each class of printer) who would port PostScript to that device.

Sometimes, projects ran long and when that happened, other engineers would be brought onto the project to help bring it to completion.

In one case another engineer, Jeannie, was working on a product for HP. They wanted a cartridge that would plug into any one of the HP LaserLet III series of printers and provide PostScript.

The problem in this was there were three separate products that needed to use this cartridge, the IIIp, III, and IIID. All of them are Canon engines, but the IIIp is the Canon LX engine, which is the “low cost” laser printer. The latter two were based on the Canon SX engine, and the IIID was a duplex printer, meaning it could print on both sides.

My boss had a formula for estimating the length of time of a printer project that depended on the number of complications in the project. 3 printers on one code base was one, running off a cartridge (where PostScript didn’t own the machine) was another, duplex was another, an LCD front panel was another. There may have been more, but that was plenty.

I was assigned to work on the project to help get the printer out the door. Unfortunately, adding more engineers always results in an immediate slowdown in the project because it takes time for new engineers to come up to speed and the expert will be busy fielding questions instead of fixing bugs.

In this project, we had some interesting conflicts because of different QA departments. HP had theirs and Adobe had theirs. Often, an printer manufacturer would approve a product that Adobe would reject or vice versa because of difference in acceptance criteria. HP, for example, had a metric where bugs were rated in severity from 1 – 10, where 10 was most severe. They would accept a product when the sum of the squares of the severity of all the bugs was less than or equal to 100. In other words, if there was 1 10 bug and no others, they would accept it, or 100 1 bugs.

I was assigned an interesting bug. When a page was printer in duplex and printing a legal-sized page, there was a long delay from when the engine fired up and when the page fed. If I recall correctly, the delay was close to 4 seconds.

The problem was that in this printer, the back side of the page needs to be printer before the front side of the page and you don’t know that the page is duplex until the very last moment. And then the full page image needed to be rotated 180 degrees.

In my examination of the code, it appeared that it was the rotation code that was the culprit, which was a shame, because it was very pretty code. I think it was originally written by Amy Fasnacht.

char *start = fmbufstart;
char *end = fmbufend;
while (start <= end) {
    char temp = bitrev[*start];
    *start++ = bitrev[*end];
    *end-- = temp;
}

It was a tight little loop, but when run on 1.4Mb of frame buffer on an 8MHz 68K, processor, it took a big chunk of time.

So how do you make this better, when the loop is so tight?

The problem is the while loop. That test will get run 1.4 million times and it will fail 1.4 million times. So the first this I tried was Duff’s device. I will point out, that I learned Duff’s device by reading some of his code when I was in high school working at Bell Labs (thanks, Tom!). This brought the time down to 2.5 seconds. We ran that past HP and they were still not happy with it.

,Damn.

A brief interlude on compiler design. If I recall correctly (and it has been more than 25 years), this particular chunk of code was compiled using the Sun 68K compiler, which was decent enough, but it couldn’t compile PostScript correctly with optimization on, so we were forced to ship with non-optimized code. In addition, when compilers generate code, they often use a subset of the available instructions because that’s what the back-end code generator chose. Unfortunately, that usually means that there are a number of instructions available that the compiler will ignore that may be very convenient for certain tasks. The 68K had a very useful instruction, SOBGTR, which the Sun compiler wouldn’t use in debug. It means subtract one and branch if greater than zero. Using this instruction with carefully set up registers can cut the overhead of loop way down compared to a typical SUB/DEC/BLT triple.

Also, I would like to say that I loved working with the 68K processor and was disappointed that it has more or less disappeared. It was easy to write a compiler target for it, the instructions were mosly sane and had relatively few bizarro exceptions, it had a buttload of registers at hand, and it was pretty easy to read and write. Beyond that, I could look at a 68K disassembly and figure out the source language, and in some cases the compiler, fairly easily.

Back to this task. I rewrote the routine in 68K assembly, unrolling the loop and using SOBGTR to drive the loop. I got the execute time down to 1.5 seconds, and we sent that off to HP.

Still not acceptable.

Damn.

I had one other trick in my bag. The code, as written, operates on bytes. This printer used a plain-Jane 68000, which does not like to operate on bytes as it has a 16 bit data bus. It really wants to operate on 16 bit words. The problem is that the bit reverse buffer for 8 bits is 256 bytes, but for 16 bits it’s 128K. Since we were on a cartridge, we had a limited budget for space, and using a 128K buffer would blow us past it.

HP had the memory partitioned off and they had a no-man’s-land in the address space that they had set aside aside for just such a contingency. We made a request for the extra memory and it was granted. Now, with the the huge reverse buffer in play, I could operate on 16 bits at a time. In addition, I put in one other optimization. On a typical printed page, most of the frame buffer is white, which in the PostScript world is a 0 word. Fortunately, when you read memory in the 68K, it sets the condition codes, so I could probabilistically skip the table look up on 0 words.

The final result was .5 seconds, and HP accepted it. And thank goodness, because I had no idea what I was going to do next.

I really liked doing this kind of work and when the group worked together well, it was fantastic.

I remember that this project carried in to Hallowe’en and we had a weekly conference call with HP in Boise. The manager, John Gaffney, showed up in the conference room with a package of wax teeth and lips for every engineer in the project.

John wore his teeth up until every moment when he had to speak. Jeannie was wearing big red wax lips and was nearly apoplectic trying to keep herself from laughing.

I have a theory. Actually, I have a lot of theories, most of them wrong, but this theory there exists a specific Valhalla for software engineers. When we die, we go there and drink mead/beer/wine and tell tales of engineering prowess. This particular one will be one of mine.

 

On modern CPUs, there is a lot we take for granted. Memory abounds and we’ve more or less settled on a set of addressing modes that make it reasonably straightforward to implement high level language features.

I started coding on an Apple II+ that my dad bought in 1979. We hooked it up to a TV up until the point where my dad picked up a black and white monitor from a computer swap meet. Initially, I started with BASIC, but grew frustrated with its (lack of) speed.

One day, when I was home sick, I sat down at the computer with a copy of the Apple II Reference manual and started looking at the routines that were in ROM. The one I started with was called COUT and lived at address $FDED. The manual said that the character to print would be in the A register. OK – easy enough. There was an instruction to put a value in A, called LDA and the variant I wanted was 0xA9, which I found out by looking it up in Programming the 6502. I decided to print HI, so I started it at address $300, which had, more or less, 256 bytes of space to work from. I dropped into the monitor and entered my code:

300:A9 C8 A9 C9 20 ED FD 4C 00 03

which turned into:

300: LDA #$C8 ; Put ‘H’ in A

302: JSR $FDED ; call COUT

305: LDA #$C9 ; Put ‘I’ in A

307: JSR $FDED ; call COUT

30A: JMP $0300 ; Jump to 300, the start

I ran it an the screen filled before I knew it. It was amazing. I was hooked on the speed.

Unfortunately, the 6502 is a fairly broken CPU in a number of ways. It’s got registers you can use as indexes, but they’re 8 bit, so you can’t address anywhere near the whole address range. It’s a serious hassle. Also the stack on most 6502 machines maxes out at 256 bytes, which means that things most programmers take for granted like using the stack for passing parameters or for locals is pretty much off the table. You also can’t nest functions particularly deeply and no way are you going to do recursion. Heaven forbid.

Since I started writing assembly by typing in raw bytes, I had in intimate familiarity with the encoding of the instructions. I found that I could make up for weaknesses in the hardware architecture by rewriting code on the fly. Don’t have a 16 bit address register? No problem – just rewrite the high byte of the target address of the instruction you want to change and ta-da: 16 bit indexing.

I also found out that this is pretty much the fastest way to get things to work, especially when I was writing graphics routines where the 1MHz CPU was just not quite fast enough.

When I got to college and took CS courses, I took an assembly language course and the professor spent a small amount of time talking about self-modifying code and said with reverence that John Von Neumann often wrote self-modifying code. Wait? Was it supposed to be hard? Seriously? I had been writing self-modifying code since I was 13.

A few years back, there was a thread on Metafilter about the Apple II, and I fired up an emulator and knocked together this little chunk of code for grins, which naturally, contains self-modifying code – in this case to get around the limitation of stack space for parameters, a dearth of registers, and to avoid the cost of doing arithmetic in the middle of a loop by precalculating the invariant values and stuffing them into the right places in the code.

When I was in high school, my French teacher, a truly wonderful woman, was hired by Max Mathews to teach him French. She started an extracurricular club call The Erasmus Club, which was a nice haven for the geek set in my High School. Max did a demonstration for the club of one of his research projects, an electronic violin that was designed to sound as close as possible to an Amati violin.

Through this set of connections, my French teacher arranged (or at least I think so), to get me a job working at Bell Labs with Max Mathews. My job was violin maker. I measured the parts in his prototype, made drawings, and then made many copies. I think I made something like 14 or the instruments.

I loved the job. I got to work in an illegal machine shop within Murray Hill Bell Labs, work with wood, aluminium, and nuts and bolts.

But wait – there’s more! After working for a little while, I asked for an account on their UNIX systems, and it was granted. I had accounts on two of the Vaxes: Alice and Rabbit. I played a lot of Rogue, learned the ins and outs of UNIX, had email, and a compiler.

Oh and did I mention what was down the hall? The UNIX room. This is where I met Brian Kernighan, Dennis Ritchie, Rob Pike, Tom Duff, Andrew Hume, Ken Arnold, Joe Condon, and Robert Morris (just to name a few). I poked my head into the UNIX room frequently to see what they were working on.

It was truly a formative experience.

I also would walk the halls of Bell Labs and see what else there was to see. Once I found the chemical stock room and for grins, I paged through the catalog of things you could get. It might as well have been the periodic table of the elements. Hmmm…I paged to the P’s. Yup. You could, in fact, order Plutonium, but there were a number of annotations indicating that you pretty much needed military clearance and a signature from God in order to actually get it. Fair enough.

I know that I wasted a lot of time there, but in the process I learned so very much and was very lucky to have the opportunity to do so.

I will also mention that there is one job that I refuse to remove from my resume, and it’s Violin Maker.

While I was in college, I took a year off and worked as a contractor at Bell Communications research. During that time, I found out about the merit review process at Bellcore. The first thing each employee does is fill out a form that was pejoratively called an “I Am Great Form”. In this form, you have to sing the praises of everything you accomplished in the previous year. Next your manager collects the forms and then all the managers cloister themselves in an office and order the forms, sorting by greatness. The greater your ranking, the greater the raise you would get.

It’s a horrible process and I bet that the managers hated it even more than the employees.

My friend Dan and I had a brilliant idea. The office that the managers were in had a speaker which was hooked into an experimental phone system that could pipe you notifications of phone calls read by a DECtalk speech synthesizer (samples, here). As a user of the system, you could either take the default, which was “call for <username>”, but a lot of people changed theirs and put in the magic escape codes to change the output voice. One user had a high female voice say, “bridge to Captain Kirk”. Another had a gravelly voice saying, “Yo! Talk to me baby!”.

This is neither here nor there. What was important to us was not that there a speaker, but that there was a wire going to a speaker that we could try to use as a microphone. So we found the terminus of that line in the machine room and hooked up an amplifier and tried to get sound out of it. No good. We could get hints of voices, but nothing clear.

Since there had been a big computer recently installed (I think it was a Pyramid Technologies machine), there was a correspondingly big cardboard box. Dan and I cut one side panel off the box, got a big marker and wrote “SPEAK UP! WE CAN’T HEAR YOU!” on it. Then we got a microphone on a long cord and some long lengths of wire to tie to the sign. Dan and I climbed out onto the roof and went to the corner above the office full of managers, then dangle the sign and the microphone by the window. Then, since it was a corner office, we went to the other side of the corner and dangled there for good measure.

We heard later that they liked that quite a bit – broke up the torture.

It turns out that managers have to go through the same process with their bosses, so in this case shit rolls uphill.

At the time, I had written a raster driver for an early HP ink jet printer so you could print graphics from windows in the in-house windowing system, MGR. I did a lot of hacking on MGR and one of the early fonts looked like kid’s handwriting. I used that for many of my windows just for grins.

My manager saw it and that I could print with it and asked me to print his “I Am Great” form in that font. You dance with them what brung ya, so I obliged. He also asked me to change the title from “Performance Review, 1985 – T. K. Landauer” to “WHAT I DID LAST YEAR BY TOMMY LANDAUER.” For good measure I popped a couple bitmapped fingerprints onto the page underneath the text. He was thrilled.

As a side note, most of the department had standardized on Sun 3 computers, which were OK when they ran, but were prone to failure. I personally witnessed my machine shit the bed 3 times. Each time, the Sun maintenance tech who was in the building would come in and swap the motherboard, but would put the Ethernet MAC address ROM from the bad board into the new one. After the third time, he told me that when a motherboard failed, it was sent back for testing and if it tested green, it went back out for use as a replacement. They wouldn’t scrap it unless it had failed a certain number of times in the field. At least that’s what he told me. He could have been yanking my chain.

Still, a manager on my floor, Lynn Streeter, had a conversation one weekend with a non tech person. I’ll write it the way that person had heard it.

Non-tech person: “How are things going?”

Lynn: “Well, my son died last week.”

NTP: “I’m so sorry!”

Lynn: “It’s OK – I’m getting another one; I’m going to name it Bullwinkle.”

Lynn picked up the look of abject horror on the person’s face and explained the Sun/son issue.

Of course you then have to explain the whole computer naming thing to people as well. This department used a bunch of different conventions. For example, file servers were words that ended in ‘day’. The days of the week were the first 7, there was holiday, one-a-day, Farraday, etc. One set of machines was named after Dwarfs that never were: Filthy, Grubby, Sleazy. When 3 NeXT stations appeared in the lab, I insisted that they be named Dacron, Orlon, and Rayon since they looked and acted so differently from everything else.

My Sun was named Peabody.

Printer manufacturers had some interesting challenges in front of them in the 90’s. For the most part, Canon was one of the big print engine manufacturers and if you were a making a printer with PostScript using a Canon engine, you were competing against HP, DEC, Canon, Apple and probably a few other big names.

You need to look for ways to differentiate your product from other manufacturers. If you bought into Adobe PostScript, that was one thing, but that raised the cost of your printer. In for a penny, in for a pound, so lets make the PostScript implementation more complicated by having a front panel UI and a whole bunch of custom PostScript operators that were “value add” (and for which no driver except maybe the one in the manufacturer’s proprietary print server software would use.

Still, we saw some wacky printers that should never have seen the light of day. One was worked on by my co-worker, Kathie. This printer had the code name ‘Jolt’ and was a designed by Rube Goldberg.

Our boss, John, had a technique for estimating the time that it would take to complete a PostScript printer implementation. It was, if I recall correctly, based on a log scale on the number of complications in the project. Jolt had so very many.

First off, let’s talk about the controller. A printer controller is an embedded system with hardware for reading input for print jobs, controlling the print mechanism, and reading user input. Many manufacturers used very common controllers and Adobe very often would have reference designs that printer manufacturers could use. If they chose a reference design, the time to market was usually quicker. Not this company. Oh no. They chose the Weitek 8220, which was an early RISC processor with a lot of high ideals which interfered with practical engineering. The first was that it ran on a Harvard Architecture model which meant that there were two separate address and data buses. One of code and one for data and never the twain shall meet. Unfortunately, this meant that it was next to impossible to have any kind of reasonable debugger for it, since debuggers often modify existing code to replace existing instructions with break instructions.

Next, the printer was color (there were not many at that point), and instead of a toner or dye sublimation printer, this one used crayons. It had four hoppers that were similar to Pez dispensers that were loaded with color pellets. When you turned on the printer, the print head had a heater that kept 4 separate molten pools of wax that were shot out of the print head onto the paper.

Paper was sheet or tractor fed, but the hybrid design was clumsy. In order to keep the paper flat against the platen, the plated had a row of holes in it that were connected to a modified pump that acted as a vacuum, rather than a pump. The actual device in the printer was an actual aquarium pump.

When the printer sensed that it was low on crayons, it would send the print head to the hopper to feed itself. If you made the mistake of turning the printer off, it would require a much longer start up time in order to melt the wax and very often the jets would get clogged. Therefore the printer required special cleaning paper which was a piece of paper with a puffy diaper in the middle and some markings on the edge that the printer would recognize when it was fed in that would send the printer into “cleaning mode”. At the point, the printer would attempt a technicolor Montezuma’s revenge onto the diaper complete with a back and forth wipe at the end.

It was a miracle to see it work.

Meaning, it rarely worked.

I felt very sorry for Kathie, and if I recall correctly, this was the printer that made her decide that maybe printer development was not her cup of tea. I was sad to see her go.

Working at Adobe was an astounding experience. When I was stuck on a problem, I made a point of walking around and seeing what other people were working on. It was a good way to meet people and to see what was going on in the company beyond my cube. At this point, I’m working at home and miss the ability to do this.

At one point, I stumbled across Joe Holt, who was a developer on Illustrator for the Mac. I was always grateful for his friendship as he and I got along so easily. It was a commonality I’ve found in just a handful of people. At that point, I owned an Asteroids machine which was living in my dad’s house and I had the discovered that an Arcade in Oakland was closing, so I called Joe and on a Saturday, I got some cash, rented a truck and appliance dolly and Joe and I went up to scout out what they had.

We settled on a Robotron machine, paid for it, and hauled it back to Adobe where we set it up in a storage closet. We opened it up and set it up for free play and played a bunch of games. I returned the truck before it got too late and headed home.

Joe stayed late and pulled the ROMs from the game and put them into a ROM burner/reader and pulled the code. This is what I mean by resources. Since Adobe did a lot of embedded development, there were a number of machines at our disposal for reading/writing ROMS and they could handle the obsolete ROMs from the Robotron. Joe found a 6809 reference manual and wrote a 6809 disassembler in C and ran it on the ROMs, producing an ASCII listing which he put in a shared directory on his machine. Over the next few months, we went through the code and started commenting it and we figured out how the game worked at a fairly high level.

We had gotten stuck on some of the hardware addresses that were being used in the game, so I found the mailing address of Eugene Jarvis and asked him for some hints. He sent me a listing printed with an impact printer.

I found out the Robotron’s predecessor, Stargate, would run on a Robotron, so we bought some vintage ROMs from a local surplus house, downloaded Stargate, and burned some ROMs. Another engineer, an avid wood worker, built us a control panel that we wired up to play Stargate. We also set up Joust.

Since we had access to a hardware lab, we built a little panel of switches and buttons that we could wire up to a spare sound board and play with all the sounds in isolation of the game. Eventually, I ended up writing an emulator for the sound board and was able to get the sounds in realtime on a Macintosh Quadra.

It was so fantastic to be able to work and play in the same place. The reason it worked out so well was that Adobe had a very open and flexible approach to the hardware lab and so many of the engineers were willing and/or excited to talk about what they were working on or playing with.

I was very lucky on a number of fronts in my career. In many things, it was a “right place, right time” confluence that worked out in my favor. For example, I knew 6502 assembly language as my second programming language and as a result had a fairly deep understanding of how software works at the lowest levels. I got involved in a program at Bell Labs in high school where I learned C one night a week. By the time I started college, I was already familiar with many of the concepts before I encountered them. And for no good reason, I did a private study in college on the language FORTH and did a reference implementation for the Macintosh.

Similarly, while I was in college, I worked at Bell Communications Research, doing work on Macintosh machines and Sun workstations before I decided to see what other opportunities lay before me. I managed to wrangle interviews at Apple and Adobe and thanks to some pull I had with an employee at Adobe (hey Curtis!), I landed a job there where, to my great relief, I discovered that development was being done on Sun workstations. Of course, I had no idea what PostScript was or how to do embedded software development, but I grabbed copies of the all the PostScript manuals on my first day and read up over the weekend. Oh hey – FORTH and PostScript share a lot in common. I can deal with this.

I took over on a project that was owned by DEC to work on their DECLaser 2200 series of printers. I was taking over the project from a woman who was going on maternity leave (hi Kathie!). On the day I started, she was in the middle of an early release and had no time to do much information transfer.

I took over, bouncing between my cube and her cube, trying to learn the ins and outs of not only PostScript, but also the sets of printers (there were two models that were going to be built from one code base), and their controllers. The underlying hardware was running on a National Semiconductor NS32CG16 processor which was sold to DEC as being a much better choice than the competing Motorola 68K series. National was lying through their teeth on a number of fronts, not the least of which was because the processor, which had an optional floating point unit, was designed in such a way that when the FP unit was installed, half of the machine registers (IIRC) magically went away, being dedicated to the FP unit. This meant that if you had to compile code to run with and without the FP unit, you had to assume that you had half as many registers as you might have. It made the code nearly impossible for a compiler to optimize, so the end result ran OK. Just OK.

In the middle of the project, I was given an awkward assignment to work with engineers from National to try and address the problems DEC was seeing. It involved a lot of benchmarks and we were stuck using GCC for the main compiler, but the National engineers were telling us that if we used their compiler, we’d get better performance, but the output file format was incompatible with the rest of out tool chain for making code that would embed on this controller. I was required to die on that hill.

Still, in the process, I learned a lot about development and the benefit of having a solid QA department who was on your side. To keep organized, I kept a binder where I put copies of the bug reports. I had them in sections for done, in progress, and needs to be done.

One of the problem with printer bugs is that sometimes it takes a long time just to reliably reproduce a bug. For example, there was a bug that was reported that would occur once in a while with only a certain set of serial settings when the printer was printing in duplex (both sides of the paper). I think I ended up using close to an entire case of paper finding and fixing that bug. It turned out to be a bug in the low-level code that handled serial input through a ring buffer. The stars had to align just right in terms of rate of input and interrupts for that one to happen. I found the bug and made sure that it made it into the main code base so that future printers wouldn’t have the problem. This was the “right thing”, but it turned out that it didn’t really matter because most of development at that point was focused on the next version of PostScript, which had none of the old I/O code.

Working on that printer was really entertaining because it was my first exposure to an ICE (In Circuit Emulator), which was a chunk of hardware that plugged into the CPU socket and acted like the main CPU, but let me set instruction level breakpoints, look at registers and memory, and generally figure out what was going on.

In the process of working on this printer, I learned a lot about corporate/political processes. There was a bug that opened up on the printer wherein the PostScript code would draw a solid 50% gray box on the page then render an image of a teeny-tiny helicopter scaled up to fit the box. When rendered on the page, the helicopter was supposed to cover the entire gray box, but on my printer (and in fact on all PostScript printers), there was a gray hair line on two edges – either the top or bottom and the left or right. Math said it shouldn’t happen, but it did.

I spent a week on this bug. After a lot of hacking and fiddling, I was able to determine that it was due to floating point error and that the location of the hairlines was predictable based on the floating point implementation and the margins settings of the print engine. DEC accepted my analysis and closed the bug. My boss was shocked – DEC opened this bug on every printer they worked on with Adobe. It was a tradition. Oh, we’re working with Adobe on a printer? Let’s just open this bug up on it. Suck it DEC, I have the answer!

At one point, as this project progressed, I had gotten a little behind and DEC was getting unhappy. They had a number of high priority bugs that they wanted fixed and were worried about the schedule. In total, there were around 40 bugs open on the printer. I managed to sort through their high priority bugs fairly quickly and then pounded through the rest. We had a conference call with them where they were assuming that I hadn’t fixed the high priority bugs and were ready to do some intense horse trading on them. When they asked about the status, I said, “I fixed all the bugs.” There was a long pause. “All the high priority bugs?” “No,” I replied, “all the bugs.” There was another long pause. “Is that OK? I mean, I’m pretty sure I can put some of them back if you’d like…” Peals of laughter. They faxed me a thank you card later in the day.

Unfortunately, there was one solid fuck-up in the printer and it was all on me. The printer had a tiny amount of non-volatile RAM that was used to store settings like the number of pages printed, the serial settings and so on. Through many changes and revisions, I had managed to subtly break the NVRAM code such that it worked great as long as the NVRAM had been written to once. If it had never been written to, the printer ended up going to a dreaded procedure named “CantHappen”, which would make the printer reboot. So when a brand new printer had the PostScript cartridge installed, it would boot, then try to read NVRAM, fail and reboot. Lather, rinse, repeat. DEC was in a panic about this because the printer was in production and the ROMs that went into it were already “masked”, which means that instead of burning each ROM individually, they were making the chips directly with my bug in them and they had already paid for thousands of them. I came up with a solution which was a chunk of code that would initialize the NVRAM and print a happy message on the front panel. Adobe footed the bill to have this code put onto a one-time-programmable cartridge and shipped with all the version 1.0 printers. You put in the OTP cartridge, it formatted the NVRAM and made it so that the version 1.0 printer would work. I wasn’t proud that that bug had made it into final release, but DEC and Adobe were amicable about the solution.

There’s no doubt that credit cards have overtaken other forms of payment, especially for online transactions. Unfortunately, many sites that accept credit cards have payment systems that have security flaws in them, not the least of which include storing the credit card information. This is supposed to make it easier for repeat customers – lower the barrier to entering payment information and it will make it easier to buy something next time. In reality, it just provides another vector for attack.

My credit card has been replaced 3 times in the past year and a half, and every step has been done incorrectly.

The first misstep is how I find out about it – by failing to make a purchase, usually at a very inconvenient time. The credit card company has at least 3 different ways to contact me, some extremely expedient, but they don’t use them. Instead, they require a failure and a subsequent angry call from me.

The second misstep is that they need to send me a replacement card through express mail. Honestly, they could have already sent the replacement at the time of failure.

The third and most important misstep is that every time this happens, I ask what the cause was. Who was responsible for causing this inconvenience to me? I have been told every time that my card issuer has not been informed and they can’t tell me. The problem here is that this offers me no recourse – all that can happen is that I have to go through the first two missteps repeatedly.

The reason why this information is not provided is two-fold. First, the companies that are responsible are typically big targets. They think that it would be marketing suicide if the world found out that their data was compromised. Second, the companies are big enough to grab the credit card companies by the short and curlies to require them to keep it quiet.

Hiding a failure in data security makes the problem worse, not better.

Instead, what should happen is that the companies should own their mistakes. Publish the cause, publish the process by which it will be fixed, publish the progress, publish how that class of error will be prevented in the future.

Hiding the scapegoat also is a break in the free market. Without the information, I don’t have the agency to make a wise buying decision. Should I eschew credit at this establishment? Should I use paypal? Should I use virtual credit card numbers? Should I decline to create an account, but just use a guest checkout? Should I avoid the establishment until it has a fix in place?

Coding is hard, I get that. It’s been my profession for most of my adult life. I’ve made my share of bugs and mistakes. I’ve found that the best way to handle any of the issues has been to be honest and up front about it; to understand the problem fully and propose fixes. Hiding the problem – even with a fix – is sub-optimal.