I’m Old, Part XXVII: Engineering Maladies

The Acrobat project was an incredible chunk of engineering done in a very short period of time. It was seen as a “land grab” project so engineers committed a great deal of time. Or were committed to a great deal of time. It was nuts. Many weeks on end, I was in the office 60+ hours a week for weeks on end. I know that there were a number of heated discussions between management and human resources because a huge number of engineers on the team were at the limit of time off accrual. That’s a bad thing. It was worse that HR’s request to get people to go on vacation fell on deaf ears.

220px-Harveyballs_red_black_modification

At one point, I was at lunch with Nathan Graham, Alan Wootton, and Mike Pell. I was irritated because I had an involuntary twitch of my eyelid which was driving my batshit insane. Nathan noticed, and said just a little too cheerily, “oh, you have the eye twitch!” In talking with this little group, we discovered that we all had a lot of things in common to some degree or another: stomach pains, loss of appetite, grouchiness, loss of sleep and so on.

I found it so distressing, that I went into Alan’s office, erased his white board and drew in a grid. As an aside, I often did this. I’ve discovered that I think much better with white boards. I’d be happy with an office that was floor to ceiling white boards. That hasn’t happened yet. Back to the grid.

In the left most column, I listed all the Acrobat engineers. Across the top, I listed all the stress-related maladies that we’d talked about at lunch, plus a few more. While I was doing all this, Alan was busy working on some bug or another, usually one that involved a test file of his that has a scanned image of a can of Coca-Cola and some crudely polygonized numbers. Then I filled in all the boxes for me using the Consumer Reports style rating symbols above. As an afterthought, I made a key. Then we marched the engineers into Alan’s office and had them fill out the chart.

It looked like a massacre when they were done. The board was covered in red dry-erase marker. Some engineers felt the need to extend the symbology to include a fully red square and had updated the key for the meaning to be “rampant”.

It wasn’t pretty. It should have be actionable, but I don’t anything every was done. At least not by management.

In the trade rag Mac Week (a.k.a. Mac Leak), there was a column one week that was poking fun at people/products in the industry. The Acrobat marketing team was giving an unofficial “feet of clay award”  for taking forever to pick a product name (in pre-release it was called Carousel), and then picking a name with no meaning.

In talking to Alan, we decided that we should make this official. Alan and I had been fooling around with a material for doing life-casting called moulage. It’s melts at about 120F and when cooled a bit, has the consistency of pudding. You can goop it on body parts and it takes very accurate impressions. When cast with plaster, you can see pores and tiny skin lines. Alan made a moulage mold of one of his feet and then cast plaster colored with gray RIT dye into it, making a clay foot.

After Acrobat 1.0 was announced as a product, we had a meeting in the cafeteria for all of Adobe software teams (there were usually separate meetings for printer engineering and product engineering). I requested time to give out Acrobat Achievement awards. To prep, I had made a bunch of overhead transparencies. My introduction slide was a 3D rendering of the brand new Adobe logo floating over desert sands. When I put it up, I introduced the awards and then said, “by the way, does anyone know if this is correct use of the Adobe logo?”

A few days earlier, the marcom department had sent out an impenetrable memo of correct and incorrect usage of the new logo. There was a smattering of laughter, plus angry looks from a group of irritated marcom people.

We gave out the feet of clay to Rob Babcock. Then we gave out the Mary Kay Cosmetics award, a pink plastic model of a Cadillac, to Dina Sakahara for reporting the most “C” (or cosmetic) bugs. And finally, I put up a copy of the malady chart, lovingly recreated in Adobe Illustrator. Nabeel Al Shama was the winner. Although ‘winner’ is an odd word to use for his suffering. Still, I gave him a Timex watch because their slogan at the time was “takes a licking and keeps on ticking.”

As a more experienced engineer now, I think I would have tried to fix this situation by trying to work on it from the top down. People burned out really badly by the end of 1.0 and I feel like we lost a lot of engineers as a result.

If nothing else, I think that what more people needed to take to heart was a couple things with regards to vacations:

  1. Nobody is so important that they can’t go away for a couple weeks
  2. If there is no good time to take a vacation, then therefore any time is good enough
  3. You’re not given vacation. You take it. Take it jealously.

I’m Old, Part XXVI: Don’t Feed Them After Midnight

At one point, I had the opportunity to work at a very small start up called Newfire. Actually, it was initially called Axial, but was renamed later in the cycle. It was an astounding team of engineers, albeit more male that I really liked, but such was life at start ups in the 1990’s. It was an astounding team. Any one person on the team could have been the anchor of any top-notch software engineering team and yet there were very few ego battles. It was probably because most of the core product engineers had their own particular specialties and did those jobs to the best of their abilities while simultaneously being grateful about not having to do someone else’s job.

67763

We were working on a high performance VRML (Virtual Reality Markup Language) engine with the goal of making Quake performance for games in your web browser.

Today’s story has to do with a point when we were far along in the development process. We were working on a demo for upcoming trade shows and were running short on time. This meant long hours most every day.

What often happened was that we ordered dinner and working late into the evening to get things into line. At the time, I was responsible for a Java scripting engine. It was a nice design that made no distinction between single player and multiplayer network games. They all felt the same to the game engine. The engine had a component model for a lot of its features including a physics engine to handle how objects moved and collided. I didn’t have time for the details of that, so it was handed to Dave Springer, aka Cowboy Dave, who was working on an implementation of a Runge-Kutta integration engine to do the physics calculations.

For the demo game, we had interns from Cogswell College, Alan Blaine, Jeremy Krinit, Greg Lima, and James Frankle. They made a mock-up of a military vehicle for planetary assault which could drive around and fire rockets at targets and get shot at. There were problems in the physics engine/the use of the physics engine which caused the vehicle to suddenly launch itself up into the sky at such a height that the vague cloud background was enormous white squares on a blue background.

On this particular evening, our CTO, Marty Hess, couldn’t stay. He had a prior engagement. So he left Mike Pell in charge for the evening with the unfortunate instructions, “do whatever is necessary to keep the engineers happy” or something close to that.

Since this code involved the physics engine, which involved the scripting engine, we had all the interns, Cowboy Dave, Mike, and me. Mike got us dinner and after dinner, Mike asked if we needed anything else. Cowboy Dave piped up, “I could use some Knob Creek.” Mike ran out and got a bottle that came packaged with a couple glasses.

Cowboy Dave was an astounding coder. Great thinker, good problem solver. But easily distracted, and that only got worse when bourbon is in play and I think the interns had their share too. I didn’t. Whiskey/bourbon are on my “had that one bad experience; still can’t smell it” list.

Marty stopped by around 10:00 to find Cowboy Dave three sheets to the wind and in the middle of a bull session conversation. Marty was apoplectic. I don’t think he imagined that his off-the-cuff instructions would have turned into this. Oh well. We still eventually ended up with a solid enough demo, but not from that night’s code.

Coding and bourbon don’t mix.

I’m Old, Part XXV: Itty Bitty Buttons

One of the magic words that people look for in resumés is “detail-oriented.” At the time that Acrobat was being written, Apple published extensive user-interface guidelines along with piles of tech notes to help you achieve these results. It seemed that for the people who worked directly on the Mac product, me, Mike Pell, and Alan Wootton, each had their own approach to doing things. Sometimes, things got dumped on us and we did a half-assed implementation to just get it done. Other times, we might pick that particular feature as our hill to die on to get it implemented the right way.

lisamw2

For example, sticky note annotations were an endless source of bugs and the initial cut looked, well, like crap. Part of the problem was that when sticky notes required scrollbars, they were teeny tiny so as not to waste too much screen real estate. Macintosh scroll bars were designed to be 16 pixels wide for vertical bars, or 16 pixels high for horizontal ones. In sticky notes, we were making them half that size. Even though the built-in scroll bars scaled to any size, they looked especially crappy when shrunk to half their expected size.

I picked this hill to die on. In the Macintosh UI you could make custom control plug-ins called CDEFs. I did the work to figure out how to make these and put together custom scroll bars that looked good at that size, and matched the Macintosh UI. Mike Pell pointed out that Apple had just release new guidelines for how to reflect color “tinge” settings that the user might make, so I updated the code to reflect that. But it got trickier, because the user could select a variety of color settings or they might be on a Mac Plus (the oldest Mac we supported in 1.0), which was 1 bit, black and white only. So I made sure I had support for that too. Then, in talking with Joe Holt, I discovered the joys of trying to make sure that your display supported multiple monitors with multiple bit depths. He gifted me a slick, small chunk of code called “ForEachIntersectingMonitor”, which made that task much easier.

I did all of this to make sure that the user experience for itty-bitty scroll bars on sticky notes was ideal. And in Acrobat 2.0, that all got torn out and thrown away. More than once, I reported bugs on the scroll bars in sticky notes in the 2.0 product out of sheer spite.

I’m Old, Part XXIV: Flights of Fancy

When I was in college, I had a number of little social circles. One of them included some friends who decided that we should all be Super Criminals since we all had our particular talents that might apply. One of them, David Brandon, had and (still has) a tremendous imagination. He had come up with a theory about how people get bored.

mackenny04a

In his theory, there is a sub-atomic particle that he dubbed a ‘boredon’ which induces boredom. He postulated that certain people emit streams of boredons and that they are so linear that if you aren’t looking straight at them, your head naturally turns one way or the other. He also theorized that they have a spin which acts as a torquing force that, for example, in a lecture hall makes you nod off.

Fascinating.

When I was at Adobe a few years later, I was thinking about this and wondered if there weren’t a bigger picture at play. For example, I observed that when there was a group of people who were heading out to lunch, the group couldn’t make a decision where to go. Similarly, once they got there, they lost the ability to order.

I figured that what was going on here was a particle which I named a dorkon which is a particle of stupidity, but there is a nearly identical particle I called a savvyon which is identical but has a different spin. Moreover, one person’s dorkon is another person’s savvyon and vice versa. So what happens is that when you have a group of normally smart people all in a group, there is such a mix of dorkons and savvyons that everybody gets dumb and can’t make decisions.

Further, I observed that whenever one of my cats sat on my lap, I would get sleepy. There is another particle at play – a dormon – the particle of sleep. As you go through the day, you collect dormons, making you more and more tired. When you fall asleep, you shed dormons so that when you wake up, you feel rested. If you get woken up in the middle of the night, you haven’t shed enough dormons to really by awake, but you don’t have enough to fall back asleep.

So what do cats have to do with this? Well, they have a high coefficient of dormon flux. They readily accept dormons (which is why they sleep so much) and also readily shed them. When they sit on your lap, they’re shedding dormons and you get tired.

Then it occurred to me. A bordon is a complex particle made up from a dormon and a dorkon. It makes so much sense! Someone who is boring thinks they’re interesting because to them, the dorkon that is orbiting the dormon is a savvyon. To everyone else, it’s a dorkon and a dormon. Get hit by enough of these and you want to fall asleep.

This is just a small part of my Grand Unified Sleep Theory.

I’m Old, Part XXIII: Hardware is Software

When I was in high school, I played a ton of arcade video games. There was a sub shop on the way home from school that had a set of games that changed now and again. In the time that I played there, I spent a lot of my money on Asteroids, Rip Off, Moon Cresta, and Defender. There were other games, but nothing I recall so well.

galaxian

When I was in college, I was home on break and went to a local arcade that was clearly in not good shape and offered them cash for an Asteroids machine they had in the basement. It was unplugged and apparently not working. I loaded it into the back of my folk’s Dodge Omni and drove home with it half in, half out. The car could clearly barely handle the extra load as it felt like I was driving up hill. It took a little work, but it was easy to fix up.

The machine followed me to California via my dad and I eventually moved it into my cube at Adobe on a weekend. People came by now and again to play. I also started to collect old video game circuit boards and got them running into a Radio Shack CoCo monitor I picked up at a tent sale for $10.

At the time, I was working on the Macintosh full-text search engine for Acrobat. We licensed and engine from a company that was a few blocks away. They clearly put a lot of effort into their Windows and Unix products, but not so much in their Mac products. Their Mac releases were frequently behind, buggy, and slow. I would run their software, pass in the license key and watch it crash. It was like they didn’t actually test their Init function on the Mac. Many times, in frustration, I would start disassembling their code to find out what was going on.

One of the problems we had, which bit us more than once, was that their code was compiled with the Apple C compiler and we used Think C. Although they should have been compatible, they often weren’t and the code wasn’t always as portable as it should be.

The API relied on large structs passed in. Each struct had to have a field set in it that was equal to the size of the struct. If the size passed in didn’t match what they expected, they failed.

In at least one structure, they embedded an enum like this:

enum Flags {

    One = 0x1, Two = 0x2, Three = 0x4, Four = 0x8,

    Five = 0x10, Six = 0x20, Seven = 0x40, Eight = 0x80

};

Quiz question: what is sizeof(Flags)?

If you look at the C spec, the answer it that it depends. In the guidelines, they suggest that you use the smallest integral type that can represent all the values. One compiler decided that a byte was good enough because it could hold everything. Another decided that since the value of Eight hit the sign bit of a byte, that they would use a word instead.

Whenever this kind of crap happened, I’d end up tracking down the problem in the disassembly of their code and then relay it back to my contact and then wait for a fix. And wait. And wait. I mean, I couldn’t do anything if I couldn’t successfully make the init call. What a pain.

During this time, I had picked up a game board for an oddball game called “War of the Bugs”. It was a terrible Centipede clone that was running on slightly modified Galaxian hardware. I got that running in my cube and started getting curious as to how they had achieved this. For one, Galaxian had a star field that scrolled in the background that was done in hardware. I grabbed some schematics and found that the stars could get turned on and off by flipping a bit. I found the trace from the address decoding that went to the star enable/disable and found that it had been cut. I bridged it with a paper clip and now I had War of the Bugs with stars! I kept digging at the schematics and looking at how the stars worked and as I went through it, it appeared that they had implemented a maximal linear feedback shift register and were using it at intervals to make the stars. It was ingenious, really. And the more I looked at it, the more I realized that this little chunk of hardware was really software in gates. It was elegant, really. It was as if the entirety of the star field had been expressed in haiku.

Meantime, my boss, Ken Anderson, had observed that I was not making progress (how could I? their code crashed and I reported it with an easy repro case, so I was spending my time poring over schematics). He ended up calling bosses bosses at the search engine company to find out WTF was taking them so long. I felt bad when I heard him on the phone chewing out a person at the search engine company saying, “My Mac engineer is sitting here dissecting video games because your code doesn’t work.”

Many times, we took walks over to their building to pick up another turn of code and hope for the best.

Eventually, we did get code that worked well enough to ship, but it wasn’t always easy, but that’s a story for another day.

I’m Old, Part XXII: Interviewing

When I interviewed at Adobe, I had a whirlwind set of interviews in that trip. I interviewed at Adobe, Electronics for Imaging and Apple. Apple turned me down cold. EFI offered me a job, as did Adobe. Of course, I accepted the Adobe offer.

interview_panel

The interviews were very different. What I remember from EFI was that the interviewer wanted me to write code to solve Towers of Hanoi problem. This is a classic problem that is easily solved with recursion, but I had done something very similar not that long ago, so instead of going to work, I asked the question, “recursive or non-recursive?” If you understand how recursion works, writing a non-recursive equivalent is not hard. I think the interviewer was completely surprised by my question and I’m willing to bet that that stood out.

I will briefly interject that I can’t stand this kind of interview question. When you present a puzzle of this kind and all you’re looking for is a solution to the puzzle, you doing yourself and your interviewees a disservice.

Let’s break the kinds of candidates into a few categories:

  1. Maybe not that sharp, but studied puzzles and their solutions
  2. Very sharp, didn’t study these puzzles
  3. Very sharp, gets stage fright
  4. Not very sharp

Who would you want to hire? Probably 2 or 3, but 1 may very well be indistinguishable from 2 and 3 may very well be indistinguishable from 4. Clearly, this kind of interview question can’t be the only deciding factor. You need more data. I’m sure at some point later I will go into more detail on how to interview for a programming position, but not today.

At Apple, I did not have a great day and I had some very brutal interviewers.

At Adobe, I have several stark memories. I interviewed with, I think, 6 people in that day: 5 tech and 1 human resources. Of the tech, I interviewed with, here’s what I remember:

  1. Bill McCoy – Bill is a very sharp man and a strong architect. I remember he asked me why I wanted this job, and I answered in the most honest way. I said, “I really like programming.” It was true then. It’s still true today, 36 years later.
  2. Matt Foley – I don’t remember the questions Matt asked me, but I do remember asking him if he was half of Foley and Van Dam. No. Awkward.
  3. John Gaffney – John ended up being my boss. John gave me what I later learned was “the Gaffney test”. He would give the candidate a quick intro to PostScript and the imaging model. Then he would give you the PostScript:
    20 setlinewidth
    100 100 moveto 100 200 lineto 200 200 lineto 200 100 lineto 100 100 lineto stroke showpage

    And he asked you to draw what it would print on the whiteboard. If you were paying attention, you would draw a rectangle with a 20×20 square missing from the lower left hand corner, which has to do with how PostScript defines a path. I got that right.

  4. Norin Saxe – Norin also went to Oberlin, but before me. We both knew the same CS professors, but not each other. He had the lunchtime slot so we went to the cafeteria and I got a free lunch while we talked about things. Norin plays viola and was interested in my first listed job as a violin maker at Bell Labs. I have a party trick, which is I’m pretty good at knocking out a hidden line removed isometric projection of an object I’m familiar with. So in this case, I drew out one of Max Mathew’s violins and explained how I made the parts and assembled them.
  5. Dyanne Compton – Dyanne for the most part explained the benefits that came with a job at Adobe. At that point, I think I was just a deer in the headlights and at that point in my life, just out of college, a solid paying job with any kind of health insurance was way better than a poke in the eye with a sharp stick. Although I didn’t really run into Dyanne all that much in my day to day job, she always remembered me and most of the employees she helped hire. To me, that is a much better party trick.

I can’t say how many people I’ve interviewed across my career. I can say that the first several times, I was terrible. I had no idea what I was doing. I got better, but I also got brutal in a lot of interviews. I asked hard or open-ended questions that would allow candidates to just as easily shine or hang themselves. I can also say that of the people that I green-lit that we ended up hiring, I don’t regret a single one. Each brought a mix of strengths and of the weaknesses that mattered, I knew also that we could educate and the investment would be totally worth it.

At times, I felt bad. For example, when I interviewed Rick Minerich, who looked really good on paper, I dug very deeply into his knowledge and didn’t let him half-ass any answer. Rick was clearly nervous and I remember that his hands were shaking so much that he was holding onto his phone to try and hide the tremors. He ended up dropping it a couple times. I don’t take off points for being nervous, and Rick was one of the best hires we’ve made. Rick also gave me a nice atta-boy here.

Ultimately, I feel like interviewing should always be a two way street. When you’re looking for a job, you need to try to figure out, “do I really want to work here?” And as an interviewer, you need to put up a good front when you’re interviewing because you’re not just hiring, you’re selling. If you’re going to be horrible when you’re interviewing, you shouldn’t expect great hires. In the words of Wil Wheaton, “Don’t be a dick.”

I’m Old, Part XXI: I Love Deadlines. I Love The Whooshing Sound They Make As They Fly By

Software estimation is hard.

No, that’s not quite right. Software estimation is tremendously easy. Accurate software estimation is hard. When I started on the Acrobat team, the first thing that I was assigned was the task of making the “Find Text” feature work. Many of the features in Acrobat were initially implemented by one hurried engineer and were half-assed. If you asked the engineer who did it why it didn’t do some particular thing, the inevitable response was something like, “Because I wrote it in an hour/day. Go ahead and make it better.”

0NMMZ0b

Find Text, as it was, could find one and only one word and even then, it ran like molasses in January and often missed words. It also stopped after finding the word with no context for continuing. My job was to make it run faster and to operate more like a word processor. So it had to find phrases, it had to handle find/find next/find prev, and it had to have I/O cover-up when needed.

I/O cover-up is something that you put into an app to show the user that something is going on. It might be a spinning beach ball, a progress bar, or some other visual cue. For spinning beach ball cursors, you had to have a chunk of code that was hit my your working loop that updated the cursor – but not too quickly and not too slowly. The Mac didn’t have real multiprocessing at that point, so you either did your own time slicing or you did what PhotoShop did (I think it was PhotoShop – it might have been Illustrator), which was to install an interrupt level task attached to the vertical blanking interrupt to spin the beach ball. The was great in that the cursor spun very smoothly. It was lousy because if the app hung, you still had a beautiful spinning cursor and no indication that something horrible has happened.

I was asked how long the tasks would take and I pulled a number right out of my ass: 3 weeks. I was off by a factor of 5 because I just didn’t know. The reason why the original code was so bad was because of PDF. PDF is not a text file. Any given page is a program that gets executed to get rendered. The underlying code had a setup where you could render a page and get a call-back called whenever text was placed on the page. The problem was that the callbacks happened whenever any arbitrary string was placed and that was at the whim of the program that placed the text. So, for example, the string “hi there” might have been put on the page in one shot or in as many as eight separate blocks. The original code couldn’t deal with this.

So one of the first things I did was to write code that received text and when it made sense, broke up words by separating at spaces or put disparate word blocks back together again if the code thought it was a word. This code was tricky because text might be rendered in different fonts with slightly different sizes and on top of that the text might not be on the page in a way that was rectilinear. It took me more than a week to tune the heuristics, then I had to tame the memory and performance issues that resulted because through all this I needed the metrics of the fonts being used and who knows how many fonts were on the page. There was a lot of “just slog through this” code. Not elegant, but it worked.

Then there was troff. God damn troff. When it generated PostScript (then converted into PDF), it placed all the “plain” text first, then the bold text, then the italic text. So in addition to putting words back together, words had to be sorted on the page into some semblance of reading order. Oh wait. What’s reading order? Left-to-right, top-to-bottom? Sure, if you use a typical European based language. What about Arabic? Japanese? Sumerian? OK, I never saw that one, but still. I seem to recall punting on reading order.

PDF has many aspects that make it non-trivial for someone to just write their own. Every engineer on the team contributed at least one. Mine was that when you found text, it should be displayed on the page with an accurate highlight. The previous code just used an axis-aligned rectangle which was set to the min and max of the word. This was accurate as long as the word was axis aligned, but we had documents (maps, label designs, etc.) that were not so friendly for that kind of highlighting. What I did instead was make that bounding area be a set of quadrilaterals that bounded the word fragments. I sweated the details a lot to make the highlights look good, including correctly joining up contiguous quadrilaterals. So when you look in the spec at the highlight annotation, the reason why it’s all quadrilateral based is because search worked that way. Search worked that way because I insisted that it should work that way.

Then there were ligatures. In typography, there are letter sets that are replaced with a single character that represents those letters all tied together. For example fi and fl are two of the most common ligatures because in many fonts when you draw an f next to an i or an f next to an l, it looks crappy because of the overlap. I put in support for ligatures, so if you had the word ‘waffle’ on the page rendered as waffle, with the ffl ligature, Acrobat would find it.

I had no idea how much I was going to learn about typography in this process. When I was done, I was very pleased with the result.

But yeah, that took a lot of time. Oops. Little did I know that most of that work would end up on the trash heap after Acrobat 1.0 shipped.

Still, I had a good lesson in software estimation: don’t ever commit to a number until you understand the problem space. It was a good lesson to learn at age 27.

I’m Old, Part XX: I ♥ 6502

When I started out writing programs of any size, I started with BASIC on an Apple II. There were programs supplied in a bunch of the original manuals that you could type in and try. This was how I learned. My dad picked up another book called More Basic Computer Games by David H. Ahl, published by Creative Computing. Me and my brother Pat typed in the games from the book and learned a lot of practical lessons about portability, especially since BASIC had no particular standard at the time and every micro that had BASIC had its own particular variation. We also learned debugging since we typed in all kinds of interesting bugs.

a2refman

I had seen games for the Apple that had a teeny BASIC prologue with an inevitable CALL into machine language. All the arcade style games were like that. My brother Pat was picking up the Apple’s 6502 machine language and I watched over his shoulder. Our machine had the “Integer BASIC firmware card”, which included a small monitor with a mini assembler, so if the switch on the card was in the right position, hitting reset (or typing CALL -151) put you into the monitor and F666G started up the mini assembler.

One day I was home from school, ostensibly sick and I got tired of hanging around in bed, so I sat down at the computer and opened the Apple II Reference Manual and started looking over the routines available in the reference. They were cookbook. The 6502 had 3 easily accessible registers (in reality, it has 6, but the stack pointer, program counter, and status register aren’t really directly accessible). A reference entry would tell you the address, the name of the routine, what it did, what registers had to be what in order to do it, and what registers were ruined/preserved upon return. To call a routine, you used the JSR instruction, which was a mnemonic for “Jump to SubRoutine”.

I started with:

$FDED – COUT Output a character

COUT is the standard character output subroutine. The character to be output should be in the accumulator. COUT calls the current character output subroutine whose address is stored in CSW (locations $36 and $37), usually COUT1 (see below).

So I fired up the mini assembler and wrote the following program:

0300: A9 C8    LDA #$C8
0302: 20 ED FD JSR $FDED
0305: A9 C9    LDA #$C9
0307: 20 ED FD JSR $FDED
030A: 4C 00 03 JMP $0300

Which was equivalent to the BASIC:

10 PRINT "HI"; : GOTO 10

When I ran it, I was aghast. The screen filled up with “HIHIHIHIHIHIHI….” before I could react. It was astounding.

With all of this in hand, I started writing 6502 machine language programs. The manual had routines for text I/O, low-res graphics, beeping the speaker, etc. I was off and running.

Writing code in in 6502 was tedious, but it was a great first exercise in practical Turing Completeness. I could write anything in 6502 that could be written in BASIC, and it ran a hell of a lot faster, but I had to write a lot more code to do it.

If I recall correctly, the 6502 had 56 instructions, many just register juggling, and in typical coding, you usually just use a handful.

I started disassembling other people’s games and learned to find the main loop that ran the game itself. These were easy to spot as they were usually just a long string of JSR instructions with a JMP up to the top. I used other tricks – like finding the code the read the keyboard and looked for code that compared the values read to the key values that started or controlled the game. Finding what called that routine usually lead back to the main loop. I would change one of the JSR’s to a BRK instruction to drop into the monitor. Then I could manually call one of those routines and see what happened on screen. When I found something interesting, I disassembled that routine and took it apart.

By looking at code written by Bob Bishop, Gary Shannon, Bill Budge, Nasir Gebelli and more, I started to learn how write code that did high-res graphics. This code taught me how to do a table look up. The standard routine to calculate the address of a scanline in 6502 was…interesting. And slow. A table lookup was quicker.

After 8th grade, my dad and my brother Pat and I went on a trip across country by car. We went to the Grand Canyon and hiking down a bit on the Bright Angel trail. It’s a lot easier to hike down than up, and on the way back, my brother Pat, who had the water took off on his own. On the way back up, in the heat, I imagined little happy creatures throwing themselves off the top of the canyon and plummeting to their deaths. When we got home, I started turning the concept into a game called, “Suicide!”. From a grown-up stand point, it’s not all that playable, but it’s fairly astounding that it was produced by a 14 year old, especially because it was all in 6502. Historically, it was the first published game with punctuation in its title. It’s a little thing, I know, but I enjoy that.

When I was in high school, I drew all kinds of doodles in my notes in class, but I also wrote hundreds of lines of 6502 to solve problems that had stumped me. Through the local computer shop, Stonehenge Computing, I found a small, thriving community of young men like me who were hacking on the Apple II, all suffering through the foibles of the 6502 and Woz’s spartan hardware design.

In college, I still hacked 6502. One of the last things I wrote for the Apple II was an implementation of Conway’s Rules of Life. A few years ago, I had a small stack of my remaining 5.25″ floppies read out and I found the source, which I put up on codeplex. To my surprise, I had commented the living hell out of it. Good for me.

1000 * THE FOLLOWING IS AN ADAPTATION
1010 * OF JOHN CONWAYS GAME OF 'LIFE'
1020 * THE RULES ARE SIMPLE:
1030 * THE SCREEN IS DIVIDED IN CELLS.
1040 * A CELL IS EITHER DEAD OR ALIVE.
1050 * IF A CELL HAS EITHER 2 OR 3
1060 * NEIGHBORS, IT WILL LIVE IN THE
1070 * NEXT GENERATION.
1080 * A DEAD CELL WILL REINCARNATE IF
1090 * IT HAS EXACTLY 3 NEIGHBORS.
1100 * ANY OTHER NUMBER OF NEIGHBORS
1110 * WILL MEAN THAT CELL IS DEAD IN
1120 * THE NEXT GENERATION.
1130 *---------------------------------
1140 * MOST IMPLEMENTATIONS USE A TWO
1150 * BUFFER SYSTEM: ONE FOR THE
1160 * CURRENT GENERATION, AND ONE FOR
1170 * THE NEXT GENERATION. THIS
1180 * VERSION USES ONE BUFFER FOR THE
1190 * CURRENT GENERATION AND KEEPS
1200 * A LIST OF CELLS THAT HAVE
1210 * CHANGED. THIS SAVES TIME AND
1220 * MEMORY. SINCE DEATH AND BIRTH
1230 * ARE A BINARY PROCESS THE CELL
1240 * DRAWING/ERASE ROUTINES ARE ONE
1250 * AND THE SAME. THIS IS DONE VIA
1260 * EXCLUSIVE-OR DRAWING.
1270 *---------------------------------
1280 * IMPLEMENTAION FOR APPLE II
1290 * SERIES COMPUTERS BY
1300 * STEPHEN HAWLEY   9/85

I put this comment on here because this was the point when I independently invented the display list. This was a repeating theme in college. I invented a ton of things I found out later were CS staples, and all of them in 6502.

I’m Old, Part XIX: A Brief Unix Room Memory

When I was at Bell Labs, I was a violin maker working for Max Mathews. Seriously. It was a great job. I worked for Max Mathews and in addition to hacking up pieces of aluminum and wood to put together violins, I routinely came in during the evenings and weekends to get access to the Unix systems to play.

att_logo_change

The Unix room was at the far end of the hall from room 2D-562, which was Max’s lab. It was also almost directly below the illegal machine shop on the 6th floor where I did most of my work for Max.

The room had glass doors in with a slope up to the raised panel floor. I was in there one Saturday at a point where Andrew Hume lost it over something. I never knew what. Andrew was an imposing Australian who frequently wore tan ragg wool sweaters and shorts, even in the winter.

Andrew let out a roar and grabbed Rob Pike’s chair, complete with Rob, and shoved him down the ramp and down the long hallway. Rob coasted some distance and slowed to a stop where he was met by a security guard, who he asked for a push. Hume went after him and shortly thereafter, Pike went flying up the ramp back into the room.

I loved this very human aspect to work that is often seen as inhuman or dispassionate.

As a side note, I will mention that for many of these vignettes, I rely on my memory which is primary visual and auditory. What stinks for me is name recall. For the past decade, it’s gotten progressively worse. In conversation with other tech people, I put the blame of my kids for destroying my name cache. Sometimes it takes me hours to dredge names up from long-term storage. I can see Andrew Hume in 1983, just as clearly as I can see the breakfast I ate today. It took me close to an hour to pull his name up through endless circles of incorrect name fragments. This is not a denigration of Andrew, but of my poor brain.

I’m Old, Part XVIII: Acrobat Lacks Absorbency

The Acrobat team was built with a decent amount of infrastructure by 1990’s standards. One problem that faced the team was that the product was designed out of the gate to run on multiple target platforms. Initially, it was MacOS (I think they were on System 7 at that point), Windows 3.1 or 3.5, and DOS. There was a UNIX version, but the demand was low so it trailed far behind.

Outside of writing code, this presents a problem because it means that the infrastructure tools needed to run cross platform as well. That meant that the bug database had to have Mac and Windows clients as well as the source control system.

H96566k

For the bug database, the group used a 4D database as there were clients available for everyone. It ran like a dog, but it did the job.

When QA reported a bug, it was classified as A, B, or C. An ‘A’ bug was a crasher or significantly incorrect output. If you followed the steps, you would cause Acrobat to crash or in far too many cases, to take the entire machine down. B bugs were incorrect functionality or unexpected behavior. C bugs were cosmetic bugs, e.g. missing the ring around the OK button of a Macintosh dialog box or poor wording or a message that ran off the edge of a window.

One of the dreaded problems in software development is recidivation. You fix a bug and then at some point later in development it returns in a different form. For the most part recidivation happens when an engineer hasn’t fully understood the cause of a problem and has instead fixed an apparent cause or when the fix was in the right place but was fragile or a hack.

At one point, there was a piece of paper put up on a bulletin board outside of Bob Wulff’s office that read, “2-4-6-8, we will not recidivate!” I think it was put up by Jonathan Sjordal. It was a nice sentiment, but I don’t think it helped in any practical sense.

When working on a long-term project such as Acrobat, the development and QA engineers all tended to go non-linear. It’s not surprising. Acrobat was a project that falls into the category of “land-grab”. It was a product that had a clear spot in the market and we were trying to fill it first and before anyone else. Under those conditions, it’s not surprising that people go funny.

There was a bug report opened with the title “Acrobat Lacks Absorbency”. In the description, there was a test file that you were directed to print out, then you were supposed to take a dump and wipe with the print out and observe the results.

This bug was opened and closed and reopened and assigned and reassigned and commented and moved to QA and bounced to an engineer and then bounced back and closed and reopened and deferred and undeferred and closed and reopened.

If someone reopened or reassigned it, it was a clear sign that engineers were either tired or punchy or both. I think that particular bug set a record for the most number of comments on a bug. It showed up in morning meetings and I’m pretty sure that Bob face-palmed more than once when it showed up. He just couldn’t kill it.

Personally, I think it was a good sign when it showed up. It was an indication that people still had a sense of humor, no matter how crude.