madhadron

Context

This is the part of the series on getting better as a programmer. The articles are:

When we speak of specializations among programmers, we categorize them in two very different ways. On one side we have terms like frontend programmer, mainframe programmer, embedded programmer, or even Java programmer. On the other we have terms like programmer in finance, in advertising, in automobiles, in video games, or in desktop publishing. The former categorizes programmers based on the tools they use, which we’ll talk about in the next essay. The latter categorizes them based on the settings they produce programs for.

So what makes programming in these settings different? Consider what it means to produce software. When we deploy a computer running a piece of software, we are injecting a symbolic automaton into our world. That automaton has some means of receiving signals from the world which are interpreted as symbols, it manipulates those symbols, and produces symbols which trigger signals back out into the world. If we’re going to produce software that does this well, we need to know what signals we should receive, how to interpret them as symbols, and what emitting given signals in response is likely to do in the world.

On a very concrete level, we can imagine a robot in a factory. It has sensors that detect the presence and location of particular, physical things. It has a means of converting those sensor readings into symbols that can be computed upon. And it computes and emits commands to its actuators to move the robot to hopefully do the job that it is designed to do. The programmer needs enough knowledge of physics to extract that part which the robot will deal with and express that subset symbolically. They also need enough knowledge of assembly lines and industrial engineering to formulate a symbolic system to express policy—what happens in case of problems, for example?—and work with experts in the field to decide on and encode the policy needed.

On the other hand, someone working on electronic medical records needs enough knowledge of the practice of medicine to produce a symbolic system in which medical practitioners can express what they know about a patient and share it. Like the robotics programmer, they need to produce a symbolic framework for policy, but this time it’s policy based on the legal and professional structure around medicine, so they need a certain amount of legal and medical knowledge. And they need to know enough about how medical information is abused to design their symbolic systems in a way to prevent it.

Both these examples involved the knowledge that the programmer needs to be able to encode in their programs. There is another kind of knowledge that is not generally encoded symbolically, but is equally vital: the contextual knowledge to understand how a program fits into the organization that is producing it, the organizations paying for it, and the organizations using it. Are there enough people able and willing to pay for a particular program to pay for its creation? If so, can it support the ongoing cost of maintaining a software team? What are the incentives and constraints on the people in those organizations? A programmer in academia needs to know about grants, citations, how research groups are structured, and how academics adopt tools. A programmer in a commercial company needs to know about cashflow, profit and loss, assets and liabilities, and corporate power structures.

Knowledge for formal expression

When a programmer produces a program that takes some information about the world and produces an output, they are creating a theory in the same way that a scientist does. I’ve written a book on what happens when a scientist does this, but for programmers we can get by with a much shorter account.

When a programmer builds a theory, they

  1. select the observations and actions that the theory will concern itself with,
  2. construct a symbolic representation of the results of those observations and the available actions, and
  3. produce a program that generates symbolic representations of the actions from symbolic representations of the observations.

One of the common errors in our field is thinking that you know what the right observations and actions your theory should concern itself with are without deep knowledge of the day to day work and worldview of the people who will interact with your program. In Domain Driven Design, Eric Evans recounts working on software for bills of lading for shipping. They team began with what they thought were the obvious pieces, but people actually involved in shipping reacted with confusion because the pieces that seemed obvious to the programmer were often secondary or incidental to the people actually doing the work.

The people that will use your software already have their own slate of observations and actions that they think in terms of. They typically won’t interact with them in terms of formal symbolic models, but they have internalized them in something like Wittgenstein’s language games. If you haven’t run across the concept of language games before, I urge you to read the first 30 or so aphorisms in Wittgenstein’s Philosophical Investigations, which is one of the best books on software design ever written. Failing that, consider that when people interact to do something they usually work out a limited language that they use. Sports coaches calling plays to their teams, musicians signalling each other while playing chamber music, or a surgeon calling for instruments and specific actions to an assisting nurse are all using a very restricted means of expression. They don’t think of it in formal, symbolic terms, and it usually emerges from a deep worldview and body of knowledge that is required for it to make sense. A programmer must tackle all of this. Otherwise you might end up with a program meant for use in surgery where nurses record a queue of commands for the surgeon, the language game only consists of the surgeon saying the name of an instrument and it being handed to them, and measures of whether the patient is doing okay are taken from vital signs during day to day living, not what happens under anesthesia.

There are few areas of knowledge that seem to be ubiquitously important for programming. The biggest one that seems to be crime and abuse. Any piece of software that ends up used by multiple people in lots of ways. If those uses are not legitimate, we think of them as abuse.

Legitimacy is a difficult question. In US hospitals, patient records are supposed to be protected from anyone viewing them but the health care workers treating the patient. IT departments often implemented this by requiring doctors and nurses to type in a username and password to log into a terminal each time they wanted to access medical records, and to log out when they left the terminal.

What actually happened is that a doctor or nurse would log into the terminal and leave it logged in because requiring each nurse to free up both hands, type a long password and wait for a logon process to complete was unacceptable. This was described in a paper with the catchy title, “You want my password or a dead patient?” The solution in practice is to stop using usernames and passwords and have medical personnel carry around a smart card attached to their coats—hospital personnel already have to carry around such cards for door access—that they tap on a reader to log in. Make sure logging in takes a fraction of a second, and, for extra points, make the card lightly magnetic so it sticks to the reader and pulls off when the user walks away.

From the point of view of the IT department, this was illegitimate access and abusive. From the point of view of the medical staff their use was legitimate and working around the illegitimate demands IT was placing on them. A patient putting on a lab coat and sitting down at a logged in terminal to browse everyone’s medical records would be clearly illegitimate to both groups. Legitimacy is based on the social structure that using software takes place in. The software itself gets no say. Indeed, the intentions of the software’s creators may regarded as abusive and illegitimate, such as Microsoft’s placement of advertisements in recent consumer versions of Windows.

Then there is what to do about abuse. The most important thing to know to stop a pattern of abuse is to know why it is happening. Who benefits, and how? If you remove the benefit, that pattern will adapt, either by finding a different way to abuse your system or by going and abusing someone else’s system. Preventing abuse in software systems begins with economics and psychology, not code.

Abusers generally want some combination of something you have, something you are, and something you know (because something you have, something you are, and something you know are the three ways of authenticating someone and doing something in their name). Something you have is typically access such as a way to run software on a user’s machine or authentication to another system the abuser is interested in. Something you are comes back to legitimacy. Your system may be the system of record for some purpose, or in some other way blessed with legitimacy, such as being able to publish documents on whitehouse.gov. A recent wrinkle is poorly controlled software like chatbots being given legitimacy as support agents, as in the 2024 care where Air Canada was required to honor a made up policy told to a customer by its chatbot. Something you know may be in depth like medical or financial records, or as simple as emails and passwords of your users. Many people reuse passwords accross sites so lax password security on your part can lead to a breach elsewhere. If you do nothing else, running through this list and trying to dream up nefarious ways that what you have, what you are, and what you know can be used abusively is a good start.

The second area of knowledge that seems to be everywhere is accounting, both of money and of shared resources. Most people think they understand accounting of money, but how many times have you seen tutorials on transactions in databases use debiting money from one account and crediting it to another as an example of something you need transactions for? This would require every bank in the world to be in the same transactional domain, and would make writing checks impossible. Those some examples often fail if the debit would take the account balance below zero. In the real world account balances can be any number the bank is willing or is forced to allow. Nor does money flow like a conserved fluid from account to account. It is regularly created and destroyed in delocalized ways. Writing a check for $10 creates $10 of money in your hand. When the check gets used for payment, gets cashed, and the bank it is cashed at reconciles transactions with the bank the check is written on, the $10 is destroyed. The $10 need not be destroyed, depending on who issues it, though. A bank loaning $10 creates those $10. When someone pays $1.10 on the loan, $1 of principal and $0.10 of interest, $1 of the $10 is destroyed again. But what happens if the recipient of the loan defaults? And what happens if a government that issues currency writes a check in that currency? How does a federal reserve system alter that? In the other direction, if an organization sets an internal budget, what happens if it goes over budget?

That’s for money. What about other resources like the time of people involved or availability of equipment? Many organizations that switched to electronic scheduling of shifts have gone from lots of flexibility so long as people make sure their shifts are covered to regimentation because the software wasn’t written to handle flexibility. Or in manufacturing, what is the right measure of utilization of machinery and how do you decide what gets priority? For assembly lines, it turns out that maximizing the fraction of time an expensive machine is in use is disastrous for production. What about freewheeling settings like a research laboratory? Sarah needs the ultracentrifuge tomorrow at 2AM or she loses 3 days of work. How do you handle that?

Knowledge for organizational setting

I treat the areas of knowledge that you need to operate in an organization are different because you don’t need to abstract them into symbols.

The key to understanding an organization is that, despite what its organizational chart might show, it isn’t a single governing structure. It never is. Humans govern themselves in a polycentric fashion, that is, with multiple, interlocking systems of governance. Consider this very partial list of the systems of governance that a programmer in a large corporation may find themselves part of:

The tree-like hierarchy of managers adopted from the US military when 19th century railroads started to get large is the most visible system of governance, but it often isn’t the most binding one. I used to go hiking with the VP of R&D at one place I worked. The manager in my group, who was later fired for abusive behavior and gross incompetence, would regularly demand that I not tell him certain things or suggest I shouldn’t have this hiking buddy for the exact reason that this was a system of governance that I was involved in that cut straight across his domain without him being involved. In another instance, there was a cabal of engineers from early in the product’s development who carried an enormous amount of moral authority. Management could say that we were going to do X, Y, and Z, but if they disapproved it probably wouldn’t happen. That group of engineers was later carefully split up, isolated from the product, or fired as more politically savvy upper management took over.

There may not be that hierarchical structure at all. Valve famously runs with no managers and no particular hierarchy. There are specific checks in place such as a short list of people who are allowed to sign contracts that oblige the company to pay nontrivial amounts of money. In another dimension are unionized workplaces where the labor union represents another significant set of governance structures, or worker coops, which may or may not have a hierarchical management structure, but have a democratic process which the management structure is subordinate to.1

For a middle manager in standard large corporation in the United States today, they find themselves in a set of interlocking governance structures that may be entirely divorced from both the outside pressures, goals, and health of the company as a whole and the actual production or work that the company does, with predictably pathological effects.2 Similarly, an employee working remotely may be well connected in one company, but in another they may be completely excluded from critical systems of governance that only happen in person.

Many young programmers look at all this and say, “I just want to write code. Tell me what code to write and leave me out of it.” But, as the old saying goes, you may not be interested in politics, but politics is interested in you. Even soldiers in modern militaries don’t get to merely do what they’re told, both because they are responsible for judging the legality of orders they are given, and because our societies paid a brutal cost to learn the effectiveness of independence to interpret command intent.

How to ascend

A good place to start is Peter Naur’s paper “Programming as theory building” where he discusses our relationship as programmers with the theories we construct. It provides a feel for what we are trying to build for ourselves.

Then you must learn how to distill formal versions of areas of knowledge, which you only learn by doing. That means going out and learning enough of field after field to try and distill them. But how do you go about it?

First, you need access to at least a few professionals who are willing to answer questions and explain things. If you happen to be a professional in a field besides programming you can get away with interrogating yourself the first time, but getting used to finding and asking what seem like trivial, stupid questions is an important part of the skill. Second, start learning the theoretical side as if you were going to be a professional. You don’t have to memorize all the medical knowledge that doctors have to or build the physical skills of a violinist, but you should at know enough to recognize when they are drawing them. You need to know that the theoretical descriptions most fields give of themselves are a pale shadow of what they actually think and do, but it at least gives you a skeleton to start from.

Then you begin trying to produce a formal description of some piece of the field. You can do this in whatever medium feels best to you. It could be types and functions in Haskell or OCaml, a formal specification language like Alloy, relational database schemas, or UML class diagrams. The important thing is that it be something that a computer can actually run, which is an unbelievably strong forcing function. In the preface to Structure and Interpretation of Classical Mechanics, Sussman and Wisdom write: “Classical mechanics is deceptively simple…When we started we expected that using this approach to formulate mechanics would be easy. We quickly learned that many things we thought we understood we did not in fact understand. Our requirement that our mathematical notations be explicit and precise enough that they can be interpreted automatically, as by a computer, is very effective in uncovering puns and flaws in reasoning. The resulting struggle to make the mathematics precise, yet clear and computationally effective, lasted far longer than we anticipated. We learned a great deal about both mechanics and computation by this process.”

And then you try to break it. Take your model’s assumptions and use them to derive the most blanket statements that you can, then go to your professionals have them poke holes in those statements. At first it will be simple things like saying to a doctor, “So, except for antibiotic resistant strains that arise, any antibiotic will kill any bacterium?” or asking a violinist if every C note on the instrument has a frequency that is some power of 2 times 256Hz. Both are wrong, but they’re wrong in ways that will elicit a great flow of detail. Then you must sort through that detail and figure out what parts you need to incorporate in your formal system and what parts you can ignore. You repeat this cycle of study the domain, extract a system, and test it on professionals until it converges to something.

Don’t show the professionals your formal system. Many people, especially those in fields that required them to do well in school, when faced with a formal system, revert to the mindset of solving word problems in math class and they unconsciously simplify their world to match. It is your job to take the assumptions of your system, derive the consequences, and then go talk about those consequences in their language and on their terms.

Coming to the ubiquitous domains of abuse and accounting, if you read one book about abuse and security, I suggest Shostack and Stewart’s The New School of Information Security. If you read two, add Ross Anderson’s Security Engineering.

For accounting, any general finance textbook will do for money. For resources, it’s worth reading Goldratt’s The Goal and then studying how factories use kanban (the business novel Gold Mine is a good exposition, in the same vein as The Goal). There is a great deal more to this, but that’s a starting point.

Tanya Reilly’s The Staff Engineer’s Path has a lot of good advice on mapping out the organization around you. Slywotzky’s The Art of Profitability adds the clearest description of how to model companies’ engagement in the larger economy and Deming’s Out of the Crisis teaches a huge amount about thinking properly about organizations. After that you may want to dive into the later work of Eleanor Ostrom.

And you may find that you don’t have the patience or stomach to be an adept actor in many of the governance structures that matter in some organizations, but it’s better to know that than suffer blindly.

  1. It is a very strange thing that democratic nations regard the ubiquity of autocratic regimes in companies as normal, and not a perversion of their national principles.↩︎

  2. I have heard that Jackall’s book Moral Mazes documents this quite well, but I haven’t read it.↩︎


« Home