Essential vs. Accidental Arguments in Novelty Claims for System Design

(With credit to Salzer et al.’s “End-to-End Arguments in System Design” for the title.)

Note: I originally wrote this post to coach students writing research papers. Efe Barlas pointed out that some of the thoughts also apply to system design analysis, e.g. design docs and design reviews. If you’re a practicing engineer, think about how these ideas might transfer to your design work.

In computing systems research and development, and I suppose in any discipline whose contributions consist of a new way of doing things, one must explain why the new way of doing things is a substantial innovation and improvement over the current way of doing things. One typically argues this by considering the requirements of the system, and showing that either (1) existing systems failed to satisfy known requirements; or that (2) there are new requirements for which the existing systems are unsuitable. In either event, one is faced with the following analysis task: show through reason and through measurement why the old way of doing things is inadequate.

I have had many conversations about system design, and I have read and peer-reviewed many research papers, and I have observed a common error in this analysis task. Here is the error: analysts often treat a prior work in terms of its embodiment instead of in terms of the underlying concept. When you describe the shortcomings of an embodiment there are a million things you can complain about — everything from the specific file formats that the prior work supports, to how slow it is, to anything in between. There is a lot of scope for criticism, but all such is misplaced. Except in rare cases, the only legitimate target for critique is the conceptual underpinning of the prior work.

Some philosophy

These chairs are both chairs in their essence, though they vary in the details. The table, meanwhile, is not a chair. Of course, all of these are symbols rather than the thing signified :-) Thanks Magritte.

To criticize prior works in this way, I often invite my students to consider the essential versus accidental properties of the prior work. I am not a philosopher…well, I suppose PhD stands for “Doctor of Philosophy”, so perhaps I am? At any rate, I suppose possessing a PhD at least permits me to stray a bit into philosophy. I digress.

The notions of essential and accidental properties have been discussed for millennia. You can think of them as follows: an essential property is one that any embodiment of it (instantiation, reification, implementation) must have in order to be faithful to that concept. An accidental property is one that can be possessed by a particular embodiment of a concept but which is not universally true of all embodiments.

• For an example in everyday life: A chair is something I can sit on. All chairs have at least one leg. These are essential properties of a chair. But not all chairs have four legs and not all chairs are made of wood; those are accidental properties, distinctive to specific chairs.
• For an example in computing: We talk in compilers about parsers, whose purpose it is to check whether a given input follows a given structured format. Parsers are often used as part of rendering that input into an intermediate representation for further processing. Well, all parsers take input and check whether they meet a format — that’s their essential property. But the formats vary by parser! Not all parsers work on JSON; not all parsers work on XML; not all parsers work on CSVs. Criticizing a particular parser because it doesn’t support certain file formats is critiquing an accidental property of that parser rather than an essential one. Meanwhile, criticizing a parser because its computational complexity is too large, or because it cannot parse input formats with certain properties, is attacking an essential property of a parser (or a class of parsers, anyway). To change those properties, the parsing algorithm represents a substantial change rather than a modest one. After the change, one could say it is indeed a different kind of parser now, that its essence has changed.

I am reminded of the “Ship of Theseus” debate, where one considers whether, having replaced the oars, the deck, the rudder, the masts, the rigging, and all the other parts of a ship, whether it is indeed still the original ship. If you can replace all of those components with comparable parts equivalents, perhaps making the ship a bit faster or a bit easier to steer, and yet preserving the integral properties of the ship, I would say that you’re swapping in and out different accidental properties (different implementations of the same design). But if you make a crosscutting change, something that spans the whole system, that would take weeks or months or years of work rather than hours or days of work, then I would say you’re probably looking at an essential property of the system. By changing it, you’ve realized a conceptually different system. A cargo ship that’s 10% bigger is still a cargo ship; it’s not an aircraft carrier and it’s not a speed boat.

Applying philosophy to make sound criticisms

Now, what does this have to do with criticizing prior work? Here’s the bottom line: If you are criticizing prior work for an accidental property then your criticism is misplaced. If the property is really an accidental one, then just change the prior work — it’ll take a few days and then the problem goes away, much easier than spending 6 months to a year on your new system. Meanwhile, if you are criticizing prior work for an essential property, then is well. By definition, there is no alternative implementation of the system you are criticizing that could possibly avoid that limitation. Thus you’re on firm ground to criticize it for that reason.

Applying philosophy to write better papers (and system design docs)

This notion of essential and accidental properties can also be applied to analyze one’s own system.

1. First of all, if you are proposing a new system, you must take the effort of articulating the conceptual changes, not just the engineering ones. Research papers must describe new concepts, not just new engineering work; this is what reviewers mean when they say that a paper is just engineering (prior to rejecting it). Of course, any new concept must be operationalized in order to evaluate it, but the contribution of the paper is the ideas, not the implementation. Make sure that your paper points out the essential properties of your idea.
2. To complement this, when I describe the limitations of my systems, I like to divide the section into two parts. The first part describes the essential limitations, and the second the accidental limitations. I take the time to make sure my reader sees which shortcomings are “baked in”, and which could be changed with a bit of coding. I hope that such an analysis helps future readers avoid making this class of analysis error, and to help them see the directions I perceive for substantial improvements (read as: new papers) by distinguishing the conceptual vs. implementation choices in my work.

What about non-systems papers?

I use this same kind of thinking when I write empirical software engineering papers. In such works, the convention is to write a Threats to Validity section which delineates threats by construct, internal, and external categories. The temptation is to enumerate all the ways in which your study might be biased, from the transcription system you used, to the projects you selected, through to what you ate for lunch that day. Of course, not all of these things are essential limitations of your work — some of them are accidental. Does it really matter which transcription service you use for your interview data? Does it really matter which survey software you used to collect survey data? No, I don’t think so. On the other hand, it matters quite a bit which software you selected for analysis, which demographic characteristics you controlled for in a sample, and which defects you selected for analysis. I always ask my students to describe the essential limitations of our work more than the accidental ones. Sometimes there are substantial “accidental” concerns that must be acknowledged, but we always start by describing what we perceive as the essential ones. Again, part of the goal is to help readers see where there is room for more research papers!

Further reading

1. Stanford encyclopedia entry on Essential vs Accidental Properties: https://plato.stanford.edu/ENTRIES/essential-accidental/
2. Lamport’s advice to State The Problem Before The Solution helps me articulate the conceptual contribution prior to the embodiment: https://lamport.azurewebsites.net/pubs/state-the-problem.pdf