Engineering Ideas #2
Simplicity, language of software design, categories of objects in OO code, trust, local vs. global objectives, noting teammates’ strengths
Simple Made Easy
One of those instant classic talks by Rich Hickey with plenty of powerful thoughts. First, a reminder:
Simplicity is a prerequisite for reliability. — Edsger Dijkstra
For me, this echoes the idea of “embracing operational simplicity” from Edmond Lau’s The Effective Engineer:
Simple solutions impose a lower operational burden because they’re easier to understand, maintain, and modify. At Instagram, simplicity was a key principle enabling the team to scale.
Hickey explains that simple means “one-folded”: one task, one role, one concept. It’s objective. Easy means “convenient, at hand, familiar”. It’s relative.
Modularity is not a silver bullet:
Modularity (partitioning and stratification) do not imply simplicity but are enabled by it. Don’t be fooled by the code organization.
Another quote from Edsger Dijkstra, in which I particularly like the part about “vigorous separation of concerns”:
Programming, when stripped of all its circumstantial irrelevancies, boils down to no more and no less than very effective thinking so as to avoid unmastered complexity, to very vigorous separation of your many different concerns.
Since simplicity means separation of concerns, the following idea becomes almost obvious:
Simplicity often means making more things, not fewer.
Which reminds me of a related observation made by John Ousterhout in
A Philosophy of Software Design:
I have seen frameworks that allowed applications to be written with only a few lines of code, but it was extremely difficult to figure out what those lines were. Sometimes an approach that requires more lines of code is actually simpler, because it reduces cognitive load.
Hickey had an insight into why Apache Kafka was to grow so massively popular in the following decade (Kafka was open-sourced less than a year before his talk):
If thing A calls thing B you just complected them with “when” and “where”. A has to know where the B is, and when that happens is whenever A makes the call. Queues are a way to get rid of this problem. If you are not using queues extensively you should start, right away.
It appears that a strong type system in the programming language doesn’t help to build simpler systems. (On the contrary, developers may probably even think that the type system will help them to handle extra complexity, which they wouldn’t dare to introduce in a project written in a language with dynamic typing.)
All reliability tools that you have (tests, type systems) are not about simplicity and don’t touch the core of this problem. They are safety nets.
Software Design and the Physics of Software
What if we got a sub-optimal language for software design?
Carlo Pescio advocates moving from the informal, vague, and inconsistent language we currently use to speak about software design and decisions, which he compares with the language of medieval alchemy, to reasoning in terms of precise definitions, which he compares to the scientific process in modern physics and chemistry:
We often speak of software as possessing properties, usually expressed using –ilities (extendibility, reusability, scalability), or by adherence to some principle (separation of concerns, don’t repeat yourself), or defined by metrics (coupling, cohesion) or even using broad, subjective characterizations as “fragile”, “robust”, “clean”, etc.
This is also true for physical materials: we define a material as a conductor, or an insulator; as being compressible or incompressible; etc. However, a property in physics is always defined in relationship to a (generalized) force applied to the material. We speak of conductors and isolators because we have a notion of electrical current and electric potential difference; we speak of compressibility because we have a notion of compression; etc.
In software, we speak of properties like “extendibility” or “fragility” but we’re left to vernacular usage or arbitrary definitions, because the forces at play are unknown and undefined. Yet there are forces acting on our software. We just don’t know them well and we don’t use them well when we design software structures.
According to Pescio, there are several problems in speaking informally about software design:
Inconsistencies and vagueness in the meaning people assign to different
“-ilities” ignites a lot of unnecessary debates.People with the best rhetoric, not the best design often win the argument when scientific principles are not used.
Language shapes thinking (and designs). Better language may inform us to design better software abstractions and systems. In the age of alchemy, people were capable to build intricate cathedrals, while modern science has enabled skyscrapers and spaceships.
To me, this is an interesting direction of thought that we may utilize to form some effective mental models and to obtain some specific results.
However, in general, I think there is a limit to how far this way of thinking could be applied.
The processes of developers learning, reasoning about, and changing software, and the models they form in their heads about it are inextricably linked to the software itself. All developers have different skills and experience, and all humans’ brains are different: for example, some people are more comfortable working with small functions, others with large. Some can more easily reason in terms of data and functions, others — in terms of objects or actors. Minimizing the mental energy developers spend while working with some software is one of the main objectives of software design, so these effects must be taken into account.
This means that much of the software design is fundamentally subjective. In teams, we can talk about how comfortable are the developers working with certain techniques and patterns “on average across the team”, but most teams are small enough so that the individual preferences can not be ignored.
In the project about software design that I develop, I try to get across a less radical idea of differentiating between high-level and primary software qualities. High-level qualities such as “readable”, “easy to change”, or “flexible” are particularly vague and subjective, while primary qualities like “succinct”, “navigable”, or “configurable” tend to be less subjective. I propose to focus the discussions about software design around the primary qualities, though different developers may prioritize these qualities differently in their designs.
ZIO environment meets constructor-based dependency injection
Adam Warsky comes up with a crystal clear classification of objects appearing in the application code:
A dependency is a type which instances are static, global and stateless:
Static: as it’s created on application startup, possibly depending on configuration, but not on dynamically-provided data.
Global: there’s usually a single instance of a given dependency in an application. You might think of this as the “singleton” scope.
Stateless: dependencies don’t usually carry any state (especially mutable state!), and are a collection of functions/methods implementing some business logic.
In other words, dependencies are service-like objects. This is opposed to data-like objects, which in contrast are dynamic, local and stateful. One exception here is configuration, which is usually static, global and stateful.
Summing up, we can roughly categorise the objects in our applications into three categories, out of which the first two can be treated as dependencies for other service-like objects:
service-like objects: static, global, stateless
configuration: static, global, stateful
data: dynamic, local, stateful
Adam also proposes to distinguish between outside world integration and business logic dependencies, and use different programming mechanisms for the two:
ZIO environment can be used for:
dependencies, which should be made explicit to the caller
low-level, effectful dependencies, which directly integrate with the outside world, such as a database connection pool, email service integration or a message broker interface.
Modules wired using constructors are best for:
dependencies, which should be hidden from the caller
business logic, encapsulated into manageable, readable, functional pieces. This business logic might use the lower-level dependencies from the ZIO environment to implement their functionality.
configuration
Goodbye, Clean Code
Continuing on the topic of trust from the previous week. Dan Abramov notes a way to damage the trust within the team — to rewrite someone’s code without asking them, which is easy to overlook:
I didn’t talk to the person who wrote the code. I rewrote it and checked it in without their input. Even if it was an improvement (which I don’t believe anymore), this is a terrible way to go about it. A healthy engineering team is constantly building trust. Rewriting your teammate’s code without a discussion is a huge blow to your ability to effectively collaborate on a codebase together.
Algorithms interviews: theory vs. practice
Dan Luu suggests that focusing on local objectives too much may not be optimal for the company in general:
When measured on actual productivity, that was the most productive company I’ve worked for. […] I think it helped that we didn’t filter out perfectly good candidates with algorithms quizzes and assumed people could pick that stuff up on the job if we had a culture of people generally doing the right thing instead of focusing on local objectives.
How to Leverage Your Teammates’ Strengths More
Edmond Lau on why not only sharing own vulnerability but also celebrating other people’s strengths can unlock the extra potential of the team:
Jean later told me that she wouldn’t have had the courage to boldly tell me to write my email in five minutes had we not had a conversation around the strengths we saw and valued in each other.
This explicit, shared awareness around each other’s strengths changed each of our behaviors. It made it easier for me to ask for help. And it gave her fuller permission to leverage her strength because she knew I valued it.
…
That’s where a shared awareness around your impact comes in. If you know that other people on the team value your leadership and your contributions, you feel a stronger sense of permission to continue to step up.
And when everyone on the team feels like they’re leaders who are aware of their impact, that’s when you have a solid foundation for an exceptional team.
You can subscribe to new Engineering Ideas via RSS (e. g. using Feedly) or e-mail.