The Threshold
Part 4 of 7: After the Data
In 1979, a software engineer named Dick Grune needed a way to work on code simultaneously with his colleagues.
The problem was simple. Multiple people. One codebase. No system for managing who changed what, when, and why.
So they did what everyone did at the time.
They emailed files back and forth. They kept manual logs of changes. They coordinated through conversation. They relied on individual memory to track what the current version of anything actually was.
It worked. Barely. For a team of three or four people working on a small project.
Then teams got bigger. Codebases got more complex. More people meant more changes. More changes meant more conflicts. More conflicts meant more time spent on coordination and less time spent on building.
The informal system (email, conversation, memory) couldn’t keep up.
So someone built infrastructure.
Version control. The ability to track every change, understand who made it, know why it was made, and restore any previous state. What started as a workaround became a standard. What started as a tool for a few engineers became foundational infrastructure for every software team on earth.
Today we take it for granted. Code without version control isn’t a professional option. It’s not even a serious conversation.
But for decades, it didn’t exist. And the coordination tax that teams paid in its absence was just accepted as the cost of building software.
Until it wasn’t.
The pattern underneath the history
That story isn’t unique to software.
It’s a pattern. One that repeats across different domains, different eras, different types of organizational complexity.
The pattern looks like this.
A new kind of coordination problem emerges. Usually because something that used to be small gets big. A team grows. A system gets more complex. A process that worked at ten people breaks at a hundred.
For a while, organizations manage through informal means. Conversation. Memory. Manual processes. Workarounds built by whoever feels the pain most acutely.
The informal system works. Until it doesn’t.
At a certain threshold (a certain level of complexity, a certain scale, a certain speed) the coordination cost becomes too high to absorb informally. The tax exceeds what the organization can pay and still function well.
That’s when infrastructure gets built.
Not before. Organizations don’t build infrastructure speculatively. They build it when the absence of it becomes the constraint on everything else.
And when they do (when the right infrastructure gets built at the right moment) it doesn’t just reduce the coordination cost. It changes what’s possible. It unlocks a different scale of organization. It makes things manageable that were previously unmanageable.
And eventually, it becomes invisible. Infrastructure always does. We stop noticing it when it works. We only notice it when it breaks or when it doesn’t exist yet.
Three times this happened in living memory
Let me make the pattern concrete.
1- Customer relationships
In the early days of B2B sales, customer relationships lived in salespeople’s heads. Who the customer was. What they’d bought. What conversations had happened. What they needed next.
That worked when a salesperson had twenty accounts and had been there for ten years.
It broke when companies scaled. When salespeople left and took their customer knowledge with them. When accounts got reassigned and the new rep had no context. When leadership needed visibility into the pipeline and couldn’t get it because it lived in a hundred different heads.
The coordination cost of managing customer relationships informally crossed a threshold.
CRM got built.
Salesforce didn’t create the need for customer relationship management. The need existed for decades. Salesforce built the infrastructure when the cost of not having it became impossible to ignore.
2- Data
We covered this in Part 1.
Companies generated data long before the modern data stack existed. They just managed it informally: in spreadsheets, in siloed databases, in reports that took weeks to produce and were outdated by the time they arrived.
That worked when companies were small and data volumes were manageable.
It broke when scale crossed a threshold. When data volumes exceeded what spreadsheets could handle. When the number of source systems made manual consolidation impractical. When the cost of slow, unreliable, inconsistent data became a real competitive disadvantage.
The coordination cost of managing data informally crossed a threshold.
The modern data stack got built.
3- Infrastructure configuration
For a long time, infrastructure (servers, networks, environments) was managed manually. System administrators configured things by hand. Deployments were bespoke. Environments drifted because every change was undocumented and unrepeatable.
That worked when you had one server and one system administrator who remembered everything.
It broke at scale. When infrastructure got complex enough that manual configuration became the bottleneck. When drift between environments caused failures that took days to diagnose. When the knowledge of how the infrastructure actually worked lived in one person’s head and that person left.
The coordination cost of managing infrastructure informally crossed a threshold.
Infrastructure as code got built. Terraform. Ansible. The entire DevOps movement.
What these have in common
Every one of these infrastructure categories shares the same origin story.
Something that used to be manageable through informal coordination became unmanageable as complexity scaled.
The informal system (conversation, memory, manual process, tribal knowledge) worked below the threshold and broke above it.
The cost of the breakdown (in time, in errors, in lost knowledge, in coordination overhead) eventually exceeded the cost of building something better.
And when the infrastructure finally got built, it didn’t just solve the immediate problem. It changed the ceiling. It made a new scale of organization possible that wasn’t possible before.
CRM didn’t just make sales teams more organized. It made enterprise sales at scale possible.
The data stack didn’t just make analytics faster. It made data-driven organizations possible.
Infrastructure as code didn’t just make deployments more reliable. It made modern cloud engineering possible.
The infrastructure unlocked the next level.
The question this raises
Here’s what I keep coming back to when I look at this pattern.
Every one of these infra categories was built in response to a coordination problem that organizations had been managing informally (sometimes for decades) before someone formalized it.
In each case, the informal management worked until it didn’t. And the moment it stopped working, the moment the coordination tax became the binding constraint, the infrastructure got built.
Which raises an obvious question.
What coordination problems are we managing informally today that are approaching that threshold?
What is the thing that organizations currently handle through conversation, memory, manual processes, and tribal knowledge (that is starting to break down as complexity scales) that doesn’t yet have formal infrastructure?
I’ve been thinking about this question for a while. And I think the answer is closer than most people realize.
But to see it clearly, you need to understand one more force that’s accelerating the timeline.
Something that didn’t exist the last time this pattern played out.
Part 5 drops next week.
Until the next one,
— Adnan
P.S. Have a question or topic in mind? Drop it in the comments.
P.P.S. If this resonated, subscribe + follow on LinkedIn for more.