I’m incredibly proud to announce the general availability of System Initiative. It’s a revolutionary technology that is the future of how you will build DevOps automation. That’s a daunting sentence to write. As soon as I do - my brain immediately recoils. It says, “You shouldn’t say that. How can you be sure? Won’t it just read as meaningless hype?”

But if I’ve learned anything about building products for people, it’s this: you have to build the thing you believe should exist in the world, you have to fall in love with it, and then you have to let other people (hopefully) love it as you do. And the truth is - I love System Initiative. It’s bold, it’s novel, it’s fun to use, a delight to program, and extremely powerful. If I didn’t start by telling you it’s the future, I’d be doing you a disservice. But worse - I’d be doing myself a disservice. I would be shrinking from what I know to be true because I’m afraid about how you’ll receive it. At this point in my career? I’m done holding back. It’s the future.

I’ve been working on it for the last 5 years, but it feels like I’ve been working on it my entire career. All I’ve ever done professionally is build tools to help people build things on the Internet. I’ve always admired people who took incredible risks and built technology that changed the way we think about what’s possible. The kind of technology that, even if it isn’t a commercial success, leaves an indelible mark. System Initiative is that technology for me. It’s the best thing I can imagine building. It holds nothing back. It’s as close to a pure expression of what I love about technology, about startups, and about entrepreneurship as I can imagine making.

I built it because the path we have been on, where we build infrastructure with the same approaches and tools we use to build applications - is a technological dead end. We’ve pushed it as far as it can go. We can still make small improvements on the margins, but that’s all they will be. Small improvements that don’t really move the needle much at all on the outcome. Without System Initiative, I feel like we’re all going to be stuck - replaying the same essential pattern over and over again, pretending it will be different this time. The choice wasn’t to make System Initiative or something else - the choice was to make System Initiative or find something else to love because loving DevOps and automation was no longer an option. I simply couldn’t watch the people who love what I love struggle the same way anymore.

So we (myself, my co-founders, and everyone who works here, past and present) took a step back. We asked ourselves what we would want the experience of building technology businesses to be like if we weren’t constrained at all by prior art. If there were no limits to what we could build. No rules we couldn’t break. No problem too big to tackle. To succeed, we would have to do the best work we’ve ever done. Build the best team possible. Push each other to go further. To not settle. System Initiative is the result.

Let’s discuss how it works and why it’s the revolutionary new foundation we need.

One undeniable truth about the experience of “doing DevOps” today - the feedback loops are awful. It takes forever to get started, it takes forever to know if you did something right or wrong. It takes forever for your pipelines to run, to get your code reviewed, to know if you’ve caused a security vulnerability, to know if you’ve gone over budget. It’s like we are all driving our own submarines, communicating only by radio. It’s deeply annoying how slow it is.

One cause of our feedback loop problem is that we’re bound up by the constraints of how other people implement their systems. For example - it might take AWS up to 10 minutes to provision an EKS cluster. The current tools can tell you if you have the syntax right, but they can’t tell you if it will work or not. To learn that? Gotta wait 10 minutes and pray for rain.

The other source of poor feedback loops is that we represent the definition of our infrastructure as code in the first place. This was a good idea at the time - we gained all the benefits of version control, we could do code review, build pipelines, and do continuous integration and delivery. But code is less malleable than data - it gets locked up behind syntax, it grows in complexity as the problem domain expands, as the abstractions pile up. It’s a static representation of a dynamic environment. With application code it works great - because the results are usually straightforward: an application artifact. But with infrastructure? It turns out to be a very awkward fit. It works, I’m not contesting that. But all you need to do to understand why it’s awkward is look at how hard we’ve pushed to effectively store and manage state, to artificially segment our infrastructure due to constraints of the implementation, and our failure to inspire our peers to work with us on complex infrastructure. It works, but it’s not nearly good enough.

How do other industries solve this problem? They build simulators. So that’s what we did too - we built a system of 1:1 digital twins of cloud infrastructure. We decoupled tracking the real state of the infrastructure from what you might hypothetically want to change. This lets us solve the feedback problem - the simulator doesn’t need to build that EKS cluster; it just needs to know if your proposed configuration would build an EKS cluster that works. We keep track of both things - the real data about your cluster (what we call a resource), and the hypothetical configuration in your digital twin (a component). Both are the “truth” in different contexts. Both are structured data, updated at different intervals. They are not code.

One excellent side effect of building digital twins? They eliminate the 200% problem. In case you don’t know, the 200% problem is a perennial issue with automation abstractions - to automate something, you need to understand both 100% of the underlying domain and 100% of how that domain maps to the way the tool understands the problem. With digital twins, if you know how a VPC works in AWS, you already know how to automate it in System Initiative. Because it’s a literal 1:1 translation. There is no loss of fidelity. You don’t learn AWS and then System Initiative. You just learn AWS, and use System Initiative to express what you already know.

Here’s another truth about DevOps - it’s always complicated. When I was the CTO of Chef, I had the privilege of working with teams from some of the largest, most complex enterprises - from leading technology companies to global banks and manufacturing. What is the number one thing they all had in common? They had unique, complex technology solutions. None of them did anything the same way. None of them had the exact same solutions to the same problem. All of them had pragmatic reasons for the choices they made. If you worked at Meta on Facebook, you wore this as a source of pride - you built things most people couldn’t imagine building. If you worked at a global bank, you probably started by apologizing about how backward everything was, followed by letting me know it’s like this because you power the entire global economy (which, for the record, is an outstanding reason for technical complexity). But I never once encountered a company whose technology infrastructure was “simple.”

That’s why System Initiative was designed, from the start, to be as powerful and expressive as possible. You can’t predict what people will need a platform like this to do. No matter how hard you try to reduce it or how much you learn, the world will remind you that it’s more complicated than you think. On the bright side - the people working in those companies are some of the most intelligent, capable people I’ve ever met. If you give them the power to solve their problems, they will.

So the next challenge was to make these high fidelity digital twins programmable. It had to be easy to create new models, modify existing ones, express policy, and use the information from one model to inform the configuration of another. The answer was moving the digital twins to sit on top of a graph of reactive functions. Everything about them - their schema, the values of their properties, the actions they can take to impact the real world, their validations and qualifications - is defined as source code that is reactive to its inputs. You declare the inputs and the outputs, and the system executes the right code at the right time. You can quickly and easily write any code you need - from the most straightforward model to the most highly customized enterprise requirement, directly within System Initiative.

The impacts here are profound. When modeling AWS IAM policy in System Initiative, we realized that AWS provides a sophisticated Policy Simulator. So we modeled it, connected our IAM Policies and resources to it, and had a new, real time interface to test the validity of IAM policy. It took less than an hour from start to finish.

Let your imagination go for a minute about what you can build with this primitive. How would application deployment be expressed differently? Security and compliance policy? Financial budgeting? Network configuration? It’s a significantly more powerful way to program complex automation.

Of course, moving to a big graph of reactive functions brings a significant problem - what do you do about source control? The answer is both straightforward and audacious: you build version control into the platform directly, making it a hypergraph of functions[^1]. System Initiative has built in Change Sets - essentially forks of the entire graph, which allow you to make changes safely without impacting the real world. Change Sets include trivial things (like setting properties) and complex operations (like changing the source code). A lovely side effect of integrated Change Sets is that they automatically update when the “real world” changes - as resources change, or as people apply their changes to the world, they are immediately reflected in any open change sets. Your view never gets stale, you never wind up with a situation where the ‘plan’ no longer reflects the ‘apply’, there is nothing to ‘rebase’, and there is nothing to set up or maintain.

This brings us to another core problem with the status quo of DevOps automation. We’ve known from the start that the biggest key to unlocking great results in an organization is the amount of collaboration between the stakeholders. How close can Dev and Ops get? Security? Finance? Legal? The closer they get, the better the outcome. Even with a new software development primitive like a reactive hypergraph of functions, there is no reason to believe it would bring about better outcomes unless it also brings people together by default.

We accomplish this in System Initiative by building models that exist at various levels of semantic scale, and then building interfaces that make sense at those levels. For example - your application likely has properties that are configurable, actions it can take (like being deployed or built) and infrastructure it requires. You can use the same fundamental system of digital twins we use for low level infrastructure and express the impact with relationships. Eventually[^2], we will provide a custom interface specifically for each job, built from the same underlying data and kept up to date through simple usage. By giving people a view into the data that best fits their needs, we enable them to work together - even when they’re just doing “their” part of the job.

The second part: we extended the reactivity of functions to the user interface and made everything fully multiplayer. Need some help from an expert figuring out how to get that container deployed? They can log in to System Initiative and go directly to your Change Set. Then, you can make changes together and see the impact in real time. The difference here is remarkable - people can easily work together again. It makes the whole experience of working on infrastructure dramatically more effective and way more fun.

Think about the product implications here - fully multiplayer experiences, with interfaces designed for the specific semantics of a particular job, all working from a common baseline of reactive digital twins. This approach will create radically better experiences than the status quo - both in huge technology companies and our most complex enterprises. It will unlock the level of capability we’ve all been striving for the last 15 years.

Our first example of this is with managing infrastructure. With System Initiative, infrastructure engineers work together on a living architecture diagram. The relationships between components are configuring reactive inputs to functions that set their properties. It keeps track of the actions you’ll need to take to reconcile the simulation with reality. You can extend the system, embed policy, and collaborate in real time. It’s intuitive, it’s powerful, it’s multiplayer - and it’s incredibly fun.

The final piece of the puzzle was thinking through how a technology with this much potential gets set up to be as successful as possible. For me, there was only one answer to that question: it needed to be 100% open source software, with an open and welcoming community. We welcome you to build it with us, to build on top of it, to build a business on it, to fork it - whatever you need or want to do. Our goal is to be an incredible steward of this technology. I have complete faith that if we do that, we can all prosper.

System Initiative is the best work I can imagine doing. I love it. I believe it’s the future of DevOps Automation. Today it replaces Infrastructure as Code. Tomorrow, it will transform how you work together with your peers in the most complex environments on the planet. As of today, it no longer belongs to me. It belongs to all of you. I hope you love it too. Use it to make your work life dramatically better. Work more closely with your peers. Have fun. Build a better future for yourself. I know I am.

[^1]: It’s not actually a Hypergraph in the mathematical sense - those are when a graph has edges that can point to many nodes. It’s just what we started calling the design, and it stuck before we noticed it was a prior term of art. Forgive us, math friends.

[^2]: You can model applications today, but it’s a little awkward. The near-in road map includes building more complex management components, and slightly further out the ability to make role-specific visualizations. Today, System Initiative is great at the layer of Infrastructure.

Image from tsaiproject flickr under the CC-BY-2.0