Three years ago this cloud stuff was easy. We all knew what Infrastructure-as-a-Service was: it was what Amazon offered, and what NIST had codified. Self-service on-demand pay-as-you-go pooled resources, delivering customer agility, low cost, and operational efficiency. Anyone saying anything different was probably a desperate vendor trying to “cloud-wash” an old technology. And so a number of efforts were started to create software systems that could deliver the functional and operational benefits of this vision. Eucalyptus, Open Nebula, Cloud.com, Nimbula, Joyent, CloudStack. The common thread was the “As A Service” paradigm.
For most of these initiatives, this vision has remained intact. While fortunes have waxed and waned, companies like Eucalyptus and the newly open sourced CloudStack have persisted in delivering systems defined in a top-down manner by their APIs. But for OpenStack, things are not so straightforward.
The first thing that caught my attention was when Brocade and others led an effort to add Fiber Channel support to OpenStack. That was odd – FC didn’t sound like the kind of technology that belonged in an AWS-like cloud. Then I heard of a customer with a diverse collection of network gear that wanted to use Neutron (the OpenStack networking subsystem) to orchestrate the VLAN provisioning process. They weren’t planning to use any of the rest of OpenStack, but they thought they could hack this together. Really? And then I heard a murmur that became a roar: customers looking for a cheap (preferably open source) alternative to VMware for managing their VM clusters. In the same way that Oracle users rebelled and (ironically) flocked to MySQL, people wanted to avoid the VMware tax.
The important thing to note is that this is different from the original move to the cloud. It’s bottom-up, not top down; it’s looking for new tools to manage existing infrastructure and applications, rather than a green-fields vision implemented with good-enough white-box gear. The scale is relatively small, and performance is more important than homogeneity. Most significantly, the organizational refactoring required to deliver “As A Service” is usually absent. The emphasis on industry standard APIs is largely absent, because this kind of “private cloud” is as self-contained as the VMware cluster it replaces.
The upshot is that while OpenStack remains a single open source project (with numerous sub-projects), it is serving a range of different customer use cases; everything from the massive public clouds of Rackspace and HP to ex-VMware clusters. It’s fairly obvious that this unexpected diversity of uses will have an effect on the project: architecture, development, conformance, releases, and so forth. We should expect to see an increase in the number of projects that seek to support multivendor configurations, vendor-specific API extensions, and interoperability with legacy management systems. This will make it more difficult to achieve semantic consistency, which will limit interoperability and may lead to de facto forking of the project.
As I write this, the debate about AWS API compatibility in OpenStack is raging once again. I happen to agree with Simon Wardley and Randy Bias that it is dumb for OpenStack to de-emphasize compatibility with the industry-leading API. (As I’ve said elsewhere, it’s not about exposing API endpoints; it’s all about the deep semantics.) As Randy wrote earlier:
We wanted something that would be as compatible as possible with Amazon Web Services (AWS) and our version of OpenStack delivers on this promise.
Looking ahead to the next three years, there are significant threats to the ability of OpenStack to continue on the route to AWS compatibility. We cannot let that happen.
Simon Wardley, a Cloudscaling advisor and prominent thought leader in elastic cloud computing, is fond of saying that OpenStack’s only chance of survival is to immediately embrace the AWS API set, supported by architectural configurations that allow the infrastructure to deliver services that are behaviorally consistent with their AWS counterparts.
Simon and I agree on this. By its sixth birthday, OpenStack needs to have accomplished this objective if not as an exclusive path then as a fully supported alternative to a differentiated API set.
But as I look at what’s happening to OpenStack, it seems less and less likely that they can succeed. OpenStack is becoming a vast matrix of plugins, mixins, drivers, hypervisors, proxies, and other configurable systems, held together with a loose collection of shared services. For people seeking to put together a homegrown replacement for VMware, it’s an attractive (if complicated) proposition. For interoperable NIST-inspired clouds, not so much.
Please don’t infer from this that I’m down on OpenStack. I think that the idea of building an open source alternative to VMware is challenging and fascinating, and there’s obviously a big market for it. In fact I’m working on a really exciting project which I hope will contribute to it. But we probably need to come up with some new language to avoid the confusion and potential equivocation around “cloud”.