A funny thing happened on the way to the cloud….

Three years ago this cloud stuff was easy. We all knew what Infrastructure-as-a-Service was: it was what Amazon offered, and what NIST had codified. Self-service on-demand pay-as-you-go pooled resources, delivering customer agility, low cost, and operational efficiency. Anyone saying anything different was probably a desperate vendor trying to “cloud-wash” an old technology. And so a number of efforts were started to create software systems that could deliver the functional and operational benefits of this vision. Eucalyptus, Open Nebula, Cloud.com, Nimbula, Joyent, CloudStack. The common thread was the “As A Service” paradigm.

For most of these initiatives, this vision has remained intact. While fortunes have waxed and waned, companies like Eucalyptus and the newly open sourced CloudStack have persisted in delivering systems defined in a top-down manner by their APIs. But for OpenStack, things are not so straightforward.

The first thing that caught my attention was when Brocade and others led an effort to add Fiber Channel support to OpenStack. That was odd – FC didn’t sound like the kind of technology that belonged in an AWS-like cloud. Then I heard of a customer with a diverse collection of network gear that wanted to use Neutron (the OpenStack networking subsystem) to orchestrate the VLAN provisioning process. They weren’t planning to use any of the rest of OpenStack, but they thought they could hack this together. Really? And then I heard a murmur that became a roar: customers looking for a cheap (preferably open source) alternative to VMware for managing their VM clusters. In the same way that Oracle users rebelled and (ironically) flocked to MySQL, people wanted to avoid the VMware tax.

The important thing to note is that this is different from the original move to the cloud. It’s bottom-up, not top down; it’s looking for new tools to manage existing infrastructure and applications, rather than a green-fields vision implemented with good-enough white-box gear. The scale is relatively small, and performance is more important than homogeneity. Most significantly, the organizational refactoring required to deliver “As A Service” is usually absent. The emphasis on industry standard APIs is largely absent, because this kind of “private cloud” is as self-contained as the VMware cluster it replaces.

The upshot is that while OpenStack remains a single open source project (with numerous sub-projects), it is serving a range of different customer use cases; everything from the massive public clouds of Rackspace and HP to ex-VMware clusters. It’s fairly obvious that this unexpected diversity of uses will have an effect on the project: architecture, development, conformance, releases, and so forth. We should expect to see an increase in the number of projects that seek to support multivendor configurations, vendor-specific API extensions, and interoperability with legacy management systems. This will make it more difficult to achieve semantic consistency, which will limit interoperability and may lead to de facto forking of the project.

As I write this, the debate about AWS API compatibility in OpenStack is raging once again. I happen to agree with Simon Wardley and Randy Bias that it is dumb for OpenStack to de-emphasize compatibility with the industry-leading API. (As I’ve said elsewhere, it’s not about exposing API endpoints; it’s all about the deep semantics.) As Randy wrote earlier:

We wanted something that would be as compatible as possible with Amazon Web Services (AWS) and our version of OpenStack delivers on this promise.

Looking ahead to the next three years, there are significant threats to the ability of OpenStack to continue on the route to AWS compatibility. We cannot let that happen.

Simon Wardley, a Cloudscaling advisor and prominent thought leader in elastic cloud computing, is fond of saying that OpenStack’s only chance of survival is to immediately embrace the AWS API set, supported by architectural configurations that allow the infrastructure to deliver services that are behaviorally consistent with their AWS counterparts.

Simon and I agree on this. By its sixth birthday, OpenStack needs to have accomplished this objective if not as an exclusive path then as a fully supported alternative to a differentiated API set.

But as I look at what’s happening to OpenStack, it seems less and less likely that they can succeed. OpenStack is becoming a vast matrix of plugins, mixins, drivers, hypervisors, proxies, and other configurable systems, held together with a loose collection of shared services. For people seeking to put together a homegrown replacement for VMware, it’s an attractive (if complicated) proposition. For interoperable NIST-inspired clouds, not so much.

POST SCRIPT:
Please don’t infer from this that I’m down on OpenStack. I think that the idea of building an open source alternative to VMware is challenging and fascinating, and there’s obviously a big market for it. In fact I’m working on a really exciting project which I hope will contribute to it. But we probably need to come up with some new language to avoid the confusion and potential equivocation around “cloud”.

3 Comments to "A funny thing happened on the way to the cloud…."

  1. July 19, 2013 - 10:46 PM | Permalink

    I think this is a legitimate point of view. I think the unspoken assumption is that OpenStack is a complete cloud operating system. It is not. It’s a cloud OS kernel, just like the Linux kernel. You can’t just download it and go. Like Linux, it’s got a plugin architecture (LKMs anyone?) and can be made to fit anywhere. This is actually it’s strength and why it will ultimately *win*. Public clouds will have a very specific approach, but people are going to build private clouds to do a lot of different things, including being as compatible as possible with public clouds, but also crazy edge stuff as you just described.

    You don’t download Linux and run it. You get a product or distribution that solves for a specific problem area:

    - Monte Vista or Raspbian for embedded appliances
    - RedHat/SUSE for enterprise servers
    - Canonical’s Ubuntu for desktops
    - ArchLinux or Gentoo for hackers
    - CrayLinux or Beowulf for HPC and super computers

    I think it’s a mistake to look at OpenStack as monolithic. It’s a toolset, a framework, a kernel, for building a real cloud operating system, like Cloudscaling’s own Open Cloud System. There will be many options over the long haul to consume OpenStack in a manner that makes sense for your use case.

  2. July 20, 2013 - 5:02 AM | Permalink

    I think this is a particularly acute set of observations. My position on OpenStack has been (and continues to be) that its destiny is to become an element at the core of a bunch of other software, most of which will be proprietary rather than FOSS… and that creates enormous challenges for compatibility and interoperability as vendors build differentiation around that core. The private cloud voices are now stronger than the public cloud ones, within the OpenStack community — as they probably should be, given the market opportunity in the private cloud space. But the fragmentation means that the Voice of AWS, so to speak, becomes ever louder, since it is the introvertible “standard” that it makes sense for everyone to be compatible with, offering a bedrock clarity.

Comments are closed.