OpenStack and AWS APIs: Getting to the core of the matter

First, go and read Randy Bias’s open letter to the community: OpenStack’s Future Depends on Embracing Amazon. Now. I strongly agree with him that without full AWS API compatibility (not just syntax, but deep semantics), OpenStack will be unable to deliver on the hybrid future which most of us expect. (Gartner gets this right.)

But then what? Suppose that the OpenStack board said to Randy, “You’re right. We agree 100%. Let’s commit ourselves to this goal.” Then what? What would be different?

So your next reading assignment is the series of blog posts by Rob Hirschfeld (first here, second here, more coming) on how we should think about the core of OpenStack.

One immediate concern that we would have to address is what we choose to define as “AWS compatibility”. It’s obvious that AWS has been adding features faster than OpenStack, and that in a very real sense we’re falling behind. On the other hand, the adoption of these new features is slower; it’s gated by many factors, including the rate of change of layered tools such as those from Rightscale and Enstratius. So while the AWS API is a moving target, it should be possible to define a compatibility roadmap.

We also need to consider the status of existing OpenStack features and contributions that are incompatible with AWS APIs. There are plenty of these in every area – compute, object storage, block storage, networking, security, and so forth. In some cases there are missing features, which it should be possible to add in. (I’m looking at Swift here.) However what do we do about incompatibilities that arise because of third-party components that OpenStack has incorporated but does not control? Do we exclude them (difficult)? Maybe we introduce a new feature categorization, in which components would be tagged as “AWS compatible”; anyone who wanted to assemble an OpenStack system which delivered AWS API compatibility would have to use only “AWS compatible” subsystems. This also feels deeply problematic, and could well lead to a fork.

I don’t know what the answers will be to these questions, nor how the answers will influence the overall outcome. However I believe that those of us who would advance the case for AWS compatibility need to dig into this. We need to come up with a recommendation that spells out the consequences, so that the community can make a fully informed decision.

5 Comments to "OpenStack and AWS APIs: Getting to the core of the matter"

  1. July 24, 2013 - 9:34 AM | Permalink

    Geoff, a serious question:

    What, in your opinion, prevents users who care about AWS compatibility from simply choosing Eucalyptus? Because from where I sit, it looks like OpenStack has essentially ceded this market to us, believing that AWS compat is more of a hindrance than a benefit. And we have accelerated our AWS compat efforts.

    So why not Euca?

  2. July 24, 2013 - 6:34 PM | Permalink

    Eucalyptus has a long history of not embracing the ecosystem. I was hopeful it would have changed when I met with Marten and team last year; it didn’t. Eucalyptus has determined it’s own fate.

  3. July 26, 2013 - 12:51 PM | Permalink

    There is a reason that NONE of the “stack-specific” APIs have reached the level of acceptance of OCCI, which is now the second most popular cloud infrastructure API – second only to AWS – and has adapters or maybe interfaces to ALL of the major cloud stack projects such as OpenStack, Cloudstack and OpenNebula. It is not about the API! In the words of Nick Barcet of the Ceilometer project, “the API should be pluggable!” The point of each cloud software project should be to deliver best value and usability to its users and allow the API to deliver interoperability and access to the software features.

    OCCI provides a platform- and stack-independent way to deliver access to these features without getting in the way of software development, and has already been used to provide many working examples of cloud federation, interopable operation and cloud brokerage in practice WITHOUT getting in the way of continued development of features of each of the software to which it provides interfaces. The concept of mixins provides a flexible and extensible way to add capabilities, and the basic features that allow it to work with all of the major stacks are already there. Plus, it is developed in an open process with open access that enables full community participation in further development – something that AWS has never had, and probably never will. OCCI already runs at scale in several major projects and has a large number of downloadable open source implementations. Visit or search on “OCCI cloud” for more.

    [WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.

Comments are closed.