Wednesday, June 19, 2013

Exploring Eucalyptus 3.3.0: Introduction

The recent release of Eucalyptus 3.3.0 delivers meaningful features which make possible a new level of IaaS usage.  This is the start of a series of blog posts which illustrate what is now possible to do with  in Eucalyptus.

New Features

The 3.3.0 release has a number of noteworthy new features.  First, we have new services:
Then, we have expanded the known set of SDKs which are meant to work against AWS, but can also be pointed (by configuring the endpoint and service path appropriately) at Eucalyptus:
These are the two facets of the release this series of blog posts will focus on. Much more went into the release, and is fantastically covered by @mdshaonimran in his great post summarizing the changes in this release.

The fruits of our labors

Expanding the AWS services implemented and improving the coverage of SDKs enable a new (and unpeered) level of compatibility.  New because tools which depend on missing services are non-starters:  three new services arrived in this release.  Unpeered because, to date, related efforts have yet to be fruitful.  Today, Eucalyptus makes it possible to do exciting new things.

New capabilities

With the 3.3.0 release, use cases previously restricted to AWS are now possible using Eucalyptus.  I'll be writing a series of posts on how to get going , the main category is:
  • Netflix OSS:  Progressively we add the ability to run parts (and soon enough all of) the Netflix cloud platform.
    • Asgard:  Web interface for application deployments and cloud management in Amazon Web Services (AWS)
    • SimianArmy: Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
    • Edda: Service to track changes in your cloud
Subsequent posts will look at current progress on Eureka, Servo, Ice, and Priam.

Others have made similar posts and surely more will follow:

This is the story of Eucalyptus

You can read along with me in your data center.  You'll know its time to turn the page when your hear the chimes ring like *this*.

All of these posts will assume that you have a working 3.3.0 cloud setup.  To do that you have a couple of options:

Next time:  Eucaflix: Part 1 

Asgard -- Netflix OSS on Eucalyptus

Wednesday, January 9, 2013

Eucalyptus Architecture

The documents which describe the specification, design, and architecture of Eucalyptus have moved to a new home.  Now publicly available on github and pull requests are welcome check out the documents at :

Briefly, the repository eucalyptus/architecture contains the technical specifications, design, and high level architecture documents used during feature planning and development.  The purpose of these documents is to serve as technical specifications, background, analysis, and record of architecturally significant decisions in support of the development of all new features in Eucalyptus.  These would be of interest to those looking to extend or contribute to Eucalyptus.

NOTE: The repository is evolving in a couple of ways:  First, it is not retroactively complete; docs for past features are missing.  Second, it will be ever changing for on-going feature work.  Expect the first to be addressed in the near future when the documents exist (not always the case), but the second is inherent with the purpose of this repository.  If you wish to see more long-lived and stable architecture documents please refer to eucalyptus/architecture-docs.

What is there...

The repository is the canonical reference for two classes of documents:  
  • Feature specifications/designs which are release-independent descriptions of the functional characteristics and quality attribute objectives for each feature.
  • Generational specifications/designs which are release-specific subsets of a feature's technical specification as it relates to the context of the release which evolve iteratively with the ongoing implementation during a release.

What it is used for...

The purpose of the documents aggregated there is to support the development of new features in Eucalyptus. To that end, they play a role in several stages of the development process:
  1. Feature specification: associated with each feature is an overall document describing the important technical characteristics in a release (or generation) independent fashion. These reflect the understanding of the feature which is used in subsequent steps.
  2. Scope & Estimation: prior to release planning a technical assessment is made about the aspects of functionality to go into a generation of the feature. The spec documents which are associated with a release start life at that time. These subsequently evolve until they have been committed to for a particular release (but may be present even when not committed to a release).
  3. Design & Implementation: a feature which has been committed to a release is supported by a design document for that release. The design documents start their life when work towards that release does. These subsequently evolve until work stops for that release of the feature. (They would then be ported forward to the final documentation).

What you will find here...

There are two classes of documents which you can expect to find the authoritative copy of here: feature specifications used in planning/estimation of releases and generational design documents which evolve iteratively with the ongoing implementation of the system.  

Feature Specification documents are statements of our intentions, understanding, and long-term scope:
  • are "timeless" in that they are overarching and meant to be definitive of the feature overall.
  • evolve over time independent of the features current implementation status.
  • reflect our current understanding of the features definition in the broadest sense.
Feature Specification documents consist of:
  • Specification: overall technical specification of the functional and architectural/quality characteristics.
  • High level design/Architecture: definition of fundamental components, interfaces, behaviours including information, control, and concurrency models.
  • Supporting Documents: API/Service specifications, client tool chains, WSDLs, TCKs.
Generational Documents are specific to a release and meant to serve the tasks surrounding the planning, design, implementation, and delivery of a feature.
  • Serve to support the scoping, design, and implementation effort of a feature during a release.
  • Define the design and implementation objectives and constraints for a particular version of a feature.
  • Specific to the context and determined by the constraints of a particular release time frame.
  • Change as needed to support the above objectives and are quiesced after those tasks are completed.
Generational documents consist of:
  • Functional requirements: as identified by a corresponding epic in the Eucalyptus Bug Tracker.
  • Specification: release-specific technical interpretation of functional requirements 
  • Designs: release-specific design models and descriptions, starting with just-enough and co-evolving with the implementation.

What's good...

Currently, there are two categories of documents in the repo worth taking a look at:
  • 3.2 design docs which talk about what was done in Eucalyptus 3.2
  • 3.3 design docs that are being used during the design and implementation of Eucalyptus 3.3

What's next...

Following posts will highlight things from the design of Eucalyptus, both past and present. Additionally, you can expect to see more on the various testing, debugging, development, and design topics relevant to building Eucalyptus.