Wednesday, January 9, 2013

Eucalyptus Architecture

The documents which describe the specification, design, and architecture of Eucalyptus have moved to a new home.  Now publicly available on github and pull requests are welcome check out the documents at :

Briefly, the repository eucalyptus/architecture contains the technical specifications, design, and high level architecture documents used during feature planning and development.  The purpose of these documents is to serve as technical specifications, background, analysis, and record of architecturally significant decisions in support of the development of all new features in Eucalyptus.  These would be of interest to those looking to extend or contribute to Eucalyptus.

NOTE: The repository is evolving in a couple of ways:  First, it is not retroactively complete; docs for past features are missing.  Second, it will be ever changing for on-going feature work.  Expect the first to be addressed in the near future when the documents exist (not always the case), but the second is inherent with the purpose of this repository.  If you wish to see more long-lived and stable architecture documents please refer to eucalyptus/architecture-docs.

What is there...

The repository is the canonical reference for two classes of documents:  
  • Feature specifications/designs which are release-independent descriptions of the functional characteristics and quality attribute objectives for each feature.
  • Generational specifications/designs which are release-specific subsets of a feature's technical specification as it relates to the context of the release which evolve iteratively with the ongoing implementation during a release.

What it is used for...

The purpose of the documents aggregated there is to support the development of new features in Eucalyptus. To that end, they play a role in several stages of the development process:
  1. Feature specification: associated with each feature is an overall document describing the important technical characteristics in a release (or generation) independent fashion. These reflect the understanding of the feature which is used in subsequent steps.
  2. Scope & Estimation: prior to release planning a technical assessment is made about the aspects of functionality to go into a generation of the feature. The spec documents which are associated with a release start life at that time. These subsequently evolve until they have been committed to for a particular release (but may be present even when not committed to a release).
  3. Design & Implementation: a feature which has been committed to a release is supported by a design document for that release. The design documents start their life when work towards that release does. These subsequently evolve until work stops for that release of the feature. (They would then be ported forward to the final documentation).

What you will find here...

There are two classes of documents which you can expect to find the authoritative copy of here: feature specifications used in planning/estimation of releases and generational design documents which evolve iteratively with the ongoing implementation of the system.  

Feature Specification documents are statements of our intentions, understanding, and long-term scope:
  • are "timeless" in that they are overarching and meant to be definitive of the feature overall.
  • evolve over time independent of the features current implementation status.
  • reflect our current understanding of the features definition in the broadest sense.
Feature Specification documents consist of:
  • Specification: overall technical specification of the functional and architectural/quality characteristics.
  • High level design/Architecture: definition of fundamental components, interfaces, behaviours including information, control, and concurrency models.
  • Supporting Documents: API/Service specifications, client tool chains, WSDLs, TCKs.
Generational Documents are specific to a release and meant to serve the tasks surrounding the planning, design, implementation, and delivery of a feature.
  • Serve to support the scoping, design, and implementation effort of a feature during a release.
  • Define the design and implementation objectives and constraints for a particular version of a feature.
  • Specific to the context and determined by the constraints of a particular release time frame.
  • Change as needed to support the above objectives and are quiesced after those tasks are completed.
Generational documents consist of:
  • Functional requirements: as identified by a corresponding epic in the Eucalyptus Bug Tracker.
  • Specification: release-specific technical interpretation of functional requirements 
  • Designs: release-specific design models and descriptions, starting with just-enough and co-evolving with the implementation.

What's good...

Currently, there are two categories of documents in the repo worth taking a look at:
  • 3.2 design docs which talk about what was done in Eucalyptus 3.2
  • 3.3 design docs that are being used during the design and implementation of Eucalyptus 3.3

What's next...

Following posts will highlight things from the design of Eucalyptus, both past and present. Additionally, you can expect to see more on the various testing, debugging, development, and design topics relevant to building Eucalyptus.