Apache Helix

Apache Helix is a generic cluster management framework used for the automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes. Helix provides the following features:

  • Automatic assignment of resource/partition to nodes
  • Node failure detection and recovery
  • Dynamic addition of Resources
  • Dynamic addition of nodes to the cluster
  • Pluggable distributed state machine to manage the state of a resource via state transitions
  • Automatic load balancing and throttling of transitions

How it works

Helix manages the state of a resource by supporting a pluggable distributed state machine. One can define the state machine table along with the constraints for each state.

Here are some common state models used:

  • Master, Slave
  • Online, Offline
  • Leader, Standby

How we use it

We currently use Helix to manage our search-as-a-service clusters hosting multiple search applications, Databus, our data change capture component, and Espresso, our indexed, timeline-consistent, document-oriented data store.

How to get it

Apache Helix is available for download.