Norbert is a library that provides easy cluster management and workload distribution. With Norbert, you can quickly distribute a simple client/server architecture to create a highly scalable architecture capable of handling heavy traffic.

Implemented in Scala, Norbert wraps ZooKeeper and Netty and uses Protocol Buffers for transport to make it easy to build a cluster aware application. A Java API is provided and pluggable routing strategies are supported with a consistent hash strategy provided out of the box.

Norbert was designed to addresses the following challenges and hide the complexity and details from the users.

  • Providing group management. Norbert makes it easy to add or remove service nodes or to change configurations in the cluster.
  • Using software load balancing to partition workload.
  • Providing asynchronies client/server RPC and notifications

Norbert uses zookeeper for the underlying group management. Both the client and server part of Norbert communicate with zookeeper to track changes in the cluster configuration. Zookeeper ensures Norbert always has accurate and consistent information about the cluster, freeing users from having to worry about cluster management details.

Before turning a centralized service into a clustered service, you must decide how to distribute the work. First, you should choose a partition space, e.g. member Id, to split the workload across. Next, you can either choose to use a provided load balancing strategy or write your own. Norbert currently uses Protocol Buffers as the serialization strategy within the system and for RPC with your application. Netty is used to provide faster NIO.

Sample code for a Java based client and server can be found here and here.

norbert architecture