Simula Research Laboratory / Center for Resilient Networks and Applications / NorNet
Homepage of Thomas Dreibholz / Thomas Dreibholz's Reliable Server Pooling (RSerPool) Page


Welcome to Thomas Dreibholz's
Reliable Server Pooling (RSerPool) Page
The RSPLIB Project
The RSPLIB Project



📰 News


📺 RSerPool/RSPLIB in the Media

October 27, 2010

Our cooperation project with Hainan University on RSerPool optimizations has been awarded with the "Scientific Research Accomplishments of Institutions of Higher Education of Hainan Province Award". A scan of the award document, including an English translation, can be found here!

November 30, 2009

I have been awarded for my work on RSerPool with the award "Wissenschaftspreis der Sparkasse Essen".

News Articles

Photos

September 26, 2008

RSerPool/RSPLIB presentation to software developers and press in Haikou, Hainan/China.

TV Report

News Articles

Photos


💡 What is Reliable Server Pooling?

Introduction

The development and standardization of an application-independent server pooling architecture has been set as the goal of the IETF RSerPool WG. As a result, the working group has created their concept Reliable Server Pooling, abbreviated as RSerPool, which at the moment consists of eight RFCs, several Internet Drafts and our prototype RSPLIB as reference implementation.

Requirements to the Reliable Server Pooling Architecture

As key requirements to the Reliable Server Pooling architecture, the following points has been identified:

Lightweight

The RSerPool solution may not require a significant amount of resources (e.g. CPU power or memory). In particular, it should be possible to realize RSerPool-based systems also on low-power devices like mobile phones, PDAs and embedded devices.

Real-Time

Real-time services like telephone signalling have very strict limitations on the duration of failovers. In the case of component failures, it may be necessary that a "normal" system state is re-established within a duration of just a few hundreds of milliseconds. In telephone signalling, such a feature is in particular crucial when dealing with emergency calls.

Scalability

Providing services like distributed computing, it is necessary to manage pools of many hundreds or even thousands of servers (e.g. animation rendering pools). The RSerPool architecture must be able to efficiently handle such pools. But the amount and size of pools are limited to a company or organization. In particular, it is not a goal of RSerPool to handle the global Internet in one pool set.

Extensibility

It must be possible to easily adapt the RSerPool architecture to future applications. In particular, this means to have the possibility to add new server selection procedures. That is, new applications can define special rules on which server of the pool is the most appropriate to use for the processing of a request (e.g. the least-used server). The configuration effort of RSerPool components (e.g. adding or removing servers) should be as small as possible. In the ideal case, the configuration should happen automatically, i.e. it should e.g. only be necessary to turn on a new server and it will configure automatically.

The Reliable Server Pooling Architecture

The RSerPool Architecture
Click here for full size!

The figure above shows the building blocks of the RSerPool architecture, which has been defined by the IETF RSerPool WG in RFC 5351. In the terminology of RSerPool a server is denoted as a Pool Element (PE). In its Pool, it is identified by its Pool Element Identifier (PE ID), a 32-bit number. The PE ID is randomly chosen upon a PE's registration to its pool. The set of all pools is denoted as the Handlespace. In older literature, it may be denoted as Namespace. This denomination has been dropped in order to avoid confusion with the Domain Name System (DNS). Each pool in a handlespace is identified by a unique Pool Handle (PH), which is represented by an arbitrary byte vector. Usually, this is an ASCII or Unicode representation of the pool, e.g. "DownloadPool" or "WebServerPool".

Each handlespace has a certain scope (e.g. an organization or company), which is denoted as Operation Scope. It is an explicit non-goal of RSerPool to manage the global Internet's pools within a single handlespace. Due to the limitation of operation scopes, it is possible to keep the handlespace "flat". That is, PHs do not have any hierarchy in contrast to the DNS with its top-level and sub-domains. This constraint results in a significant simplification of the handlespace management.

Within an operation scope, the handlespace is managed by redundant Registrars. In literature, this component is also denoted as ENRP Server or Name Server. Since "registrar" is the most expressive term, this denotation is used here. PRs have to be redundant in order to avoid a PR to become a single point of failure (SPoF). Each PR of an operation scope is identified by its Registrar ID (PR ID), which is a 32-bit random number. It is not necessary to ensure uniqueness of PR IDs. A PR contains a complete copy of the operation scope's handlespace. PRs of an operation scope synchronize their view of the handlespace using the Endpoint HaNdlespace Redundancy Protocol (ENRP) defined in RFC 5353. Older versions of this protocol use the term Endpoint Namespace Redundancy Protocol; this naming has been replaced to avoid confusion with DNS, but the abbreviation has been kept. Due to handlespace synchronization by ENRP, PRs of an operation scope are functionally equal. That is, if any of the PRs fails, each other PR is able to seamlessly replace it.

By using the Aggregate Server Access Protocol (ASAP), defined in RFC 5352, a PE can add itself to a pool or remove it from by requesting a registration or deregistration from an arbitrary PR of the operation scope. In case of successful registration, the PR chosen for registration becomes the PE's Home-PR (PR-H). A PR-H not only informs the other PRs of the operation scope about the registration or deregistration of its PEs, it also monitors the availability of its PEs by ASAP Keep-Alive messages. A keep-alive message by a PR-H has to be acknowledged by the PE within a certain time interval. If the PE fails to answer within a certain timeout, it is assumed to be dead and immediately removed from the handlespace. Furthermore, a PE is expected to re-register regularly. At a re-registration, it is also possible for the PE to change its list of transport addresses or its policy information (to be explained later).

To use the service of a pool, a client – called Pool User (PU) in RSerPool terminology – first has to request the resolution of the pool's PH to a list of PE identities at an arbitrary PR of the operation scope. This selection procedure is denoted as Handle Resolution. For the case that the requested pool is existing, the PR will select a list of PE identities according to the pool's Pool Member Selection Policy, also simply denoted as Pool Policy. RFC 5356 defines some standard pool policies.

Possible pool policies are e.g. a random selection (Random) or the least-loaded PE (Least Used). While in the first case it is not necessary to have any selection information (PEs are selected randomly), it is required to maintain up-to-date load information in the second case of selecting the least-loaded PE. By using an appropriate selection policy, it is e.g. possible to equally distribute the request load onto the pool's PEs.

After reception of a list of PE identities from a PR, a PU will write the PE information into its local cache. This cache is denoted as PU-side Cache. Out of its cache, the PU will select exactly one PE – again by applying the pool's selection policy – and establish a connection to it by using the application's protocol, e.g. HTTP over SCTP or TCP in case of a web server. Over this connection, the service provided by the server can be used. For the case that the establishment of the connection fails or the connection is aborted during service usage, a new PE can be selected by repeating the described selection procedure. If the information in the PU-side cache is not outdated, a PE identity may be directly selected from cache, skipping the effort of asking a PR for handle resolution. After re-establishing a connection with a new PE, the state of the application session has to be re-instantiated on the new PE. The procedure necessary for session resumption is denoted as failover procedure and is of course application-specific. For an FTP download for example, the failover procedure could mean to tell the new FTP server the file name and the last received data position. By that, the FTP server will be able to resume the download session. Since the failover procedure is highly application-dependent, it is not part of RSerPool itself, though RSerPool provides far-reaching support for the implementation of arbitrary failover schemes by its Session Layer mechanisms.

To make it possible for RSerPool components to configure automatically, PRs can announce themselves via UDP over IP multicast. These announces can be received by PEs, PUs and other PRs, allowing them to learn the list of PRs currently available in the operation scope. The advantage of using IP multicast instead of broadcast is that this mechanism will also work over routers (e.g. LANs connected via a VPN) and the announces will – for the case of e.g. a switched Ethernet – only be heard and processed by stations actually interested in this information. For the case that IP multicast is not available, it is of course possible to statically configure PR addresses.

A Migration Path for Legacy Applications

RSerPool is a completely new protocol framework. To make it possible for existing specialized or proprietary server pooling solutions to iteratively migrate to an RSerPool-based solution, it is mandatory to provide a migration path. For clients without support for RSerPool, the RSerPool concept provides the possibility of a Proxy PU (PPU). A PPU handles requests of non-RSerPool clients and provides an intermediation instance between them and the RSerPool-based server pool. From a PE's perspective, PPUs behave like regular PUs. Similar to a PPU allowing the usage of a non-RSerPool client, it is possible to use a Proxy PE (PPE) to continue using a non-RSerPool server in an RSerPool environment.

The Protocol Stack

The RSerPool Protocol Stack
Click here for full size!

The figure above shows the protocol stack of PR, PE and PU. The ENRP protocol is only used for the handlespace synchronization between PRs, all communications between PE and PR (registration, re-registration, deregistration, monitoring) and PU and PR (handle resolution, failure reporting) is based on the ASAP protocol. The failover support, based on an optional Session Layer between PU and PE, is also using ASAP. In this case, the ASAP protocol data (Control Channel) is multiplexed with the application protocol's data (Data Channel) over the same connection. By using the Session Layer functionality of ASAP, a pool can be viewed as a single, highly available server from the PU's Application Layer perspective. Failure detection and handling is mainly handled automatically in the Session Layer, transparent for the Application Layer.

The transport protocol used for RSerPool is usually SCTP, defined in RFC 4960. The important properties of SCTP requiring its usage instead of TCP are the following:

For the transport of PR announces by ASAP and ENRP via IP multicast, UDP is used as transport protocol. The usage of SCTP is mandatory for all ENRP communication between PRs and the ASAP communication between PEs and PRs. For the ASAP communication between PU and PR and the Session Layer communication between PE and PU, it is recommended to use SCTP. However, the usage of TCP together with an adaptation layer defined in [draft-ietf-rserpool-tcpmapping] is possible. This adaptation layer adds functionalities like Heartbeats, message framing and protocol identification on top of a TCP connection. But nevertheless, some important advantages of SCTP are missing – especially the high immunity against flooding attacks and the multi-homing property. The only meaningful reason to use TCP is when the PU implementation cannot be equipped with an SCTP stack, e.g. when using a proprietary embedded system providing only a TCP stack.


📽️ Demonstration

Demo Video

Watch an RSerPool/RSPLIB demonstration video here (Xvid codec required)!

A screenshot of our fractal graphics demo application: two PEs provide a fractal graphics computation service, which is requested by two PUs.

A screenshot of our fractal graphics demo application
Click here to view in full size!

Demo VM

You want to try the demo by yourself? Get a VirtualBox VM image, and run the demo:

Instructions for using the VM images can be found in the provided README file!


🖋️ Publications

The complete BibTeX references in a single file can be found here!

2024

2023

2022

2021

2019

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002


📖 Standardisation

The complete BibTeX references in a single file can be found here!

2024

2023

2019

2009

2008

2006

2005

2004

2002

If something is missing, see the Internet Draft archive at Watersprings.org. Please also contact me in case of missing or dead links.


👨‍🏫 Teaching Material

LaTeX source versions of the following teaching material are available on request. Write e-mail to Thomas Dreibholz.


💾 The Implementation RSPLIB

Binary Package Installation

Please use the issue tracker at https://github.com/dreibh/rsplib/issues to report bugs and issues!

Ubuntu Linux

For ready-to-install Ubuntu Linux packages of RSPLIB, see Launchpad PPA for Thomas Dreibholz!

sudo apt-add-repository -sy ppa:dreibh/ppa
sudo apt-get update
sudo apt-get install rsplib-all

Fedora Linux

For ready-to-install Fedora Linux packages of RSPLIB, see COPR PPA for Thomas Dreibholz!

sudo dnf copr enable -y dreibh/ppa
sudo dnf install rsplib-all

FreeBSD

For ready-to-install FreeBSD packages of RSPLIB, it is included in the ports collection, see FreeBSD ports tree index of net/rsplib/!

pkg install rsplib

Alternatively, to compile it from the ports sources:

cd /usr/ports/net/rsplib
make
make install

Sources Download

RSPLIB is released under the GNU General Public Licence (GPL).

Please use the issue tracker at https://github.com/dreibh/rsplib/issues to report bugs and issues!

Development Version

The Git repository of the RSPLIB sources can be found at https://github.com/dreibh/rsplib:

git clone https://github.com/dreibh/rsplib
cd rsplib
cmake .
make

Contributions:

Current Stable Release

The tarball has been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Its authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify rsplib-<VERSION>.tar.xz.asc rsplib-<VERSION>.tar.xz

If you do not want to use kernel SCTP, you also need our userland SCTP implementation sctplib/socketapi. The latest versions can be found at sctplib and socketapi – The User-Space SCTP Library (sctplib) and Socket API Library (socketapi). If you are unsure, just use kernel SCTP. See the handbook for details!

Old Releases

The tarballs have been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Their authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify rsplib-<VERSION>.tar.xz.asc rsplib-<VERSION>.tar.xz

💾 The RSerPool Demo Tool

Sources Download

This is the demonstration tool for RSPLIB, that is the GUI used e.g. for in the demo videos here and here. For RSPLIB itself, see the section 💾 The Implementation RSPLIB!

RSerPool Demo Tool is released under the GNU General Public Licence (GPL).

Please use the issue tracker at https://github.com/dreibh/rserpooldemo/issues to report bugs and issues!

Development Version

The Git repository of the RSerPool Demo Tool sources can be found at https://github.com/dreibh/rserpooldemo:

git clone https://github.com/dreibh/rserpooldemo
cd rserpooldemo
cmake .
make

Contributions:

Current Stable Release

The tarball has been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Its authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify rserpooldemo-<VERSION>.tar.gz.asc rserpooldemo-<VERSION>.tar.gz

Old Releases

The tarballs have been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Their authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify rserpooldemo-<VERSION>.tar.gz.asc rserpooldemo-<VERSION>.tar.gz

💾 The Simulation Model RSPSIM

Sources Download

This is the simulation model RSPSIM for OMNeT++. For the real implementation RSPLIB, see the section 💾 The Implementation RSPLIB!

RSPLIB is released under the GNU General Public Licence (GPL).

Please use the issue tracker at https://github.com/dreibh/rspsim/issues to report bugs and issues!

Development Version

The Git repository of the RSPSIM sources can be found at https://github.com/dreibh/rspsim:

git clone https://github.com/dreibh/rspsim
cd rspsim
opp_makemake -f -I.
make

Contributions:

Current Stable Release

The tarball has been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Its authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify rspsim-<VERSION>.tar.gz.asc rspsim-<VERSION>.tar.gz

Old Releases

The tarballs have been signed with my GnuPG key 21412672­518D8B2D­1862EFEF­5CD5D12A­A0877B49. Their authenticity and integrity can be verified by:

gpg --keyserver hkp://keyserver.ubuntu.com --recv-keys 21412672518D8B2D1862EFEF5CD5D12AA0877B49
gpg --verify rspsim-<VERSION>.tar.gz.asc rspsim-<VERSION>.tar.gz

📧 Our Mailing Lists


RSerPool Introductions on Wikipedia in Different Languages

What about helping Wikipedia by adding an article in your language?

Other Resources