Enabling scalable, repeatable “HPC in the Cloud” end-user deployments with OpenHPC containers

UberCloud developed quick-to-roll-out and easy-to-use containers based on OpenHPC for use in High Performance Computing, whether they be on-premise, or in the public or a private cloud. Using popular Linux Container technology, UberCloud containers resolve the deployment challenges of traditional HPC environments with the goal to make HPC ubiquitous and widely available.

UberCloud uses OpenHPC to develop pre-configured, ready-to-run software containers to ease usability, accessibility, and portability challenges for development and execution of High Performance Computing applications in public and private cloud environments. UberCloud OpenHPC containers are one elegant way to deliver software packages effectively and efficiently into the cloud, and execute them in a single-tenant environment.

OPENHPC CONTAINERS – BUSINESS BENEFITSOpenHPC1

Business benefits for Independent Software Vendors (ISVs), end users, and developers

  • Agile and adaptable container-based software delivery
  • Scalable and parallel application deployment and execution
  • Operational efficiency and differentiated services

TECHNICAL REQUIREMENTS

  • CentOS 7.1 or above, shared file system access, optional low latency fabric
  • Single tenant deployment model

High Performance Computing environments are traditionally built for one key objective: performance. To use HPC for a wider set of applications, and to make it ubiquitous, other concerns such as reusability, repeatability, time to deployment, and ease of use need to be addressed.

To squeeze out the last ounce of performance, HPC environments make use of proprietary hardware architectures, specialized resource management tools and sophisticated messaging protocols, to list a few: This degree of specialization often leads to complex user interfaces, long deployment cycles, and limits the accessibility of HPC environments to organizations, which have built the necessary capabilities over many years.

UBERCLOUD OPENHPC CONTAINERS OFFER AN EFFICIENT AND SCALABLE DEPLOYMENT ALTERNATIVE

The OpenHPC stack aggregates a number of common, tested and validated HPC software building blocks required to deploy and manage High Performance Computing Linux clusters including provisioning tools, resource management, I/O clients, development tools, and a variety of scientific libraries. UberCloud has developed software containers based on OpenHPC for on-premise and Cloud HPC environments.

UberCloud’s production-ready containers bundle OS, libraries and tools as well as application codes. Containers include necessary configurations for these components to work together seamlessly. By eliminating the need to install software and configure the high performance computing environment, deployments are feasible, predictable, and repeatable in less time.

“HPC IN THE CLOUD” DEPLOYMENTS WITH OPENHPC APPLICATION CONTAINERS

UberCloud containers are quick to roll out and easy to use, based on OpenHPC, for use in High Performance Computing, whether they be on-premise, on public or on private clouds. Using popular Linux Container technology, UberCloud containers resolve deployment challenges of traditional HPC environments, while avoiding virtual machine overhead, with the goal to make HPC ubiquitous and widely available.

Main benefitors in the HPC realm, when used in single tenant run-time environments: Cloud Solution Providers (CSPs), ISVs, HPC developers, CSP-marketplaces, end-users

Key benefits of OpenHPC based containers:

  1. Portability: Build container images once, deploy them rapidly in various infrastructures
  2. Low Overhead: Run at performance levels comparable to bare-metal deployments
  3. Scalability: Deploy containers on multiple hosts to increase performance
  4. Consistency: Deploy identical containers across various infrastructure domains
  5. Rapid Deployment: Simplify and compress deployment time of HPC applications

UberCloud OpenHPC based containers can be managed by traditional cluster management tools and take full advantage of high performance computing architectures.’

CONTAINERS OFFER EASE OF USE FOR HPC

Linux containers were designed for repeatable, rapid distribution of ready to run, tested software. UberCloud OpenHPC containers are further enhanced to address deployment challenges of HPC environments. They are pre-built with just the right components, such as MPI libraries, resource management tools, compilers, and drivers for high performance computing as well as the desired software applications, e.g. Computational Fluid Dynamics applications like OpenFOAM®, or other end-user applications.

OpenHPC containers reduce HPC environment deployment times from days to hours, minimizing administrative costs, and increasing repeatability.

OpenHPCimageOPENHPC IMAGE IS A BUILDING BLOCK

An attractive property of container images is their ability to be stacked together similar to Lego* blocks. The OpenHPC Container image offers a powerful way for the OpenHPC community to develop a rich set of HPC application container images, all built on a common set of HPC tools and the operating system. End users then pull a specific image with their desired application already based on OpenHPC.

OPENHPC CONTAINERS IN THE PUBLIC CLOUD

While using OpenHPC containers in the Cloud, organizations can create an on-demand HPC Cluster user experience with bare metal or virtualized Cloud infrastructure.

OpenHPC containers are an excellent fit into pay-per-use and single-tenant Cloud environments, because they speed up deployment cycles, reducing cloud expenditures. Since the container images are updated by UberCloud with current OpenHPC components, the administration effort is greatly reduced.

TYPICAL SOLUTION ARCHITECTURE FOR ON-PREMISE AND CLOUD-BASED DEPLOYMENTS

Container architectureUberCloud’s OpenHPC-based containers are designed to be used in on-premise, private and public cloud environments, following a one container per host deployment model. In a typical on-premise deployment, UberCloud OpenHPC containers can be used together with existing HPC infrastructure, for example a resource manager (Slurm* in the sample diagram above).

In a typical on-premise deployment, there are often two network fabrics available. Servers connect to a TCP/IP Network Switch through Network Interface Controller (NIC) cards. This network is

OpenHPC container has minimal performance impact when compared to bare metal performance used for file transfers and control traffic. Similarly, servers and processors can connect to and utilize low latency interconnects (like Infiniband). This high performance network is for  Message Passing Interface (MPI) communication by the application’s processes.

In this example, Slurm controls the compute resources of the cluster. Users queue their requests on to Slurm and Slurm assigns resources to the jobs, launches, monitors, and controls the related operating system processes. The Slurm Daemon (Slurmd) instantiates the container run-time (Dockerd) on each of the compute nodes allocated. Then, again Slurm, pulls the requested UberCloud OpenHPC Container Image from the UberCloud Private Container Registry and starts one UberCloud OpenHPC Container per host using this image. The containers are configured to automatically connect with each other and form a mini cluster, which supports MPI traffic using OpenHPC supplied drivers and libraries.

Use OpenHPC based Containers without compromising run-time performanceOpenHPC OF Performance 2

Containers, unlike virtual machines, don’t require a hypervisor. Therefore containers eliminate bottlenecks in computing and I/O to achieve bare metal like performance, making them an ideal technology for running HPC applications. Further, each application shares components of the host operating system, making container images easier to store and transfer.

Performance tests conducted on the Hyperion Cluster at Lawrence Livermore National Laboratory (LLNL) demonstrated that a medical device simulation with OpenFOAM application code running on the OpenHPC Container achieved near bare metal solution times.

Get the OpenHPC Container

The OpenHPC Container from UberCloud is a great fit to simplify your deployments for:

  • Complex HPC applications
  • Agile development and testing teams
  • OpenHPC evaluations
  • Fast changing, public or private clouds
  • Applications which need portability from workstation, to cluster, to cloud, and
  • Continuous integration environments for HPC software development teams.

To get the OpenHPC container, please contact UberCloud at www.theubercloud.com/help/.

Learn More