GEL HPC - Powering genomic research

Genomics England Limited (GEL) is a world leader in genomic sequencing, and makes its vast genomics databases available for transformational genomic research. When GEL needed to move its High Performance Computing (HPC) environments to the cloud, they turned to one of the few organisations that has the required skills: The Server Labs.

Home>>

Resources

>>GEL HPC Double Helix Powering Genomic Research

Value at a glance

Enabling genomics research
Highly performant cloud HPC
Cloud environments that match researchers’ needs
Changing the face of medical research

Hand in a latex glove, holding a pipette above a petri dish

Value at a glance

Enabling genomics research
Highly performant cloud HPC
Cloud environments that match researchers’ needs
Changing the face of medical research

World leading genomics research

HPC in the cloud

Genomics England Limited (GEL) was founded by the UK Government and is a partnership with the NHS that aims to enable genomic research and embed genomics into healthcare for better patient outcomes. There are two parts to GEL’s business model – firstly processing data for the NHS and secondly making its vast genomics database available to researchers from around the world, to enable better understanding of the causes of disease and the development of new treatments.

Genomics research is very compute intensive. The human genome is made up of up to 300 million DNA base pairs distributed across 23 chromosomes. A research project could, for example, need to gather 100 genomes - 50 with a given condition and 50 without – to determine what the commonalities are.

Analysing data on this scale requires the massive compute power that only a High Performance Computing (HPC) environment can deliver.

Until 2024, genomics research was run on an HPC cluster environment run on-premise, on GEL’s own servers. But this approach had two fundamental limitations:

Unpredictable demand – because GEL serves external clients, it is very difficult to predict demand. At times of high demand, servers were working at maximum capacity and research clients had to wait for their results.
Mis-alignment between technical and financial models – many research projects are relatively short-lived: generating demand spikes of a few weeks or months. But investments in servers had to be depreciated over a much longer period, meaning that the income stream did not match the expenditure timeframe.

GEL therefore decided to move their HPC research environment to the cloud. Successfully setting up and running an HPC environment in the cloud requires highly specialist skills – skills held by a limited number of organisations.

GEL engaged one of those very few – The Server Labs (TSL) – to help ensure the success of their HPC cloud project. The HPC on-premise environment had been named ‘Helix’, so the new, cloud environment became known as ‘Double Helix’.

Cloud business model

GEL needed an HPC cloud environment that matched its business model; which was to provide a free service to academics and to charge a fixed fee to corporate researchers. So GEL and TSL came up with a plan to offer three different HPC cloud environments:

Emulator – for academic clients GEL wanted to keep their user experience as similar as possible to the on-premise. Academic institutions wanted to continue to run their existing code, so GEL set up a cloud HPC environment that emulated the on-premise one.
Private environment – for corporate clients, GEL wanted the facility to create a dedicated virtual environment. The client pays to use the environment for the duration of their research, then GEL spins it down afterwards.
Pay as you go – some private clients prefer to pay per job, rather than having a dedicated environment. For these clients, GEL decided to use the AWS cloud to provide whatever resources are needed, on demand. The client doesn’t wait for access to their own environment, they use what is available within AWS’ capacity.

Cloud business model

Emulator – for academic clients GEL wanted to keep their user experience as similar as possible to the on-premise. Academic institutions wanted to continue to run their existing code, so GEL set up a cloud HPC environment that emulated the on-premise one.
Private environment – for corporate clients, GEL wanted the facility to create a dedicated virtual environment. The client pays to use the environment for the duration of their research, then GEL spins it down afterwards.
Pay as you go – some private clients prefer to pay per job, rather than having a dedicated environment. For these clients, GEL decided to use the AWS cloud to provide whatever resources are needed, on demand. The client doesn’t wait for access to their own environment, they use what is available within AWS’ capacity.

TSL's deep HPC skills - ensuring performance

Goals achieved

TSL designed and set up the emulated environment, using Terraform Infrastructure-as-Code (IaC). GEL can create parallels and can use the same code to create the private environments.

TSL’s design work was based on extensive testing and comparisons between on-premise and cloud to ensure that they selected the most performant options for:

Compute – ensuring the appropriate instance types and size.
Scheduler – review of the LSF job scheduler.
Storage - detailed research into whether and when fast tier storage was needed – as this makes a significant difference when researchers want to pull multiple genomes at once.TSL reviewed Lustre and WACA fast tier storage.

With TSL’s help, GEL has met its goals for ‘Double Helix’:

To offer cloud HPC environments that meet the technical and financial needs of its different clients
To have cloud environments which are no more expensive to run than on-premise
To be able to continue to upgrade to take advantage of future speed and cost benefits.
Eliminate capital investment and align themselves with their clients’ move to a ‘pay as you go’ model.

‘GEL has a platform which offers HPC processing that will allow it to grow as its clients’ demands and needs grow. They have the flexibility to meet researchers’ needs now and in the future and are strongly positioned to continue to support this vital work as it continues to transform healthcare.’