Value at a glance
- Ensuring the best HPC environment for bioinformatics pipeline
- Detailed evaluation of HPC options
- Expert recommendation
- Enabling informed decision making
- Recommending a platform for performance and scale
HPC cloud optimisation
How The Server Labs helped Genomics England to optimise the cloud environments for their bioinformatics workloads
When Genomics England were planning a new bioinformatics pipeline solution, they wanted to ensure it ran in an optimal HPC environment. The Server Labs carried out a detailed analysis of every HPC option, so that Genomics England could be 100% sure they had the most efficient and cost effective platform.
Value at a glance
- Ensuring the best HPC environment for bioinformatics pipeline
- Detailed evaluation of HPC options
- Expert recommendation
- Enabling informed decision making
- Recommending a platform for performance and scale
Advanced Genomic Healthcare
A new bioinformatics pipeline solution
Genomics England Limited (GEL) – owned by the UK Department of Health and Social Services - is the world’s largest community in genomic healthcare and medical research. Using its genomic datasets, GEL powers lifesciences research that results in life-transforming medicines, treatments and diagnostics.
The bioinformatics pipeline is crucial to GEL’s continued ground-breaking work. GEL had been using the ‘Bertha’ suite of tools to run bioinformatics code in production, but had selected a new ‘Genie’ system, designed to:
- Ingest genomic and clinical data
- Orchestrate bioinformatics workflows to automate execution of patients' genome analysis
- Host bioinformatics analysis code
- Test new/modified workflows
- Manage and monitor workflow runs
The Genie solution needed an optimal High Performance Computing (HPC) environment to run effectively and efficiently, and GEL wanted to evaluate the options before finalising their HPC set-up.
The HPC demands of the new solution
GEL wanted eliminate the issues they’d preivously had, by creating an HPC environment that could:
- Run workflows at scale (bursting to AWS)
- Use compute infrastructure on the cloud in addition to on-prem
- Optimise workflows for cost by picking the most cost-efficient compute set-up
- Support Genie’s selection of a standard workflow orchestration tool
- Run workflows using different types of accelerated computing
- Port workflows to different infrastructures
Additionally, they wanted to identify a Disaster Recovery setup for the existing pipelines whilst the new one is being set up.
The HPC demands of the new solution
GEL wanted eliminate the issues they’d preivously had, by creating an HPC environment that could:
- Run workflows at scale (bursting to AWS)
- Use compute infrastructure on the cloud in addition to on-prem
- Optimise workflows for cost by picking the most cost-efficient compute set-up
- Support Genie’s selection of a standard workflow orchestration tool
- Run workflows using different types of accelerated computing
- Port workflows to different infrastructures
Additionally, they wanted to identify a Disaster Recovery setup for the existing pipelines whilst the new one is being set up.
Calling on the experts
The Server Labs' approach
GEL had worked with The Server Labs (TSL) in the past and knew that their cloud expertise would be invaluable in helping them select the best environment. They asked TSL to evaluate the different options so that they could make a highly informed decision.
TSL tested the workloads on different compute environments, documented the results and provided GEL with observations and recommendations. TSL set up and ran four different architecture patterns for hybrid computing for the Genie workload:
• Pattern 1 - Synced storage
• Pattern 2 - Shared storage
• Pattern 3 - Single executor
• Pattern 4 - Pattern 2 with single orchestrator
TSL trialled the following technologies:
Compute Orchestration
1. AWS Batch
- C5 cpu nodes + F1.2xlarge fpga nodes
- M5 cpu nodes + F1.2xlarge fpga nodes
- R5 cpu nodes + F1.2xlarge fpga nodes
- 'Optimal' cpu nodes + F1 fpga nodes
- Best performing node + F1.4xlarge fpga nodes
2. IBM LSF
- Job queue with all nodes on-prem
- Job queue with all nodes on AWS
- Job queues with nodes on both AWS and on-prem
Fast-tier / Persistent Storage
1. AWS FSx for Lustre
- Mounted on nodes on AWS
- Mounted on nodes on-prem
2. Weka on-prem
- Mounted on nodes on-prem
- Mounted on nodes on AWS
3. Weka on AWS
- Mounted on nodes on-prem
- Mounted on nodes on AWS
4. S3
- Staged and run on on-prem
- Staged and run on AWS
8 key recommendations
TSL provided detailed, documented test outcomes: • CPU and elapsed timings for different scenarios • Costings for the different technology configurations TSL provided eight key recommendations, as well as areas for further exploration.
8
Key recommendations
Timings
CPU and elapsed timings for different scnarios
Costings
Costings for the different technology configurations
Business and technical goals achieved
Based on TSL’s feedback and recommendations, GEL now has a clear understanding of how to provide the best compute environment for different Genie workloads, and the associated costs of each.
Business and technical goals achieved
Based on TSL’s feedback and recommendations, GEL now has a clear understanding of how to provide the best compute environment for different Genie workloads, and the associated costs of each.
Selecting the right environment is essential to the performance of the bioinformatics pipeline workload. Testing every option means GEL now know they have the optimal HPC setup.
Paul Parsons
Chief Technnology Officer at The Server Labs
Download the Case Study
You can download the case study in PDF format from here
Download the Case Study
You can download the case study in PDF format from here
