The client is a world leader in life sciences solutions and laboratory instruments, operating across 65 worldwide locations. The organization has a rich history of acquisitions — both large and small — and has experienced recent hypergrowth due to increasing demand during the recent global pandemic.
The Challenge
The client had limited access to data generated from 20+ ERP systems and stored across silos. With limited analytics capabilities, they found it challenging to manage this data and were increasingly unable to generate insights due to fragmentation.
They also required help to ensure data security with efficient access control. With an inadequate Change Data Capture process, they couldn’t identify and capture changes made to data which led to significant potential gaps.
The Solution
The client commissioned Persistent to build a robust data lake using various AWS components. Persistent set up a data ingestion pipeline. It was a critical first step toward creating the data lake by collecting structured and unstructured data from several sources and systems across the organization. It seamlessly transferred multiple data types from 20+ ERP systems to the data lake built on Amazon Simple Storage Service (Amazon S3).
To assist the client in processing data at scale, Persistent designed a framework deploying AWS Lamda. Implementing this serverless, event driven computing service was essential to enable the client with powerful machine learning insights. Additionally, with Amazon EMR, Persistent executed Apache Spark, an open source unified analytics engine to run and manage big data workloads.
Necessary arrangements were made to implement data security best practices using AWS-followed AES-256, an advanced encryption standard. Plus, the client needed to provide system access rights only to the intended user to further consolidate data security. For this, Persistent implemented role-based access control with AWS Identity and Access Management roles and policies.
Persistent provided the right technology support to the life sciences organization to store, process and analyze data in a central repository with a cost-effective, secure, and scalable data lake.
The Outcome
The data lake has helped the client effectively and rapidly consolidate data from over 20+ ERP systems. Their analysts now have flexible and easy access to all their data stored in a centralized location. Most importantly, this AWS-powered, cloud based data lake allows them to gain actionable insights to make data-driven decisions cost-effectively.
Technology Used
- AWS
- AWS — Data Lake
