Problem Statement
Client is seeking to modernize its Legacy DWH system to new age platform along with to create a unified data engine for their Digital data sources e.g., Pega, Sitecore, Salesforce etc.
Solution Overview
Envisaged an AWS cloud-based enterprise data lake and warehouse using Snowflake, Airflow and AWS tech stack.
- AWS based scalable & future ready architecture.
- Snowflake is being used as EDW.
- Data replicated from system databases using Attunity or accumulated from messages.
- Airflow for orchestration.
- Centralized and controlled management of reference data.
- DevSecOps Configuration for faster release .
Outcomes
- UI based business rule engine enabled business to make change in real time.
- Meta model service enabled reuse of ETL mapping and automatically incorporate changes.
- Recon service established business transparency and control on data quality.
- Solid data governance & security
