Supan S- Lead Data Engineer

Major Projects

AWS Glue Catalog Solution – Scalable Banking Data Architecture

Developed a cloud-native data pipeline architecture for a banking client, enabling automated ingestion, transformation, and reporting of quarterly datasets. Leveraging AWS Glue and S3, the platform delivers structured, partitioned data ready for business analytics via PowerBI. Infrastructure as Code (IaC) principles were applied using CloudFormation to ensure reliable deployments. The solution optimized reporting workflows and significantly reduced manual intervention in data preparation.

USA

Cloud & Data Engineering:

AWS Athena
AWS CloudFormation
AWS Glue
AWS Glue Catalog
AWS S3
Python

Business Intelligence:

PowerBI

Cloud & DevOps:

AWS CloudFormation
CloudFormation Templates
Scheduled Pipelines

Data Catalog & Governance Platform – Full-Stack Observability & Metadata Integration

Built a robust data observability and governance platform supporting 25+ databases and connectors, enabling comprehensive metadata extraction, quality checks, and query-driven insights. The system features in-built query support, 20+ data quality metrics, and automated ingestion pipelines. It facilitates seamless integration of on-prem and cloud data sources into a unified metadata layer, enhancing data trust and visibility across the enterprise.

Japan

Backend & Data Engineering:

Airflow
Apache Atlas
Java
MySQL
Python
REST APIs
SQL

DevOps & Infrastructure:

AWS CodeBuild
Docker
Terraform

Platform & Data Tools:

Airbyte
AWS Glue
Databricks
Lake Formation
Matillion
MSSQL
PostgreSQL
Snowflake

Snowflake Data Quality Native App – Streamlit-Powered Quality Governance Tool

Developed a native Snowflake application to streamline data quality and profiling processes using Streamlit and Snowpark. The app delivers automated data structure analysis, customizable quality rules, and integration with leading data validation libraries. Designed to embed directly within the Snowflake environment, it supports real-time quality monitoring with seamless user experience for analysts and engineers.

USA

Data Engineering & Platform:

Python
Snow-SQL
Snowflake
Snowpark
Streamlit

Data Quality & Profiling:

Great Expectations
Soda Core

Alibaba Cloud Telecom Data Solution – Scalable Analytics & Recommendation System

Architected an end-to-end data pipeline on Alibaba Cloud to process and analyze daily telecom data for business intelligence and personalized recommendations. Leveraged scalable cloud services for ingestion, transformation, and visualization. Integrated a machine learning–powered recommendation engine to enhance user engagement by suggesting optimal plans based on historical data patterns.

China

Analytics & BI:

PowerBI

AI/ML & Personalization:

Custom ML Algorithms
Historical User Behavior Modeling
Platform for AI

Google Cloud Marketing Data Analysis – Scalable Customer Intelligence Platform

Developed a marketing analytics platform on Google Cloud to unify campaign data, automate ETL workflows, and deliver actionable insights through advanced dashboards. The system supports pricing optimization and customer segmentation strategies by providing enriched, analysis-ready data to business and data science teams.

UK

Cloud & Data Engineering:

BigQuery
Google Cloud Storage
Matplotlib
NumPy
Pandas
Python

Analytics & AI/ML:

Vertex AI

AWS Cloud Data Migration & Database Redesign – Unified Data Platform Modernization

Led the redesign and migration of three legacy databases into a unified, scalable architecture on AWS Cloud. The solution improved data quality, enforced referential integrity, and enabled future-proof reporting and analytics with optimized schema design and automated pipelines.

USA