Hardik Goel

After leading a 14-engineer org at Paytm, chose a deep individual-contributor architect track — designing and owning AI platforms hands-on. End-to-end architecture ownership of an AI commodity-forecasting platform supporting a £3.5B procurement domain.

• Architected and delivered a production-grade AI commodity forecasting platform supporting £3.5B in annual procurement across multiple commodities and prediction horizons.
• Designed scalable MLOps workflows on Azure ML (MLflow, SHAP, PSI drift monitoring); established governed deployment pipelines with DQ gates, model metadata registries, security scanning, audit logs, Microsoft Purview lineage, and Maker-Checker controls.
• Defined and enforced engineering governance standards across Data Engineering and ML — modular architecture, typed interfaces, reusable pipeline templates, and automated pytest coverage gates — improving deployment consistency and platform reliability.
• Built reusable metadata-driven data and ML pipelines integrating LSEG and enterprise RDBMS sources, reducing commodity onboarding effort and improving scalability across new forecasting use cases.
• Delivered stakeholder analytics dashboards and data enrichment, improving commercial visibility into commodity trends and model outputs.

• Pioneered AI-augmented engineering workflows using Claude and GPT agents for automated code review, test generation, documentation, and pipeline scaffolding — accelerating delivery velocity.
• Designed a persona-based agentic framework with specialised AI roles for Data Engineering, ML Review, Security, and MLOps Governance — one of the earliest enterprise implementations of role-based AI agents in a production data platform.
• Authored AI-enforceable markdown standards enabling automated enforcement of coding practices, security guardrails, and quality gates through structured context orchestration and RAG workflows.

Led a 14-member engineering team owning data-platform strategy, architecture, and delivery across incentive management, merchant analytics, and GenAI product initiatives.

• Architected a high-throughput incentive processing platform: reduced payment SLA from 15 days → 3 days, improved accuracy to >97%, and cut attrition by 40%.
• Designed rule-engine pipelines using Airflow, AWS EMR, PySpark, and MongoDB; leveraged DBT for SQL transformations and ran Databricks DBR impact assessment.
• Drove >50% cost reduction on Incentives; architected PGOLAP on Databricks DBR, cutting costs by >70% for 110M daily events at Dream11 scale (1.8B events/day).
• Built FSEBuddy — a self-service platform enabling Product-team autonomy and reducing engineering dependency for routine data tasks.
• Delivered CustomerEchoEase — a GenAI-powered customer-insights solution — as an internal innovation initiative.

• Architected a real-time merchant analytics platform: drove >40% offline merchant sales growth and >30% cross-sell-rate uplift.
• Engineered high-throughput event processing with Kafka, NiFi, Spark, Apache Hudi, and ElasticSearch to power sub-second API queries for merchant dashboards.

• ICON (Ingest, Catalog, Optimize, Notify) — architected a data-lake governance framework for a US airline, reducing ingestion TAT by >40% and error rates by >90% using AWS Lambda, Glue, Athena, RDS, PySpark.
• Delivered ~99% latency reduction (2s → 20ms) on a sensitive-data search engine using Apache Solr + ELK; storage tiering with Cassandra for cold parameters and Kafka as data backbone.
• Built real-time sensor anomaly-detection dashboards (Kafka, Flink, QuickSight, Redshift Spectrum, DBT, MWAA, Apache Beam) for a semiconductor client; built an internal NLP sentiment analyser (NLTK) outperforming VADER benchmarks.

• Anti-Money-Laundering (European bank): reduced AML false positives from 91%, enabling faster regulatory reporting and cutting the monthly investigation backlog against a 30-day SLA.
• Implemented an ML pipeline using LightGBM (Paylay model) with ingestion from Oracle, Teradata, and sandpits via Sqoop, Oozie, Hive/Impala, and Spark pre-processing.

• Built CDR/UDR ingestion pipelines for a US telecom client (postpaid) using Kafka, Spark, and Hive.
• Delivered MVNO mobile automation frameworks (Selenium WebDriver, Appium, TestNG, Jenkins) and led cross-functional sprint, system, and UAT testing across US & UK telecom clients.