Srinivas Reddy Alluri | Cloud Data/ML Solutions Lead

Summary

Accomplished Cloud Data/ML Solutions Lead with over a decade of experience in designing and developing data/ml products and platforms for analytical consumers, adept in both cloud and on-premises environments.
Highly skilled in leveraging and building frameworks with advanced technologies, including Apache Spark, Apache Kafka, Apache Flink, Apache Iceberg, AWS Cloud, Mlflow, LangChain, Presto, ScyllaDB, and Airflow.
Proven expertise in managing high-scale data ingestion pipelines processing terabytes daily, meeting stringent SLAs, and developing petabyte-scale, multi-tenant cloud data platforms on AWS.
Demonstrated leadership in team management, project oversight, and mentorship, ensuring successful project delivery and fostering professional growth.
Key Contributions:
- Ranked among the top 11 all-time contributors on StackOverflow for Apache Spark, Apache Hadoop, and Spark SQL.
- Made contributions to the Apache Kafka codebase, including Kafka CLI commands unification.
- Delivered impactful presentations on "Telemetry Data Management" @ Grab and "Apache Kafka Connect" @ DBS to external audiences.

Oct 2019 - Present | Engineering Lead

Architected and led the development of a scalable Spark-based low-code ETL platform, enabling thousands of pipelines to run daily. Empowered non-engineering teams to define business logic while offloading orchestration and infrastructure management to the data platform.
Designed and implemented a GenAI-driven system to generate intuitive explanations for clickstream event timelines, providing actionable insights into user sessions.
Conceptualized and built a multi-tenant data platform supporting four Grab subsidiaries. Developed a modular authorization (AuthZ) framework to optimize resource access, streamline user permissions, and ensure seamless multi-tenancy across the Experimentation and Event Tracking platform.
Developed a high-scale event data quality validation system processing terabytes of data daily, significantly improving clickstream data accuracy across thousands of event types.
Designed and implemented an auto-adaptive feedback loop for ML pipelines, integrating real-time model inference monitoring with ground truth pipelines to enhance fault tolerance, efficiency, and experiment lifecycle management.
Led the seamless migration of a petabyte-scale cloud data lake from AWS to Azure, minimizing disruption to downstream analytics and operational workflows.
Optimized experiment sample prediction by replacing costly terabyte-scale data scans with HyperLogLog-based approximations and regression models, reducing computational overhead while maintaining accuracy.

Lead and mentor a team of data and backend engineers, fostering professional growth and ensuring the timely delivery of scalable data products and platform features.
Oversee the development and operationalization of systems, managing daily data volumes of terabytes and processing hundreds of billions of records while maintaining performance SLAs.
Collaborate with cross-functional teams, including product managers and business stakeholders, to align engineering priorities with organizational goals.
Create and present solution proposals with comprehensive documentation, ensuring alignment across teams and smooth execution of projects.
Establish and enforce engineering best practices, conducting regular code and document reviews to maintain high-quality standards and promote continuous improvement.
Drive technical innovation and architecture decisions, leveraging emerging technologies to enhance system scalability, reliability, and performance.

Feb 2017 - Oct 2019 | Lead Data/ML Engineer (Digital Data Consultant)

Implemented cell-level security enforcement on datasets within the DBS Bank ADA data lake platform using BlueTalon and Protegrity as part of the Enterprise Data Security Framework (EDSF) team.
Architected and optimized data pipelines for the data lake platform, utilizing tools such as Airflow, Alluxio, Spark, YARN, Hive, Object Store, and Kafka.
Established and integrated Kafka, Kafka Connect, and Schema Registry with the metadata system and implemented Kerberos authentication with Active Directory for the Enterprise Streaming Platform team.
Developed and shared best practices, including code samples and documentation, to guide application teams in effectively utilizing platform tools.
Led a successful migration project for Neilson, transitioning from on-premise systems to the Azure cloud environment by migrating legacy infrastructure to Docker, Azure Batch Services, Azure Blob Storage, and HDInsight.

Jun 2016 - Jan 2017 | Senior Software Engineer

Written Hive Generic UDFs to implement customized functions in Spark SQL.
Involved in setting up Hadoop cluster using Apache components for Dev and QA environments.
Rewritten Oracle SQL to ANSI SQL compliant ANTLR v4 grammar utility.
Tuned job performance using YARN Application Master and Spark History Server.
Developed Spark applications for Government Pricing calculation and deployed them on EMR-YARN.
Transformed data in S3 from CSV to Parquet using Spark SQL external tables.

Feb 2014 - Jun 2016 | Senior Software Engineer

Designed and implemented MapReduce and Spark jobs to analyze end-user authentication and usage patterns.
Improved QR code authentication performance by 60% using binary optimization techniques.
Deployed an API monitoring portal using the Splunk engine and REST API
Utilized the Oozie workflow engine to manage interdependent and automated job sequences.
Developed a processing engine with Spark for the Telephone Denial of Service protection module.
Managed data ingestion from various sources, overseeing HDFS maintenance and the loading of structured and unstructured data using Sqoop and Flume.

May 2011 - Jan 2014 | Software Engineer

Designed and architected a job portal (JobCoconut), ensuring robust database design and functionality.
Integrated payment gateways for Maventus and Planlab applications, enhancing transactional capabilities.
Worked with Facebook Graph API and Twitter API to enable multimedia content posting through Maventus.
Implemented social networking authentication (Facebook, Google, Twitter, LinkedIn) for e-commerce and job portal applications.
Participated in all phases of software development, from application design through to deployment.
Programmed in Java and PHP with an emphasis on object-oriented programming, focusing on code abstraction and reuse.

Category	Details
Datastores	S3, ScyllaDB, Presto, StarRocks, Snowflake, DynamoDB, MySQL, PostgreSQL
Data/AI tools and frameworks	Apache Spark, Apache Kafka, Apache Flink, Apache Iceberg, Airflow, MLflow, LangChain, PyTorch, Superset, Hadoop
APIs	RESTful APIs, gRPC
DevOps and Infrastructure	AWS, Terraform, Kubernetes, Docker, Gitlab, Azure
Programming languages	Python, Java, Scala, Go