Srinivas Reddy Alluri

Summary

Accomplished Tech Lead with over a decade of experience in designing and developing data/ml products and platforms for analytical consumers, adept in both cloud and on-premises environments.
Highly skilled in leveraging and building frameworks with advanced technologies, including Apache Spark, Apache Kafka, AWS Cloud, Mlflow, LangChain, Presto, ScyllaDB, and Airflow.
Proven expertise in managing high-scale data ingestion pipelines processing terabytes daily, meeting stringent SLAs, and developing petabyte-scale, multi-tenant cloud data platforms on AWS.
Demonstrated leadership in team management, project oversight, and mentorship, ensuring successful project delivery and fostering professional growth.
Key Contributions:
- Ranked among the top 11 all-time contributors on StackOverflow for Apache Spark, Apache Hadoop, and Spark SQL.
- Made contributions to the Apache Kafka codebase, including Kafka CLI commands unification.
- Delivered impactful presentations on "Telemetry Data Management" @ Grab and "Apache Kafka Connect" @ DBS to external audiences.

Oct 2019 - Present | Lead Data Engineer

Led the design and development of a scalable Spark-based low-code ETL service that enabled thousands of pipelines to run daily, empowering non-engineering teams to manage business logic while offloading pipeline orchestration to the data platform.
Developed a robust event data quality check system at a scale of TBs per day, improving the accuracy of clickstream data across thousands of event types.
Conceptualized and implemented a multi-tenant data platform for Grab's four subsidiaries and developed a modular AuthZ solution to streamline resource operations, optimize user authorization, and ensure effective multi-tenancy management across the Experimentation and Event Tracking platform.
Successfully migrated a PBs scale cloud data lake from AWS to Azure with minimal disruption to downstream users.
Enhanced experiment sample prediction speed by replacing traditional terabyte-scale data scans with cost-effective HyperLogLog algorithms and regression models.
Improved data pipeline cost efficiency by 50% through strategic cluster scaling, performance tuning, and optimized compression techniques.

Lead and mentor a team of data and backend engineers to develop and operationalize data products and features, managing daily data volumes of approximately TB size and hundreds of billion records.
Create comprehensive documentation to propose solutions and ensure project execution, aligning with various stakeholders across departments.
Conduct regular code and document reviews, maintaining high standards of quality and fostering continuous improvement within the team.

Feb 2017 - Oct 2019 | Lead Data/ML Engineer (Digital Data Consultant)

Implemented cell-level security enforcement on datasets within the DBS Bank ADA data lake platform using BlueTalon and Protegrity as part of the Enterprise Data Security Framework (EDSF) team.
Architected and optimized data pipelines for the data lake platform, utilizing tools such as Airflow, Alluxio, Spark, YARN, Hive, Object Store, and Kafka.
Established and integrated Kafka, Kafka Connect, and Schema Registry with the metadata system and implemented Kerberos authentication with Active Directory for the Enterprise Streaming Platform team.
Developed and shared best practices, including code samples and documentation, to guide application teams in effectively utilizing platform tools.
Led a successful migration project for Neilson, transitioning from on-premise systems to the Azure cloud environment by migrating legacy infrastructure to Docker, Azure Batch Services, Azure Blob Storage, and HDInsight.

Jun 2016 - Jan 2017 | Senior Software Engineer

Written Hive Generic UDFs to implement customized functions in Spark SQL.
Involved in setting up Hadoop cluster using Apache components for Dev and QA environments.
Rewritten Oracle SQL to ANSI SQL compliant ANTLR v4 grammar utility.
Tuned job performance using YARN Application Master and Spark History Server.
Developed Spark applications for Government Pricing calculation and deployed them on EMR-YARN.
Transformed data in S3 from CSV to Parquet using Spark SQL external tables.

Feb 2014 - Jun 2016 | Senior Software Engineer

Designed and implemented MapReduce and Spark jobs to analyze end-user authentication and usage patterns.
Improved QR code authentication performance by 60% using binary optimization techniques.
Deployed an API monitoring portal using the Splunk engine and REST API
Utilized the Oozie workflow engine to manage interdependent and automated job sequences.
Developed a processing engine with Spark for the Telephone Denial of Service protection module.
Managed data ingestion from various sources, overseeing HDFS maintenance and the loading of structured and unstructured data using Sqoop and Flume.

May 2011 - Jan 2014 | Software Engineer

Designed and architected a job portal (JobCoconut), ensuring robust database design and functionality.
Integrated payment gateways for Maventus and Planlab applications, enhancing transactional capabilities.
Worked with Facebook Graph API and Twitter API to enable multimedia content posting through Maventus.
Implemented social networking authentication (Facebook, Google, Twitter, LinkedIn) for e-commerce and job portal applications.
Participated in all phases of software development, from application design through to deployment.
Programmed in Java and PHP with an emphasis on object-oriented programming, focusing on code abstraction and reuse.

Category	Details
Datastores	S3, ScyllaDB, Presto, StarRocks, DynamoDB, MySQL, PostgreSQL
Data/AI tools and frameworks	Apache Spark, Apache Kafka, Airflow, MLflow, LangChain, Superset, Hadoop
APIs	RESTful APIs, gRPC
DevOps and Infrastructure	AWS, Terraform, Kubernetes, Docker, Gitlab, Azure
Programming languages	Python, Java, Scala, Go