Rushabh Padalia
Senior Software & Cloud Data Engineer · AWS · Infrastructure as Code · CI/CD (Azure DevOps, GitLab)
I am a senior software engineer with nearly ten years of experience building and running software on AWS. My work sits where software engineering, data engineering and cloud infrastructure meet: data pipelines, event-driven architectures, infrastructure as code and the CI/CD that ships it.
Today I deliver cloud engineering work for Australian Government clients through Shine Solutions Group. I am currently helping the Australian Taxation Office (ATO) migrate a large-scale database and its monitoring dashboards to a new AWS account. Before that, I built statistical data pipelines for the Australian Bureau of Statistics (ABS).
I am at my best when I am handed something complex, undocumented or manual and turning it into something automated, repeatable and well understood. I am open to new opportunities in cloud, platform and data engineering.
Experience
Senior Software Engineer
Consultant engineer delivering AWS cloud and data engineering projects for Australian Government clients.
Australian Taxation Office (ATO) Current
- Building Azure DevOps CI/CD pipelines that move a large-scale production database to a new AWS account, including AWS Database Migration Service (DMS) pipelines, an automated backup-and-restore process and supporting utility pipelines.
- Took over a dashboard codebase that had not been maintained for years and had little documentation. I reverse-engineered how it worked so it could be safely migrated.
- Replaced manual deployments with Azure DevOps CI/CD pipelines that deploy every dashboard infrastructure stack consistently.
- Refactored the codebase to remove hard-coded values and account-specific configuration, so the same code deploys cleanly to the new account.
- Used CloudWatch Observability Access Manager (OAM) links and sinks to share monitoring data from the old account, so dashboards keep working while services migrate in stages.
Australian Bureau of Statistics (ABS)
- Built the Monthly Turnover Indicator data pipeline: ingesting data from legacy systems, transforming it with Pandas and PySpark, and loading it into Amazon RDS (PostgreSQL) for visualisation.
- Orchestrated the pipeline end to end with AWS Step Functions using an event-driven design.
- Contributed to the Consumer Price Index (CPI) modernisation program.
- Defined infrastructure with AWS CDK (TypeScript) and CloudFormation, and delivered it through GitLab CI/CD.
Lead Software Engineer
- Led the design and development of Cimara, a workforce management application, from architecture through to production.
- Owned the code repositories, ran code reviews and managed releases to AWS Lightsail.
- Mentored junior developers and interns, and planned and tracked team work in JIRA.
- Worked with management on project budgeting and on improving the team's software engineering practices.
Application Developer
- Developed back-end services, REST APIs and custom WordPress plugins, and wrote the technical documentation for them.
- Worked across planning, design, development and testing as part of a small delivery team.
- Supported clients directly on web design and development work.
Skills
DMS, RDS, Step Functions, Lambda, CloudWatch (dashboards, OAM cross-account observability), DynamoDB, S3, IAM, multi-account migration
AWS CDK, CloudFormation, Azure DevOps Pipelines, GitLab CI/CD, deployment automation, Docker, serverless architecture
Python, Pandas, PySpark, PostgreSQL, ETL and data pipelines, event-driven orchestration, database migration, backup and restore
Python, TypeScript, REST APIs, refactoring legacy code, code review, technical leadership and mentoring
Education
Australian National University
GPA: 5.8 / 7
James Cook University
Interests
Outside work I spend as much time outdoors as I can. In winter I go bushwalking around Canberra and in the national parks along Australia's east coast. In the warmer months I go camping and swimming at the beach.