Job Description
Job title: ETL Data Engineer - AWS & Spark
Location: Sydney
Role: Contract
Position Overview: We are seeking highly skilled ETL Data Engineer with experience in ETL development and a solid understanding of the AWS ecosystem, particularly EMR, S3, Athena, and SQL. The ideal candidate will have expertise in Spark and Scala, ETL lifecycle management, and job scheduling using Autosys. Experience with framework builds and ETL job migration to AWS environments is highly desirable.
Key Responsibilities:
Execute ETL jobs and processes from Day 1 to ensure smooth data operations.
Reconcile ETL jobs for a minimum of three runs, ensuring data accuracy and consistency.
Triage, debug, and fix issues with ETL jobs while reconciling data discrepancies.
Collaborate with multiple capability teams to debug and resolve job-related challenges effectively.
Conduct knowledge transfer (KT) sessions to ensure offshore teams are equipped to manage and support ETL processes.
Preferred Qualifications and Skills:
5-12 years of strong ETL development experience using Spark and Scala.
Hands-on experience with AWS services, including EMR, S3, Athena, and SQL.
Proficiency in Autosys for job scheduling.
Familiarity with building reusable ETL frameworks.
Prior experience in migrating ETL jobs to AWS environments.
Comprehensive understanding of the ETL lifecycle, including running, debugging, testing, reconciling, and deploying jobs in an AWS environment.
Strong problem-solving skills and ability to work effectively with cross-functional teams.
Proactively run and execute jobs to ensure operational continuity.
Perform reconciliation of jobs for at least three runs to validate data integrity.
Debug, troubleshoot, and resolve issues related to ETL job failures.
Engage and collaborate with multiple capability teams to address challenges and implement fixes.
Provide thorough onsite knowledge transfer to offshore teams for smooth transition and support.
Desired Candidate Profile:
Proactive and self-driven with a strong technical foundation in AWS and ETL processes.
Excellent communication and collaboration skills to work with diverse teams.
Proven ability to work under pressure and deliver high-quality solutions in a dynamic environment
Location: Sydney
Role: Contract
Position Overview: We are seeking highly skilled ETL Data Engineer with experience in ETL development and a solid understanding of the AWS ecosystem, particularly EMR, S3, Athena, and SQL. The ideal candidate will have expertise in Spark and Scala, ETL lifecycle management, and job scheduling using Autosys. Experience with framework builds and ETL job migration to AWS environments is highly desirable.
Key Responsibilities:
Execute ETL jobs and processes from Day 1 to ensure smooth data operations.
Reconcile ETL jobs for a minimum of three runs, ensuring data accuracy and consistency.
Triage, debug, and fix issues with ETL jobs while reconciling data discrepancies.
Collaborate with multiple capability teams to debug and resolve job-related challenges effectively.
Conduct knowledge transfer (KT) sessions to ensure offshore teams are equipped to manage and support ETL processes.
Preferred Qualifications and Skills:
5-12 years of strong ETL development experience using Spark and Scala.
Hands-on experience with AWS services, including EMR, S3, Athena, and SQL.
Proficiency in Autosys for job scheduling.
Familiarity with building reusable ETL frameworks.
Prior experience in migrating ETL jobs to AWS environments.
Comprehensive understanding of the ETL lifecycle, including running, debugging, testing, reconciling, and deploying jobs in an AWS environment.
Strong problem-solving skills and ability to work effectively with cross-functional teams.
Proactively run and execute jobs to ensure operational continuity.
Perform reconciliation of jobs for at least three runs to validate data integrity.
Debug, troubleshoot, and resolve issues related to ETL job failures.
Engage and collaborate with multiple capability teams to address challenges and implement fixes.
Provide thorough onsite knowledge transfer to offshore teams for smooth transition and support.
Desired Candidate Profile:
Proactive and self-driven with a strong technical foundation in AWS and ETL processes.
Excellent communication and collaboration skills to work with diverse teams.
Proven ability to work under pressure and deliver high-quality solutions in a dynamic environment
💡 Quick Summary
Seeking a career-building opportunity? The ETL Data Engineer - AWS & Spark position is now open for candidates interested in the IT Engineer & Developer Jobs sector. This role in Sydney offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in IT Engineer & Developer Jobs is a plus.