Job Description
Job Description
Reports To
Reports to the Feature Lead/Project Manager on the program or the lead solution designer or architect. Will take direction from the CG or client project manager for work and technical design from the solution designer or architect.
Job Overview
The role of Data Engineer involves leading complex business reengineering development projects and to solve organizational problems through IT solutions using Big Data concepts, Platform tools and software engineering languages as required.
Key Responsibilities Include
• Designing, developing and implementing solutions to customers organisational problems;
• Providing high quality service delivery on engagements;
• Designing and developing data reconciliation & metadata driven data processing frameworks;
Primary Skills
Key Skills/Experience:
• Prior experience as a data engineer
• Hands-on experience in Cloud Computing – AWS preferably
• Experience in programming and shell scripting eg. Core/Python/Scala/Java
• Experience in Hadoop Stack eg. Hive, Spark (Core and Streaming), Kafka, Flink, NIFI;
• Experience with Github and Jenkins;
• Experience working on stream transform using Spark Streaming or Flink;
• Experience working on change data capture in a big data ecosystem;
• Experience in building real time or batch ingestion and transformation pipelines;
• Experience in data orchestration tools (Airflow is preferred)
Secondary Skills
Additional Skills/responsibilities Of This Role
• Experience supporting a production service in a DevOps friendly environment;
• Experience working in Agile/Scrum teams;
• Hands on experience in API and cloud-native architecture;
• Experience with ‘container’ technology (e.g. Docker, Kubernetes);
• Experience wrangling data using library such as Pandas, Scikit-learn or Numpy;
• Experience using notebook such as Jupyter or Polyglot;
• Experience with test-driven development, test automation & continuous delivery;
• Experience in Java build automation technologies such as Gradle, Maven, Integration with Sonarqube etc
Reports To
Reports to the Feature Lead/Project Manager on the program or the lead solution designer or architect. Will take direction from the CG or client project manager for work and technical design from the solution designer or architect.
Job Overview
The role of Data Engineer involves leading complex business reengineering development projects and to solve organizational problems through IT solutions using Big Data concepts, Platform tools and software engineering languages as required.
Key Responsibilities Include
• Designing, developing and implementing solutions to customers organisational problems;
• Providing high quality service delivery on engagements;
• Designing and developing data reconciliation & metadata driven data processing frameworks;
Primary Skills
Key Skills/Experience:
• Prior experience as a data engineer
• Hands-on experience in Cloud Computing – AWS preferably
• Experience in programming and shell scripting eg. Core/Python/Scala/Java
• Experience in Hadoop Stack eg. Hive, Spark (Core and Streaming), Kafka, Flink, NIFI;
• Experience with Github and Jenkins;
• Experience working on stream transform using Spark Streaming or Flink;
• Experience working on change data capture in a big data ecosystem;
• Experience in building real time or batch ingestion and transformation pipelines;
• Experience in data orchestration tools (Airflow is preferred)
Secondary Skills
Additional Skills/responsibilities Of This Role
• Experience supporting a production service in a DevOps friendly environment;
• Experience working in Agile/Scrum teams;
• Hands on experience in API and cloud-native architecture;
• Experience with ‘container’ technology (e.g. Docker, Kubernetes);
• Experience wrangling data using library such as Pandas, Scikit-learn or Numpy;
• Experience using notebook such as Jupyter or Polyglot;
• Experience with test-driven development, test automation & continuous delivery;
• Experience in Java build automation technologies such as Gradle, Maven, Integration with Sonarqube etc
💡 Quick Summary
Seeking a career-building opportunity? The Senior AWS Software Engineer | 4 to 12 years | Bangalore Chennai Pune position is now open for candidates interested in the Software Developer Jobs sector. This role in Bangalore offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Software Developer Jobs is a plus.