Job Description
We are seeking a skilled and experienced Real-Time Data Engineer to join our dynamic team. As a Real-Time Data Engineer, you will be responsible for developing and maintaining data processing applications using Kafka (similar streaming systems like RabbitMQ, and Pulsar) and Java (or similar programming languages like Scala, and Python). You will also be working with real-time, batch, and event-driven data processing concepts, as well as frameworks such as Apache Beam, and Apache Flink (or similar frameworks like Spark/Structured Streaming, and Apache Storm).
Responsibilities
• Design, develop, and deploy real-time data processing applications using Kafka (or similar streaming systems), Java (or other programming languages), and relevant frameworks.
• Collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient data processing solutions.
• Implement data ingestion, transformation, and streaming processes to handle high-volume and high-velocity data streams.
• Optimize data pipelines for performance, reliability, and scalability to meet business needs.
• Monitor and troubleshoot data processing workflows to ensure smooth and uninterrupted data flow.
• Collaborate with data scientists and analysts to integrate real-time data pipelines into analytical and reporting systems.
• Stay up-to-date with emerging technologies, trends, and best practices in real-time data processing and streaming architectures.
• Document design specifications, procedures, and guidelines for data processing applications.
• Conduct performance testing and optimization of data processing applications to ensure optimal system performance.
• Assist in mentoring and providing technical guidance to junior team members as needed.
This job is provided by Shine.com
Responsibilities
• Design, develop, and deploy real-time data processing applications using Kafka (or similar streaming systems), Java (or other programming languages), and relevant frameworks.
• Collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient data processing solutions.
• Implement data ingestion, transformation, and streaming processes to handle high-volume and high-velocity data streams.
• Optimize data pipelines for performance, reliability, and scalability to meet business needs.
• Monitor and troubleshoot data processing workflows to ensure smooth and uninterrupted data flow.
• Collaborate with data scientists and analysts to integrate real-time data pipelines into analytical and reporting systems.
• Stay up-to-date with emerging technologies, trends, and best practices in real-time data processing and streaming architectures.
• Document design specifications, procedures, and guidelines for data processing applications.
• Conduct performance testing and optimization of data processing applications to ensure optimal system performance.
• Assist in mentoring and providing technical guidance to junior team members as needed.
This job is provided by Shine.com
💡 Quick Summary
Seeking a career-building opportunity? The Data Engineer position is now open for candidates interested in the MIS Executive sector. This role in Mumbai offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in MIS Executive is a plus.