Job Description
Technical Leadership
• Lead the design, development, and deployment of big data solutions using Java, Apache Spark, and modern orchestration tools.
• Architect and optimize data workflows, ensuring modularity, scalability, and maintainability across multiple domains and business units.
• Guide the adoption and integration of low-code transformation platforms (e.g., DBT) and ensure best practices in SQL-based transformation logic.
• Should be able to debug and fine tune workload , processing billions of records
Solution Delivery & Collaboration
• Oversee the development and management of APIs for data ingestion, transformation, and consumption, ensuring seamless integration with upstream and downstream systems.
• Collaborate with product managers, data scientists, analysts, and platform teams to translate business requirements into technical deliverables.
• Drive the implementation of metadata-first strategies, ensuring comprehensive cataloging, lineage, and data quality across the platform.
Quality, Observability & Governance
• Implement and enforce data engineering standards for quality, integrity, and security.
• Leverage observability suites for active and passive data validations, anomaly detection, and performance monitoring.
• Ensure robust governance by supporting RBAC/ABAC controls and compliance requirements.
Mentorship & Team Development
• Mentor and coach a team of big data engineers, fostering a culture of innovation, continuous improvement, and technical excellence.
• Conduct code reviews, knowledge sharing sessions, and technical upskilling initiatives.
Continuous Improvement
• Stay abreast of emerging trends in big data, cloud-native technologies, MLOps, and GenAI integration.
• Drive proof-of-concept (POC) initiatives and evaluate new tools and frameworks for adoption.
• Lead the design, development, and deployment of big data solutions using Java, Apache Spark, and modern orchestration tools.
• Architect and optimize data workflows, ensuring modularity, scalability, and maintainability across multiple domains and business units.
• Guide the adoption and integration of low-code transformation platforms (e.g., DBT) and ensure best practices in SQL-based transformation logic.
• Should be able to debug and fine tune workload , processing billions of records
Solution Delivery & Collaboration
• Oversee the development and management of APIs for data ingestion, transformation, and consumption, ensuring seamless integration with upstream and downstream systems.
• Collaborate with product managers, data scientists, analysts, and platform teams to translate business requirements into technical deliverables.
• Drive the implementation of metadata-first strategies, ensuring comprehensive cataloging, lineage, and data quality across the platform.
Quality, Observability & Governance
• Implement and enforce data engineering standards for quality, integrity, and security.
• Leverage observability suites for active and passive data validations, anomaly detection, and performance monitoring.
• Ensure robust governance by supporting RBAC/ABAC controls and compliance requirements.
Mentorship & Team Development
• Mentor and coach a team of big data engineers, fostering a culture of innovation, continuous improvement, and technical excellence.
• Conduct code reviews, knowledge sharing sessions, and technical upskilling initiatives.
Continuous Improvement
• Stay abreast of emerging trends in big data, cloud-native technologies, MLOps, and GenAI integration.
• Drive proof-of-concept (POC) initiatives and evaluate new tools and frameworks for adoption.
💡 Quick Summary
Seeking a career-building opportunity? The Development-Big Data position is now open for candidates interested in the IT Engineer & Developer Jobs sector. This role in Pune offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in IT Engineer & Developer Jobs is a plus.
