Role Description

Role Proficiency:
• The resource should be proficient in Azure Databricks and should be capable of leading a team. The resource should be able to design solution based on the requirement, develop and unit test the developed code to ensure quality. The resource has to play a pivotal role in a project aimed at migrating existing systems from mainframe and DataStage into the Azure Databricks platform. He/she will be responsible for designing, developing, and optimizing data pipelines and solutions that enable efficient data processing, transformation, and analysis. Key responsibilities: Lead the migration of existing systems from mainframe and DataStage into Azure Databricks. Design, develop, and maintain data pipelines using Azure Databricks to enable efficient data extraction, transformation, and loading. Implement data transformation logic to ensure data accuracy and compatibility with Azure Databricks. Optimize data pipelines for performance, scalability, and cost-effectiveness in an Azure environment. Integrate data from various sources into Azure Databricks, ensuring data quality and consistency. Implement data quality checks, validation processes, and monitoring systems to ensure data reliability. Collaborate with cross-functional teams to understand data requirements and provide effective data solutions.

Skills

Adb,Azure,Sql,Adf