Profile : Azure Data Engineer

Type : Full Time

Opportunity : Onsite

Experience : 1+ Year

Role and Responsibilities
• Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks
• Developing scalable and re-usable frameworks for ingesting of data sets
• Integrating the end to end data pipleline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
• Working with event based / streaming technologies to ingest and process data
• Working with other members of the project team to support delivery of additional project components (API interfaces, Search)
• Evaluating the performance and applicability of multiple tools against customer requirements

Requirements
• Demonstrated experience in end-to-end deployment of MS Azure Data solutions using Azure Databricks, ADF, Python and Synapse
• Hands-on experience in developing ETL solutions using Azure Databricks
• Cognizant is looking for Azure Data Engineer with strong Azure Databricks working experience using Spark SQL and PySpark
• Experience in writing the complex ETL code using Spark SQL, PySpark on Azure Databricks
• Proficient in implementing all transformations available in Spark and Python
• Candidate should have an in-depth understanding of Databricks Lakehouse platform
• Candidate should have a good understanding of Microsoft Azure cloud platform for building the data analytics systems
• Experience in building & debugging the Azure DataFactory Data Pipelines
• In-depth knowledge of Transact-SQL (DDL, DML)- creating indexes, Views, complex Stored Procedures, user defined functions, cursors, derived tables, common table expressions (CTEs)
• Certification(s) Preferred : Azure certifications AZ900, DP900, DP203, Databricks Data Engineer
• Microsoft Azure Platform - Mandatory
• Good understanding of MS SQL Server Dataware house ecosystem, MS SQL, SSIS (Added advantage)
• Knowledge/experience with Azure Synapse Analytics. (Added Advantage)
• Working in Agile methodology
• Knowledge of Code management (Github/BitBucket)

Personal Skills
• Systems thinking and functional decomposition skills
• Strong analytical skills to make a critical assessment of the information from numerous sources
• Modeling skills to represent requirements information in graphical form
• Setting clear goals and priorities. Time-management, ability to handle multi-tasking activities and prioritization
• High flexibility. Ability to modify approach as per changing stakeholders, conditions, circumstances and feedback. Quick learning
• Good communication, presentation, and negotiation skills
• Self-confidence. Ability to accept criticism for continuous improvement

(ref:hirist.com