We are looking for a highly skilled data engineer to join our team. In this role, you will be responsible for designing and implementing scalable and reliable data pipelines to support the needs of data scientists, analysts, and other stakeholders.
Seeking a candidate who is available to start work within 30 days
Responsibilities:
• Responsible for Azure development/databricks/Spark notebooks
• Develop the production systems (Kafka, Cassandra, Elasticsearch)
• Implementation including loading from disparate data sets, preprocessing using Hive, Scala, spark, Hbase, Sqoop and Pig.
• Scope and deliver various Big Data solutions
• Ability to design solutions independently based on high-level architecture.
• Manage the technical communication between the survey vendor and internal systems
• Collaborate with other development and research teams
• Building a cloud-based platform that allows easy development of new applications
• Conduct code review and design review to ensure the work delivered by the team is of high-quality standards.
• Mentor other developers on coding standards, problem-solving, and application architecture.
• Develop unit and integration tests for all written software programs and components.
• Follow test-driven development approach.
• Document and communicate design and application code specifications and details to other team members.
• Mentor junior developers on the team with programming standards.
• Adapt to new programming languages, methodologies, platforms, and frameworks to support business needs.
Qualifications & Technical Skills Required:
• 3-7years of experience as a Big Data Engineer or similar role
• Hands-on experience working at a high level in Architecture or Data Science or combination.
• Azure skills highly preferred, including provisioning, Configuring, and Developing solutions in ADL, ADF, ADW etc.
• Experience with non-relational & relational databases (SQL, MySQL, NoSQL, MongoDB, etc.)
• Experience with Spark, or the Hadoop ecosystem and similar frameworks
• Familiarity with various tools such as AWS, Mesos or Docker and an instinct for automation
• Understanding with data visualization tools (e.g. PowerBI, Tableau etc.)
• Data Modeling & strong analytics skills in Analytics.
• Creative and innovative approach to problem-solving
• Excellent communication and decision-making skills are essential.
• Strong analytical, problem solving and decision-making skills
• Working knowledge of tools like SQL, Excel
We offer a competitive salary and benefits package, as well as opportunities for growth and development within the company. If you are a highly skilled data engineer with a passion for working with big data, we encourage you to apply for this exciting opportunity.
To apply, please submit your resume to
[email protected]
