JD / Skill set:

Responsible for designing, modeling and architecting data engineering solutions,

Building Enterprise Data Lake with a variety of data sources including internal and external datasets.

Big Data Lake Architect Experience for strategies around Data Ingestion, Automated Data Pipelines & Building different layers of Big Data Lake

Experience in Real-time Data Streaming and building pipelines and ETLs that transform structured and unstructured data sets in large-scale, complex datasets (Hadoop, Hive) into meaningful data models.

Proven experience with AWS technologies like S3, EMR, Cloudformation etc.

Experience with CI/CD related technologies

Experience working with real time data processing using Kafka, Spark Streaming or similar technology

Experience working with Hive, Presto or other querying frameworks.

Agile methodologies and an experience in Handling People & Processes to Deliver Business Value.

Technology:

AWS – Data Engineering tools (S3, Database Kinesis, Glue, Athena, Lambda, EMR, Lake Formation, Redshift)

Kafka, Spark, Hadoop, Hive

Python

Experience: 4 to 6 Years
• Location: Bangalore