Job Description
Senior Developer - Spark
Location: Bangalore/ Permanent WFH
Jobdescription
Owns the end-to-end implementation of the assigned data processing components/product?features i.e.,?design, development, deployment and testing of the data processing components and associated flows conforming to best coding practices
Creation and optimization of data engineering pipelines for analytics projects.
Support data and cloud transformation initiatives
Contribute to our cloud strategy based on prior experience
Independently work with all stakeholders across the organization to deliver enhanced functionalities
Create and maintain automated ETL processes with special focus on data flow, error recovery, and exception handling and reporting
Gather and understand data requirements, work in the team to achieve high quality data ingestion and build systems that can process the data, transform the data
Be able to comprehend application of database index and transactions
Involve in the design and development of a Big Data predictive analytics SaaS-based customer data platform using object-oriented analysis, design and programming skills, and design patterns
Implement ETL workflows for data matching, data cleansing, data integration and management
Maintain existing data pipelines, and develop new data pipeline using big data technologies
Responsible for leading the effort of continuously improving reliability, scalability, and stability of micro services and platform
Skills Required
Experience in advanced Apache Spark processing framework, spark programming with Java with sound knowledge in?RDBMS.
At least 2+ years into Apache SPARK and Any Orchestration (Eg Apache Airflow)
At least 1 Years hands-on experience with Java,?Spring
Strong object-oriented design and development skills, data structures and algorithms, and design patterns
Experience in working with Core Spark, Spark Streaming,?Data frame?API, Data set API, RDD APIs & Spark SQL programming dealing with processing terabytes of data. Specifically, this experience must be in writing Big Data?engineering jobs for large-scale data integration in AWS.
Experience in writing spark?streaming?jobs integrating with streaming frameworks such as Apache Kafka or AWS Kinesis.
Experience in physical table design in a Big Data environment
Experience in working with external job schedulers such as?AWS?data pipeline, Airflow, etc.
Experience working in?Key Value?data stores such as?Arango,?HBase, etc.
Spark Profiles
a) Check if the candidate has the hands on in each of the below skills
ü Apache Spark
ü Experience in Java
ü Experience in Data Transformations using Java
ü Experience in physical DB tables design in Big Data environment
If answer to any of the above 4 skills is NO, we can ignore that profile
b) If the hands on is not in the recent past, i.e., not within previous 12 months, we can ignore such profile
c) Apache with non-Java programming language experience, like SCALA etc can be ignored i.e., don’t need to forward to Anvizent as we need Apache with Java programming experience only
Location: Bangalore/ Permanent WFH
Jobdescription
Owns the end-to-end implementation of the assigned data processing components/product?features i.e.,?design, development, deployment and testing of the data processing components and associated flows conforming to best coding practices
Creation and optimization of data engineering pipelines for analytics projects.
Support data and cloud transformation initiatives
Contribute to our cloud strategy based on prior experience
Independently work with all stakeholders across the organization to deliver enhanced functionalities
Create and maintain automated ETL processes with special focus on data flow, error recovery, and exception handling and reporting
Gather and understand data requirements, work in the team to achieve high quality data ingestion and build systems that can process the data, transform the data
Be able to comprehend application of database index and transactions
Involve in the design and development of a Big Data predictive analytics SaaS-based customer data platform using object-oriented analysis, design and programming skills, and design patterns
Implement ETL workflows for data matching, data cleansing, data integration and management
Maintain existing data pipelines, and develop new data pipeline using big data technologies
Responsible for leading the effort of continuously improving reliability, scalability, and stability of micro services and platform
Skills Required
Experience in advanced Apache Spark processing framework, spark programming with Java with sound knowledge in?RDBMS.
At least 2+ years into Apache SPARK and Any Orchestration (Eg Apache Airflow)
At least 1 Years hands-on experience with Java,?Spring
Strong object-oriented design and development skills, data structures and algorithms, and design patterns
Experience in working with Core Spark, Spark Streaming,?Data frame?API, Data set API, RDD APIs & Spark SQL programming dealing with processing terabytes of data. Specifically, this experience must be in writing Big Data?engineering jobs for large-scale data integration in AWS.
Experience in writing spark?streaming?jobs integrating with streaming frameworks such as Apache Kafka or AWS Kinesis.
Experience in physical table design in a Big Data environment
Experience in working with external job schedulers such as?AWS?data pipeline, Airflow, etc.
Experience working in?Key Value?data stores such as?Arango,?HBase, etc.
Spark Profiles
a) Check if the candidate has the hands on in each of the below skills
ü Apache Spark
ü Experience in Java
ü Experience in Data Transformations using Java
ü Experience in physical DB tables design in Big Data environment
If answer to any of the above 4 skills is NO, we can ignore that profile
b) If the hands on is not in the recent past, i.e., not within previous 12 months, we can ignore such profile
c) Apache with non-Java programming language experience, like SCALA etc can be ignored i.e., don’t need to forward to Anvizent as we need Apache with Java programming experience only
💡 Quick Summary
Seeking a career-building opportunity? The Senior Developer | Spark position is now open for candidates interested in the Software Developer Jobs sector. This role in Jodhpur offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Software Developer Jobs is a plus.