Spark Scala Developer

Spark & Scala Developer

Shift- Malaysian / EUR

Exp- 5+ yrs (2 Positions), +- 11 yrs (1 Position)

Full time Contract Opportunity

Fully Remote

please share resume at [email protected]

Responsibilities-

Create Scala/Spark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

Skills-

Scala (with a focus on the functional programming paradigm)

Scalatest, JUnit, Mockito, Embedded Cassandra

Apache Spark 2.x

Apache Spark RDD API

Apache Spark SQL Data Frame API

Apache Spark MLlib API

Apache Spark GraphX API

Apache Spark Streaming API

Spark query tuning and performance optimization

Data Modelling, Data Warehousing, Hadoop, Hive, ETL

SQL database integration ,Microsoft, Oracle, Postgres, and/or MySQL

Experience working with HDFS, S3, Cassandra, and/or DynamoDB

Deep understanding of distributed systems (e.g. CAP theorem partitioning, replication, consistency, and consensus
Report this listing
Typical Pay for this Type of Work

Glassdoor
₹5.8L–6.3Lper year
Scala Developer
Based on local employers