Job Description
As a Software Engineer for our Data Platform Engineering team you will join skilled Scala / Spark engineers and core database developers responsible for developing hosted cloud analytics infrastructure (Apache Spark-based), distributed SQL processing frameworks, proprietary data science platforms, and core database optimization. This team is responsible for building the automated, intelligent, and highly performant query planner and execution engines, RPC calls between data warehouse clusters shared secondary cold storage, etc. This includes building new SQL features and customer-facing functionality, developing novel query optimization techniques for industry-leading performance, and building a database system that's highly parallel, efficient and fault-tolerant. This is a vital role reporting to exec leadership and senior engineering leadership.
Skills & Qualifications
• Must have:
• Bachelors degree in computer science or related technical field.
• 4+ years experience engineering software and data platforms / enterprise-scale data warehouses
• Extensive experience in database internals, query processing, query execution
• Strong hands-on experience in Java or C++
• A track record of recruiting and leading technical teams in a demanding talent market
• Experience with rapid development cycles in a web-based environment
• Rock solid engineering fundamentals: query planning, optimizing and distributed data warehouse systems experience
• Nice to have:
• 2+ years experience with Scala and Apache Spark (or Kafka or Arrow)
• Knowledge of blockchain indexing, web3 compute paradigms, Proofs and consensus mechanisms is a strong plus but not required
• Passionate about Web3, blockchain, decentralization, and a base understanding of how data/analytics plays into this
Responsibilities:
• Writing Scala code with tools like Apache Spark + Apache Arrow + Apache Kafka to build a hosted, multi-cluster data warehouse for Web3
• Developing database optimizers, query planners, query and data routing mechanisms, cluster-to-cluster communication, and workload management techniques
• Scaling up from proof of concept to cluster scale (and eventually hundreds of clusters with hundreds of terabytes each), in terms of both infrastructure/architecture and problem structure
• Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate meta data capturing and management
• Managing a team of software engineers writing new code to build a bigger, better, faster, more optimized HTAP database (using Apache Spark, Apache Arrow, Kafka, and a wealth of other open source data tools)
• Interacting with exec team and senior engineering leadership to define, prioritize, and ensure smooth deployments with other operational components
• Highly engaged with industry trends within analytics domain from a data acquisition processing, engineering, management perspective
• Understand data and analytics use cases across Web3 / blockchains
Skills & Qualifications
• Must have:
• Bachelors degree in computer science or related technical field.
• 4+ years experience engineering software and data platforms / enterprise-scale data warehouses
• Extensive experience in database internals, query processing, query execution
• Strong hands-on experience in Java or C++
• A track record of recruiting and leading technical teams in a demanding talent market
• Experience with rapid development cycles in a web-based environment
• Rock solid engineering fundamentals: query planning, optimizing and distributed data warehouse systems experience
• Nice to have:
• 2+ years experience with Scala and Apache Spark (or Kafka or Arrow)
• Knowledge of blockchain indexing, web3 compute paradigms, Proofs and consensus mechanisms is a strong plus but not required
• Passionate about Web3, blockchain, decentralization, and a base understanding of how data/analytics plays into this
Responsibilities:
• Writing Scala code with tools like Apache Spark + Apache Arrow + Apache Kafka to build a hosted, multi-cluster data warehouse for Web3
• Developing database optimizers, query planners, query and data routing mechanisms, cluster-to-cluster communication, and workload management techniques
• Scaling up from proof of concept to cluster scale (and eventually hundreds of clusters with hundreds of terabytes each), in terms of both infrastructure/architecture and problem structure
• Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate meta data capturing and management
• Managing a team of software engineers writing new code to build a bigger, better, faster, more optimized HTAP database (using Apache Spark, Apache Arrow, Kafka, and a wealth of other open source data tools)
• Interacting with exec team and senior engineering leadership to define, prioritize, and ensure smooth deployments with other operational components
• Highly engaged with industry trends within analytics domain from a data acquisition processing, engineering, management perspective
• Understand data and analytics use cases across Web3 / blockchains
💡 Quick Summary
Seeking a career-building opportunity? The Software Engineer | Space and Time position is now open for candidates interested in the Software Developer Jobs sector. This role in Surat offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Software Developer Jobs is a plus.
