Spark/Scala Developer

Fully remote • Design • Engineering • Information Technology (IT) • Full-Time

Salary Range: $150,000 - $220,000 year

I don't need cloud architects or big data architects... rather I need core Spark/Scala developers.

I always prefer Spark engineers who write core code- not data architects or big data engineers who are experienced at using tools like Spark for data analysis or data pipelines. I need low-level core engineers who can open the Spark libraries and fork them to build out custom new libraries.

Writing Scala code with tools like Apache Spark + Apache Arrow + Apache Kafka to build a hosted, multi-cluster data warehouse for Web3

Scaling up from proof of concept to “cluster scale” (and eventually hundreds of clusters with hundreds of terabytes each), in terms of both

infrastructure/architecture and problem structure

 Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate meta data capturing and

management

 Managing a team of software engineers writing new code to build a bigger, better, faster, more optimized HTAP database (using Apache Spark, Apache

Arrow, Kafka, and a wealth of other open source data tools)

Edrstaffing

Spark/Scala Developer

Fully remote • Design • Engineering • Information Technology (IT) • Full-Time

Salary Range: $150,000 - $220,000 year

Share This Job