Scala Spark Application Developer

The Opportunity

We're looking for a talented Scala Spark Application Developer to join our early stage startup team and drive the core development of our platform's dataplane. Unlike typical Scala/Spark roles that focus on services or project-based work, this position is uniquely centered on building a sophisticated product platform. You will design and develop the dataplane of our platform - a cutting-edge Spark-based analytics engine with embedded inductive learning capabilities that forms the foundation of our data quality intelligence.

In this role, you'll architect and implement the mathematical and computational core that powers our anomaly detection algorithms. You'll work with complex statistical models, real-time data processing, and concurrent systems that operate at enterprise scale across multiple cloud environments. Our platform doesn't just process data - it learns from it, and you'll be building the brain that makes that possible.

What You'll Do

• Design and implement the core dataplane architecture for our Spark-based analytics engine

• Develop sophisticated statistical algorithms and inductive learning models for automated data quality inference

• Build high-performance, concurrent data processing pipelines using Scala and Spark that handle enterprise-scale workloads

• Implement asynchronous processing patterns and reactive systems for real-time anomaly detection and alerting

• Collaborate with our CTO & our customers to invent & embed statistical models directly into the Spark execution engine

• Optimize query performance and data processing efficiency across distributed computing clusters

• Design and implement advanced SQL generation and optimization for complex analytical workloads

• Build robust error handling and fault tolerance mechanisms for mission-critical data processing

• Establish best practices for functional programming, concurrent system design, and mathematical computation accuracy

• Mentor team members on Scala, functional programming principles, and advanced Spark optimization techniques

• Collaborate cross-functionally with product, infrastructure, and customer success teams to deliver a world-class platform

• Give & receive constructive feedback that makes the overall team and product stronger

Who You Are

• Enjoys working with excellent team members while maintaining high personal standards

• Collaborates extremely well with teammates – we want people to have fun working with you

• Has desire and ability to work both independently on complex algorithmic problems

• Comes with an optimistic attitude towards problem solving and continuous learning

• Brings experience and passion for building high-performance, mathematically rigorous systems

• Has a deep appreciation for functional programming principles and their application to distributed computing

• Takes a thoughtful approach to architectural decisions, balancing performance, maintainability, and mathematical correctness

• Has strong analytical thinking skills and enjoys perfecting algorithms that solve real-world data problems

Requirements

Experience: 3+ years of experience with Scala development and distributed systems

• Core Technologies: 2+ years of hands-on Apache Spark development experience

• Mathematical Foundation: Strong background in statistics, linear algebra, and mathematical analysis

• SQL Mastery: Advanced SQL skills with experience in query optimization and performance tuning

• Concurrent Programming: Proficiency in asynchronous and concurrent programming patterns in Scala

• Functional Programming: Deep understanding of functional programming concepts and their practical application

• Data Engineering: Experience with large-scale data processing, ETL pipelines, and data quality frameworks

• Database Systems: Familiarity with multiple datastores (PostgreSQL, Snowflake, RedShift, etc.)

• Version Control: Proficiency with Git and collaborative development workflows

Nice to Have

• ZIO Experience: Hands-on experience with ZIO for functional effects and concurrent programming

• Machine Learning: Background in ML model development and statistical learning theory

• Streaming Systems: Experience with real-time data streaming (Kafka, Kinesis, etc.)

• Cloud Platforms: Experience with cloud-native Spark deployments (EMR, Dataproc, Databricks)

• Performance Optimization: Experience with JVM tuning and Spark performance optimization

• Academic Background: Advanced degree in Computer Science, Mathematics, Statistics, or related field

• Product Development: Experience building data platform products (vs. project-based analytics)

• Startup Experience: Previous experience working in fast-paced startup environments

• Enterprise Experience: Experience working with enterprise-scale data systems and customers

Why Work Here?

Mission

We founded Qualytics based on our collective experiences working at a number of early-stage technology companies grappling with challenges related to data quality. We realized how few products are available to help organizations maintain quality & trust in their data pipelines. We're here to change that with a platform capable of detecting and proactively responding to data anomalies before they result in ML model drift, corruption of your aggregated calculations, or other pollution of your downstream systems. We believe that data is the most valuable resource to companies, and that resource isn't useful unless it's reliable.

Values

Qualytics is a values-driven organization, and we are excited about what we do. We're flexible, honest, hardworking, and collaborative. As a team, we bring our diverse backgrounds, beliefs, and experiences together to create better individual work experiences. We support and challenge one another to bring out the best in each of us, and it seems like we have a little fun along the way. We are lifelong learners, and support ongoing training and professional development on a wide variety of topics.

Perks

• Competitive salary and equity: we pay up for great talent

• Unlimited vacation – we trust you to take what you need – we clear the calendars on common holidays so you don't feel like you're going to miss an important meeting if you take off

• All the benefits: medical, dental, and vision insurance, the works

• Work with a passionate and dedicated team

• Fun environment with like minded individuals who enjoy a company giphy conversation on slack more than they should

Please note that while the position is remote, we currently require applicants to be located in the United States and cannot offer visa sponsorship at this time.