Lead Data Pipeline Engineer

Lead Data Pipeline Engineer 

Would you like to use your engineering skills and your experience with machine learning (ML) and large language models (LLMs) to help policy makers, corporate leaders, and operators make sense of increasingly complex information spaces? Are you excited by the possibilities of generative and agentic artificial intelligence? Because Two Six Technologies uses these technologies to provide Information Advantage to Government, Military, and private sector customers tasked with mission critical decision making.

Our products collect, synthesize, and enrich huge quantities of publicly available data, then serve that data to be queried by our expert analysts and to be used in machine learning analytics. As a data pipeline engineer you will work with product engineering teams, data scientists, devops, and architects to integrate data science into our products.

As a top candidate for this role, you are an experienced server-side Python engineer with a history of building products or platforms with integrated ML capabilities. You are very comfortable working with AWS, Linux, and using technologies like Docker and Kubernetes. You also have worked as a member of remote engineering teams in Agile software development environments. You are a good fit if you want to continue to grow your career in the ML space by taking data science and making it available to end users in fast, scalable, and secure applications. This role is right for you if you have managed before, or if you have been a team lead or senior engineer before and are interested in managing a small team of data pipeline engineers while also being a hands-on data pipeline engineer yourself.

Most importantly you share our passion for applying technology to solve hard problems and finding innovative ways to derive insights from high-volume data.

Responsibilities:
Developing and enhancing high-volume data pipelines where data is enriched using ML and NLP functionality, and then used to feed additional machine learning and data science
Integrating models that data science creates into applications that are fast and scalable
Building and enhancing user-facing applications that use generative AI and LLMs to help our customers gain insights from data faster
Operating in a collaborative, agile environment, with a focus on taking action and working closely with peers to enable team success
Creating proofs-of-concept and prototypes to quickly test ideas, as well as designing and building scalable, production-ready solutions
Considering security best practices at every phase of software development
Continuously learning and improving
Working in a fully remote team with a diverse set of skills and experiences
Independently identifying and solving problems, and questioning assumptions in pursuit of the right solutions
Actively participating in the peer code review process
Creating and executing both manual and automated testing
Managing a small team of data pipeline engineers

Qualifications:
Experience building scalable server-side applications and data pipelines using Python
Experience working with ML, LLMs, and collaborating closely with data science teams
Experience with Elastic or similar data stores, as well as with SQL database technologies, ideally Postgres
Experience with AWS or similar cloud-based infrastructure
Experience with the design, development, testing, and support of scalable, data-driven applications deployed to production SaaS environments
Experience working independently and as a part of an Agile team

Optional Skills / Domain Experience
Prior experience as a platform engineer or ML Ops engineer
Experience with processing, storaging, and querying of data in multiple natural languages

Elastic certification
AWS certification