Key Notes
Harness renewal and DAG-based orchestration are the work of this senior Wayve engineer. You’ll build platforms that process vehicle and partner data, support parallel pipeline execution and improve batch inference and incident response. Strong Python, production-system engineering and distributed-data experience are required, including Kafka, database consistency and testing practices. The full-time London role is listed as hybrid; Ray, Flyte, Spark and ML infrastructure experience are desirable.
What You'll Work On
- Modernise Harness and develop DAG-based orchestration for next-generation autonomous-driving data volumes, coordinating platform requirements with internal pipeline users.
- Build libraries and tooling for ingestion, processing, inference and evaluation, ensuring hundreds of parallel steps produce correct outputs with appropriate resource priorities.
- Improve scalable batch inference and performance evaluation, reducing processing costs and supporting reliable high-priority data runs across partner teams.
- Strengthen platform reliability, quality and incident response against business SLOs, diagnosing pipeline issues and improving operational usability for engineers.
What They Are Looking For
- Technical: Strong production software engineering and Python programming.
- Experience: Systems at scale, ideally distributed infrastructure, large data volumes or high-throughput processing.
- Technical: Unit, integration and test-driven development, Kafka or similar event streaming, and database consistency models.
- Technical: Traditional databases or warehouses such as Postgres, ClickHouse or column-oriented systems.
- Stakeholder: Translation of platform-user and partner requirements into scalable technical solutions.
- Preferred: Data orchestration, ML inference, batch scheduling, Ray, Flyte, Spark or shared developer tooling are advantageous.