Senior Data Engineer

My client is scaling a modern data platform and is looking for a Senior Data Engineer with strong Databricks experience to join on a 12‑month contract. You’ll work on high‑volume pipelines, event‑driven architecture, and cloud‑native engineering that actually stretches your skills.

The Role

You’ll design, build, and optimise Databricks‑centric pipelines processing billions of events daily. Expect hands‑on work across Python, SQL, Delta Lake, streaming, IaC, CI/CD, and cloud (Azure/GCP). This is a contract where you influence architecture, not just maintain it.

You will be working on:

• Building large‑scale Databricks pipelines (batch + streaming)

• Optimising Spark jobs for performance + cost

• Implementing Terraform/CloudFormation for data platform components

• Setting up CI/CD for data apps

• Troubleshooting production issues and driving permanent fixes

• Contributing to engineering standards across the platform

The ideal candidate will have:

• Strong Databricks + Spark experience

• Python + SQL at production level

• Cloud experience (Azure or GCP)

• Streaming + event‑driven architecture knowledge

• IaC (Terraform/CloudFormation)

• Solid debugging + DataOps mindset