DevOps Engineer (Azure Data Services)

About the Role

We are looking for a skilled and innovative DevOps Engineer to join our clients growing team. You'll play a pivotal role in modernising our data operations by designing and implementing Continuous Integration and Continuous Deployment (CI/CD) pipelines for Azure Databricks and Azure Data Factory (ADF). This is a fantastic opportunity to shape the future of our Azure environment, driving automation, efficiency, and reliability across our data services. This role is remote, with very occassional abroad travel (paid for) and falls inside IR35. It will be an inital 6 month contract with a likelihood of extension. We have interview slots available.

Key Responsibilities

Design, build, and maintain CI/CD pipelines for Azure Databricks and ADF
Collaborate with data engineers and developers to integrate code changes seamlessly
Automate deployment processes to reduce manual intervention and boost efficiency
Monitor and troubleshoot CI/CD pipelines, ensuring minimal downtime
Implement best practices for asset bundling and version control in Azure DevOps
Optimize Azure DevOps environments for scalability and performance
Ensure compliance with security standards across all deployment processes
Document CI/CD processes, procedures, and configurations

What We're Looking For

Proven experience with Azure DevOps and CI/CD pipeline implementation
Proficiency in Databricks Asset Bundles (DABs)
Strong knowledge of Azure Databricks and Azure Data Factory
Hands-on experience with scripting/automation tools (PowerShell, Python)
Experience in selective deployment for ADF pipelines
Familiarity with containerization/orchestration (Docker, Kubernetes) is a plus
Excellent problem-solving and troubleshooting skills
Strong communication and collaboration abilities
Knowledge of CI/CD security best practices
Detail-oriented mindset with a focus on high-quality automation solutions
Bachelor's degree in Computer Science, IT, or related field
Minimum 5 years' experience in DevOps or related roles