SC Cleared Data Engineer

Data Engineer – Multi-Cloud
Location: London (Hybrid)
Salary: £45,000 – £65,000, depending on experience

Description
We are seeking a skilled Data Engineer to support the design, development, and optimisation of data platforms across complex transformation programmes. This position is ideal for an engineer with hands-on experience in data pipelines, cloud platforms, and ETL processes who wants to deepen their technical expertise within a collaborative, delivery-focused environment.

You’ll play a key role in building reliable, scalable data solutions that enable analytics, reporting, and operational use cases.

Key Responsibilities

• Develop and maintain data pipelines and ETL processes using cloud-native technologies (e.g. Spark).

• Support integration across cloud and on-premise data sources.

• Help design and implement event-driven data pipelines, improving performance and scalability.

• Contribute to the development of data access layers and reusable ETL components.

• Support onboarding of new data sources, including preparation and transformation of datasets.

• Work with senior engineers to apply data engineering best practices and standards.

• Contribute to CI/CD pipelines and Infrastructure as Code (IaC) activities.

• Operate within AWS and Azure environments, supporting deployment and maintenance of data solutions.

• Work with services including:

  • AWS: EC2, EMR, S3, Lambda

  • Azure: Blob Storage and related services

• Collaborate with wider teams to ensure data quality, reliability, and performance.

• Participate in knowledge sharing and continuous improvement initiatives.

Skills & Experience (Essential)

• Proven experience in data engineering or ETL development.

• Strong knowledge of data pipelines and processing frameworks (e.g. Spark or similar).

• Hands-on experience with cloud platforms (AWS and/or Azure; exposure to both advantageous).

• Solid understanding of ETL processes and data integration concepts.

• Basic experience with Infrastructure as Code tools (e.g. Terraform) and/or CI/CD tools (e.g. GitLab).

• Familiarity with key cloud services such as:

  • AWS: EC2, S3 (EMR/Lambda exposure beneficial)

  • Azure: Blob Storage

• Understanding of version control and collaborative development practices.

• Awareness of data security, governance, and compliance considerations.

Desirable

• Experience working with event-driven architectures or streaming technologies.

• Exposure to data access layers and data platform design.

• Experience in public sector or regulated environments.

• Knowledge of cloud networking fundamentals.

• Background in fraud, analytics, or large-scale data use cases.

Security Requirements

• This role requires Security Clearance (SC).

• Candidates must be eligible and willing to undergo the clearance process.

Engineering Practices

• Growing understanding of DevOps and Agile delivery practices.

• Commitment to writing maintainable, scalable code.

• Enthusiasm for automation and reusable engineering patterns.

• Active participation in code reviews and collaborative development sessions.