Data Engineer

Overview of the Function

We are seeking a talented and motivated Data Engineer with 3–5 years of experience to support the development and operation of our modern cloud-based data platform. Working within the Global Data Office, the Data Engineer will design, build and maintain scalable data pipelines and data products using Microsoft Azure, Databricks and Delta Lake, while supporting enterprise data governance and quality initiatives.

The role will work closely with the Data Engineering Lead, Data Architects, Analysts, Data Governance teams and business stakeholders to deliver trusted, reliable and accessible data solutions that underpin reporting, analytics, AI and machine learning initiatives across the firm.

This is an excellent opportunity for a hands-on engineer looking to deepen their expertise in modern data platforms, data engineering best practices and cloud-native technologies within a collaborative and innovative environment.

The role operates within a fast-paced and evolving environment and requires effective collaboration across geographically dispersed teams and multiple office locations. The successful candidate will demonstrate resilience, flexibility and a constructive approach to change, building strong relationships across the wider Data Office and business stakeholder community while fostering an inclusive and supportive team culture.

Role Summary/Purpose of Job

The Data Engineer will play a key role in maintaining the organization's modern data infrastructure. This position is part of a team responsible for:-

• Building and supporting scalable cloud-based data pipelines and data products.
• Developing data solutions using Azure, Databricks, Delta Lake, Python and SQL.
• Supporting enterprise data governance and quality initiatives.
• Helping make trusted, governed data available across the organisation.
• Collaborating with architects, analysts and business teams to solve complex data challenges.
• Contributing to the evolution of the firm's cloud-first data and AI platform.
• Learning from senior engineers while growing your technical and professional capabilities.

This role provides an excellent pathway toward Senior Data Engineer and Lead Data Engineer positions while working on enterprise-scale data, analytics and AI initiatives.

Key Responsibilities and Deliverables

Data Pipeline Development

• Design, develop and maintain batch and near real-time ETL/ELT pipelines using Azure Data Factory, Databricks, PySpark and SQL.
• Support the ingestion, transformation and delivery of data from multiple enterprise source systems.
• Develop reusable and scalable data processing components aligned with engineering standards.

Data Platform Engineering

• Build and maintain data lake and lakehouse solutions using Azure Data Lake, Databricks and Delta Lake.
• Support implementation of data models that enable analytics, reporting and self-service data consumption.
• Assist with the development of reusable data products and datasets.

Data Quality and Governance

• Implement data quality controls, validation frameworks and monitoring processes within data pipelines.
• Support metadata management, data cataloguing and lineage activities using tools such as Collibra and Purview.
• Contribute to improving data quality, consistency and trust across enterprise datasets.

Performance and Reliability

• Monitor data pipelines and processing workloads to ensure reliability and operational stability.
• Investigate and resolve data and platform issues in a timely manner.
• Identify opportunities for improving performance, scalability and cloud cost efficiency.

Collaboration and Delivery

• Work closely with Data Architects, Lead Data Engineers, Data Analysts and business stakeholders to deliver data solutions aligned to business requirements.
• Participate in agile delivery ceremonies including sprint planning, stand-ups and retrospectives.
• Support testing, deployment and release activities for data products and platforms.

CI/CD and Engineering Standards

• Contribute to source control, automated deployment and testing processes using Git, Azure DevOps and related tooling.
• Follow established engineering standards, coding practices and documentation requirements.
• Participate in peer reviews and contribute to continuous improvement initiatives.

Documentation and Knowledge Sharing

• Maintain technical documentation, solution designs and operational procedures.
• Support knowledge sharing across the Data Office through collaboration and learning initiatives.
• Contribute to the development of best practices and engineering standards.

Key Requirements

Experience & Background

• 3–5 years of experience in data engineering or a related technical discipline.
• Experience developing ETL/ELT data pipelines in a cloud environment, preferably Microsoft Azure.
• Hands-on experience with Databricks, SQL and Python/PySpark.
• Experience working with enterprise data platforms, data warehouses, data lakes or lakehouse architectures.
• Familiarity with data governance, metadata management and data quality concepts.
• Experience using Git and contributing to CI/CD delivery processes.
• Strong stakeholder engagement and collaboration skills.
• Bachelor’s degree in computer science, Engineering, Information Systems or a related field, or equivalent practical experience.

Technical Skills

Azure Data Platform

Experience with:

• Azure Data Lake Storage Gen2
• Azure Data Factory
• Azure Synapse Analytics
• Azure Functions (desirable)

Databricks and Data Engineering

• Experience developing solutions in Databricks.
• Strong SQL skills.
• Good working knowledge of Python and PySpark.
• Understanding of Delta Lake concepts and modern lakehouse architectures.

Modern Data Engineering Tools

• Familiarity with dbt (data build tool).
• Experience using Git for source control.
• Exposure to CI/CD practices using Azure DevOps or GitHub.
• Basic understanding of orchestration tools such as Apache Airflow.

Data Governance and Quality

• Awareness of data governance principles, metadata management and data lineage.
• Experience working with Collibra, Purview or similar data governance platforms is advantageous.
• Understanding of data quality frameworks and controls.

Soft Skills

• Strong analytical and problem-solving skills.
• Effective verbal and written communication skills.
• Ability to work collaboratively across technical and business teams.
• Comfortable working within Agile delivery environments.
• Curious mindset with a willingness to learn new technologies and capabilities.
• Strong attention to detail and commitment to delivering high-quality solutions.