Senior Data Engineer - Capital Markets

Mgl | Financial services

1h 16m ago

On-site

Full Time

8+ years’ experience in data or software engineering, with strong Python skills and a record of delivering maintainable enterprise solutions.
Advanced SQL skills, including joins, common table expressions and window functions.
Experience building scalable extract, transform and load pipelines, reusable transformations and relational or dimensional models, including incremental loading, schema evolution and batch or streaming integration.
Experience with Spark or PySpark, including performance optimisation and cloud cost awareness.
Hands-on experience with Amazon Web Services, including Redshift, S3, Glue, Identity and Access Management and Kinesis, plus familiarity with Google Cloud and SQS, SNS or Kafka.
Experience with Airflow, Argo or dbt, including dependency management, scheduling, retries and backfills.
Experience implementing metadata, lineage, reconciliation and completeness checks, with secure access, data protection and auditability.
Working knowledge of Docker and Unix or Linux, with Kubernetes experience advantageous.
Experience with Git, code review, automated testing, continuous integration and deployment, controlled releases, observability, recoverable processing and production support.
Familiarity with approved artificial intelligence-assisted engineering tools such as GitHub Copilot, with independent validation and protection of confidential data; exposure to document processing, embeddings, vector search, retrieval-augmented generation and Amazon Bedrock or equivalent is advantageous.
Strong stakeholder engagement skills, including investigating data issues, challenging assumptions and communicating dependencies and risks; experience in agile teams, knowledge sharing and financial services or trading is advantageous.

Required Skills

pythonsqlsparkawsairflowdbtdockerkubernetes

Required Languages

🇬🇧 English

Senior Data Engineer - Capital Markets

Mgl | Financial services

1h 16m ago

On-site

Full Time

8+ years’ experience in data or software engineering, with strong Python skills and a record of delivering maintainable enterprise solutions.
Advanced SQL skills, including joins, common table expressions and window functions.
Experience building scalable extract, transform and load pipelines, reusable transformations and relational or dimensional models, including incremental loading, schema evolution and batch or streaming integration.
Experience with Spark or PySpark, including performance optimisation and cloud cost awareness.
Hands-on experience with Amazon Web Services, including Redshift, S3, Glue, Identity and Access Management and Kinesis, plus familiarity with Google Cloud and SQS, SNS or Kafka.
Experience with Airflow, Argo or dbt, including dependency management, scheduling, retries and backfills.
Experience implementing metadata, lineage, reconciliation and completeness checks, with secure access, data protection and auditability.
Working knowledge of Docker and Unix or Linux, with Kubernetes experience advantageous.
Experience with Git, code review, automated testing, continuous integration and deployment, controlled releases, observability, recoverable processing and production support.
Familiarity with approved artificial intelligence-assisted engineering tools such as GitHub Copilot, with independent validation and protection of confidential data; exposure to document processing, embeddings, vector search, retrieval-augmented generation and Amazon Bedrock or equivalent is advantageous.
Strong stakeholder engagement skills, including investigating data issues, challenging assumptions and communicating dependencies and risks; experience in agile teams, knowledge sharing and financial services or trading is advantageous.

Required Skills

pythonsqlsparkawsairflowdbtdockerkubernetes

Required Languages

🇬🇧 English