Cloud Data Engineer

Digitalisation – DSM (Digital Securities Market), LSEG

About the Role

We are looking for a skilled and motivated Cloud Data Engineer to join the Digitalisation team within DSM (Digital Securities Market) at LSEG. This role is ideal for someone with hands‑on experience building and maintaining reliable, scalable cloud data pipelines, and with working familiarity of graph databases, in a regulated enterprise environment.

You will play a key role in enabling analytics, automation, and AI initiatives by designing and operating data pipelines that ingest, transform, and serve data from multiple enterprise systems. You will work closely with AI engineers, data scientists, and analytics engineers to ensure data is high‑quality, well‑governed, and fit for purpose.

Most projects focus on core cloud data engineering (e.g. Spark, Snowflake, AWS). On selected initiatives, you will also work with graph‑based data platforms (e.g. AWS Neptune) to support relationship‑driven and network‑style data models.

This is a hands‑on individual contributor role with growing ownership and progression opportunities.

Key Responsibilities

Data Pipeline Engineering

• Design, build, and maintain batch and streaming data pipelines using Python, SQL, Spark/PySpark, Glue, Kafka/Kinesis, or similar technologies.
• Ingest and transform data from internal systems, APIs, and event streams into curated, analytics‑ready datasets.
• Implement efficient data transformations and models to support downstream analytics, reporting, and AI/ML use cases.

Data Platforms & Storage

• Work with cloud‑based data platforms including Snowflake, S3, and relational databases.
• Optimise data storage and query performance through effective schema design, partitioning, and indexing.
• Support the development of feature and embedding datasets in collaboration with AI engineers.

Graph Data Engineering (Project‑Dependent)

• Build and operate data pipelines feeding graph databases (e.g. AWS Neptune), including loading, updating, and reconciling nodes, edges, and properties.
• Work with relationship‑centric data models, representing entities, dependencies, and interactions across operational datasets.
• Write and maintain basic graph queries (e.g. OpenCypher, Gremlin or SPARQL) to validate data integrity and support downstream consumption.
• Apply consistent modelling and update patterns across graph and non‑graph data stores.

Data Quality, Reliability & Governance

• Implement data validation, reconciliation, and quality checks across pipelines.
• Contribute to data lineage, metadata, and documentation, supporting auditability and regulatory requirements.
• Ensure data pipelines and graph updates are reliable, observable, and recoverable in production.

Orchestration & Operations

• Build and operate workflows using orchestration tools such as AWS Step Functions, Apache Airflow.
• Support CI/CD processes for data pipelines, including testing and deployment.
• Troubleshoot pipeline failures, data inconsistencies, and performance issues.

Collaboration & Delivery

• Work closely with AI engineers, data scientists, and business stakeholders across multiple DSM initiatives.
• Participate in Agile ceremonies, sprint planning, and technical discussions.
• Produce clear technical documentation for pipelines, datasets, and graph models.

What Success Looks Like

• You deliver reliable data pipelines supporting analytics, AI, and graph‑based systems in live DSM workflows.
• Graph and non‑graph datasets you maintain are accurate, well‑structured, and trusted.
• You can independently own pipelines and graph data updates from development through production support.
• You demonstrate confidence working with both relational and graph‑oriented data models.

Required Skills & Experience

Core Data Engineering Skills

• Hands‑on experience building data pipelines using Python and SQL.
• Experience with Spark/PySpark or similar distributed data processing frameworks.
• Experience working with cloud data platforms such as Snowflake, AWS S3, and relational databases.

Graph & Data Modelling

• Practical experience working with graph databases (e.g. AWS Neptune, Neo4j, or similar).
• Understanding of graph concepts such as nodes, edges, properties, and relationship‑centric modelling.
• Ability to work with basic graph query languages (e.g. Gremlin or SPARQL).

Cloud & Tooling

• Experience working in AWS environments, using services such as Glue, Lambda, S3, or Step Functions.
• Familiarity with workflow orchestration tools such as Apache Airflow, or similar.
• Experience with Git‑based workflows and basic CI/CD practices.

Engineering & Ways of Working

• Solid understanding of software engineering fundamentals: modular code, testing, and maintainability.
• Strong problem‑solving skills and attention to detail.
• Clear communication skills and ability to collaborate effectively in cross‑functional teams.
• Experience working in Agile delivery environments.

Desirable Skills

• Experience with AWS Neptune in production environments.
• Familiarity with data modelling tools such as Hackolade for documenting relational, semi‑structured, or graph data models.
• Experience with streaming data platforms (Kafka, Kinesis).
• Exposure to AI/ML‑adjacent data pipelines, feature stores, or embedding datasets.
• Experience working in financial services or regulated environments.
• Knowledge of data reconciliation, lineage, and audit‑friendly data design.

Why Join Us

You will join a team applying modern data engineering practices to mission‑critical Capital Markets workflows at LSEG. This role offers hands‑on exposure to production cloud data systems, including graph‑based platforms supporting initiatives like LIPA, alongside close collaboration with AI and analytics teams.

We value engineering quality, reliability, and thoughtful design, and we work on systems where data correctness and structure truly matter.

London Stock Exchange Group (LSEG) Information:

Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.

Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.

We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyone’s race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.

You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.

LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.

Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what it’s used for, and how it’s obtained, your rights and how to contact us as a data subject.

If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice.

Career Stage:

London Stock Exchange Group (LSEG) Information:

Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.

Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.

We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyone’s race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.

You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.

LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.

Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what it’s used for, and how it’s obtained, your rights and how to contact us as a data subject.

If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice.

Location: Colombo, Sri Lanka

Time Type: Full time