Data Engineer
at Capgemini
Posted 6 days ago
No clicks
- Compensation
- Not specified
- City
- Not specified
- Country
- Malaysia
Currency: Not specified
Design and develop robust ELT pipelines using Azure Data Factory and optimize data workflows in Azure Databricks with PySpark. Write efficient Python scripts for data manipulation and automation. Collaborate with data scientists, analysts and business stakeholders to gather data requirements and ensure data quality, integrity and security across all data processes. Monitor pipelines, troubleshoot issues and implement best practices for data engineering and cloud architecture.
Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.
Job Description
Design and develop robust ELT pipelines using Azure Data Factory
Build and optimize data workflows and transformations in Azure Databricks using PySpark
Write efficient Python scripts for data manipulation automation and integration tasks
Collaborate with data scientists analysts and business stakeholders for data requirements
Ensure data quality integrity and security across all data processes
Monitor and troubleshoot data pipelines and workflows
Implement best practices for data engineering and cloud architecture
Job Description - Grade Specific
7 years of experience in data engineering
Strong proficiency in Azure Data Factory and Azure Databricks
Solid experience with Python scripting for data processing and automation
Hands on experience with PySpark SQL and Delta Lake
Familiarity with Azure Synapse Azure Blob Storage and Azure Key Vault is a plus
Excellent problem solving and communication skills
Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organizations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of €22.1 billion.
Make it real | www.capgemini.com
