Databricks Platform Engineer_D-2374
at Allianz
Posted 14 hours ago
No clicks
- Compensation
- Not specified
- City
- Not specified
- Country
- Not specified
Currency: Not specified
Databricks Platform Engineer to design, scale, and operate Allianz's enterprise Databricks platform across a regulated, multi-cloud environment. You will enable data engineers, AI/ML teams, and downstream consumers by building a secure, automated, well-governed data foundation. The role sits at the intersection of data engineering, platform engineering, and AI enablement, focusing on cloud-native platform components, governance, and cost-efficient operations. Responsibilities include deploying Databricks workspaces, CI/CD, IaC, monitoring, and cross-functional collaboration to support Agentic AI use cases.
Description
Databricks Platform Engineer
Description
Are you passionate about building enterprise‑grade data platforms that power both advanced analytics and next‑generation Agentic AI solutions?
Join us as a Databricks Platform Engineer, where you will play a key role in engineering, scaling, and operating our enterprise Databricks platform across a highly regulated, multi‑cloud environment. You will enable data engineers, AI/ML teams, and downstream platform consumers by building a secure, automated, and well‑governed data foundation.
In this role, you will work at the intersection of data engineering, platform engineering, and AI enablement.
You will design and automate cloud-native platform components, implement governance frameworks, optimize performance, and ensure compliance with strict security and data‑privacy requirements.
What you do
Platform Engineering & Operations
- Engineer, deploy, and operate Databricks workspaces, clusters, jobs, and ML runtimes across Azure, AWS, and multi-cloud architectures.
- Build and manage automated CI/CD pipelines, infrastructure-as-code (IaC), and platform lifecycle processes (provisioning, scaling, upgrades).
- Ensure platform reliability, observability, and cost efficiency through proactive monitoring, tuning, and optimization.
Data Platform & AI Enablement
- Enable scalable data ingestion, transformation, and processing pipelines using Delta Lake, Unity Catalog, and Databricks Jobs/Workflows.
- Collaborate with AI/ML teams to support feature stores, model training pipelines, and data services powering Agentic AI use cases.
- Implement and maintain enterprise-wide metadata, lineage, and data quality controls.
Governance, Security & Compliance
- Ensure strong security baselines, including Identity & Access Management (IAM), network isolation, data encryption, secret management, and secure cluster configurations.
- Implement governance guardrails aligned with regulatory, compliance, and risk frameworks (e.g., GDPR, financial regulations).
- Enforce platform standards, policies, cluster controls, and workspace configurations across Operating Entities.
Automation & Tooling
- Build reusable automation components, templates, and frameworks for platform onboarding and day‑to‑day operations.
- Support the development of monitoring dashboards, alerting systems, and platform health indicators.
Collaboration & Support
- Work with data engineers, data scientists, AI architects, cloud teams, and security teams to ensure seamless integration of Databricks with enterprise data and AI ecosystems.
- Troubleshoot platform issues and support stakeholders in building robust, scalable workloads.
- Contribute to platform documentation, best practices, and engineering playbooks.
What you bring
- Hands-on engineering experience with Databricks in enterprise environments (Azure or AWS; multi-cloud preferred).
- Strong understanding of Delta Lake, Unity Catalog, cluster architectures, Databricks Jobs/Workflows, and ML runtimes.
- Solid expertise in IaC tools such as Terraform, Pulumi, or ARM/Bicep, and CI/CD platforms such as GitHub Actions, Azure DevOps, or GitLab.
- Deep knowledge of data platform architecture, including ingestion pipelines, ETL/ELT patterns, data governance, lineage, and data quality frameworks.
- Experience operating platforms in regulated, security-sensitive industries, ensuring compliance and strong platform governance.
- Familiarity with enabling AI/ML workloads, data products, feature engineering, or MLOps pipelines.
- Strong grounding in cloud-native engineering, distributed systems, and event-driven architectures.
- Ability to work collaboratively across cross-functional engineering, cloud, security, and AI teams.
Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.
At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.
We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.
Join us. Let's care for tomorrow.
Note: Having different strengths, experiences, perspectives and approaches is an integral part of Allianz‘ company culture. One means to achieve this is a regular rotation of Allianz Executive employees across functions, Allianz entities and geographies. Therefore, the company expects from its employees a general openness and a high motivation to regularly change positions and collect experiences across Allianz Group.
