What you`ll do:

At SAP Business AI, we are exploring the next frontier of AI on structured business data like graphs, tables and more. Foundation Models on structured data, like SAP-RPT-1, have ushered in the GenAI era for business relevant scenarios. Now, we are pushing further. We want to enable companies across all industries to collaboratively train foundation models and build knowledge graphs. 

As an architect for the AI side of an open data platform for foundation model training, working alongside our lead Platform Architect, you will design and guide implementation of the AI stack — distributed training, federated training across customer sites, model lifecycle (fine-tuning, distillation, evaluation), deployment and serving of multi-modal foundation models. Your work will complement existing AI platforms and overcome gaps when it comes to bringing together current foundation model training approaches, symbolic approaches, and enterprise data in a distributed, heterogeneous environment.  

This is the ideal role for an ambitious, highly focused individual who can drive profound technical alignments, as well as swiftly maneuver in a dynamic environment with multiple involved teams and a high degree of senior management attention. 

The Role:

• Drive technical architecture and design topics of the AI training, evaluation, and serving stack within the open platform for industries – partnering with the lead Platform Architect on cross-cutting design 

• Architect distributed and federated training infrastructure: cluster orchestration, parallelism strategies, checkpointing, fault tolerance, and secure aggregation across customer data sites 

• Design the model lifecycle: fine-tuning, post-training, evaluation and benchmarking pipelines, model registries 

• Architect inference and serving for industrial scale: multi-tenant deployment, dynamic batching, KV caching, quantization, and latency/cost trade-offs across cloud and on-prem environments 

• Ensure secure, policy-compliant handling of proprietary industrial datasets and models – authentication, lineage, provenance, licensing metadata, and data residency for federated training 

• Coordinate effectively with technical, product, and management stakeholders across multiple involved teams and ensure alignment with the overarching technical concepts and project vision 

• Support delivery teams with your deep technical guidance on AI model training, evaluation and serving 

What you bring:

• 8+ years of experience architecting AI / foundation-model ML/AI solutions 

• Hands-on experience designing distributed training systems for large models – cluster orchestration, parallelism (DDP/FSDP/TP/PP), checkpointing, fault tolerance – and ideally with federated training across organizational boundaries 

• Hands-on knowledge on foundation model fine-tuning (incl. RLHF/DPO/distillation), evaluation pipelines, and serving at scale and serving 

• Strong hands-on software engineering skills, proficiency in Python and at least one other programming language 

• Working knowledge of inference serving systems (e.g., vLLM) and serving trade-offs (batching, KV cache, quantization, latency vs. cost) is a strong plus 

• Ability to translate business and research requirements into robust, scalable technical platform designs 

• Experienced in collaborating effectively with product, consulting and customer teams, as well as other stakeholders 

• Strong communication and documentation skills for technical stakeholders (engineers, data scientists, architects) 

• Being comfortable working in a complex, evolving environment with multiple partners, technologies and dependencies, iterating the architecture as components evolve 

Meet your team:

SAP's AI organization is dedicated to seamlessly infusing AI into all enterprise applications, enabling customers, partners, and developers to enhance business processes and generate remarkable business value. Join our international AI team where innovation thrives, opportunities for personal development abound, and exceptional colleagues collaborate globally.