Dynatrace
Senior/Principal Engineer (m/f/x) for Evaluations Generative AI
Job Description
**Your role at Dynatrace**
Most AI developer tools operate without any knowledge of how software actually behaves in production. Dynatrace is in a unique position to change that.
We're looking for a Senior or Principal Generative AI Engineer to design and build the evaluation and simulation capabilities at the core of our product. You'll work across the stack, from CLI tooling that engineers run locally, to large-scale simulation pipelines, to LLM-as-a-judge evaluation frameworks running against real Dynatrace AI Observability data.
This role sits inside Dynatrace's Engineering organization and works closely with product, design, and the platform teams that power Dynatrace's AI-observability stack.
Your responsibilities:
+ Conduct research in the field of Generative AI
+ Design and build systems that let users replay and stress-test AI Agents at scale. Detect regressions across model versions, prompt changes, and data drift. Define the metrics, datasets, and judging s...
Most AI developer tools operate without any knowledge of how software actually behaves in production. Dynatrace is in a unique position to change that.
We're looking for a Senior or Principal Generative AI Engineer to design and build the evaluation and simulation capabilities at the core of our product. You'll work across the stack, from CLI tooling that engineers run locally, to large-scale simulation pipelines, to LLM-as-a-judge evaluation frameworks running against real Dynatrace AI Observability data.
This role sits inside Dynatrace's Engineering organization and works closely with product, design, and the platform teams that power Dynatrace's AI-observability stack.
Your responsibilities:
+ Conduct research in the field of Generative AI
+ Design and build systems that let users replay and stress-test AI Agents at scale. Detect regressions across model versions, prompt changes, and data drift. Define the metrics, datasets, and judging s...