You run the same prompt twice and get two different answers. Not wildly different — but different enough that your test suite goes red, your reviewer is confused, and your confidence takes a hit. You set temperature to zero, expecting the randomness to stop. It doesn’t.
This is the experience that quietly unsettles every team moving from AI demos to AI production. The models are capable. The patterns from Part 1 are in place. But the thing underneath them doesn’t behave like the deterministic software you’ve built your career on — and “almost the same every time” is a genuinely new engineering material to work with.
In Part 1, we covered what to build: ten design patterns for agentic systems, and the guardrails that keep them from spiraling. This post covers why those guardrails are necessary in the first place. Why LLMs are non-deterministic even at temperature zero. Why chain-of-thought amplifies that variability. Why small early errors compound — and why, reassuringly, they compound in a bounded, engineerable way. How to evaluate a system that won’t give you the same output twice. And finally, the mental model that ties the whole thing together: three concentric rings that tell you exactly where determinism belongs and where non-determinism earns its keep.
If Part 1 was the toolkit, this is the physics.
You run the same prompt twice and get two different answers. Not wildly different — but different enough that your test suite goes red, your reviewer is confused, and your confidence takes a hit. You set temperature to zero, expecting the randomness to stop. It doesn’t.This is the experience that quietly unsettles every team moving from AI demos to AI production. The models are capable. The patterns from Part 1 are in place. But the thing underneath them doesn’t behave like the deterministic software you’ve built your career on — and “almost the same every time” is a genuinely new engineering material to work with.In Part 1, we covered what to build: ten design patterns for agentic systems, and the guardrails that keep them from spiraling. This post covers why those guardrails are necessary in the first place. Why LLMs are non-deterministic even at temperature zero. Why chain-of-thought amplifies that variability. Why small early errors compound — and why, reassuringly, they compound in a bounded, engineerable way. How to evaluate a system that won’t give you the same output twice. And finally, the mental model that ties the whole thing together: three concentric rings that tell you exactly where determinism belongs and where non-determinism earns its keep.If Part 1 was the toolkit, this is the physics. Read More Technology Blog Posts by SAP articles
#SAPCHANNEL