← Back

Deterministic Third

Earlier this year, I had the opportunity to speak at Datadog's DASH conference using a real-world example of how my company went from simple content generation to an actual autonomous agent in our product. The main point is that AI, LLMs, and agents are tools for products, not necessarily the products themselves.

As I've been thinking about what comes next, there is another step in this flow - Deterministic Third. Once we follow the Deterministic First pattern we can let agents do their probabilistic work in our systems. But what if they find patterns? What about user road blocks or frustrations in the UX?

Deterministic First

Let's take a step back. When we think of how we're going to build, most of the time the tool we use isn't the answer to the problem we are faced with. The tool might be part of the solution, but it isn't the feature itself. This was true of microservices, REST APIs, higher-level programming languages, and compilers. LLMs fit the same bucket - just because we can introduce an LLM into a system doesn't mean that we should.

Additionally, unless you are a greenfield startup, we are all dealing with the technical debt of years of existing systems and software. These have their own way of doing things, institutional foundations, and often integral, load bearing, structural functionality. They are inherently deterministic because that is just how things were all those two to three years ago.

Cleaning up tech debt is a higher priority than it has ever been. Introducing agents into a complicated, legacy system is asking for trouble. But a Deterministic Third approach may help.

Deterministic First helps teams lean on first-principles of software engineering and deliver products that are valuable for the user and the business.

The Flow

Once you've established your architecture and patterns and your agents are running, then you monitor and observe. The same way we would monitor or observe any application. You check the logs, get alerts, are notified when something breaks or goes down. Design and marketing are tracking user experience.

Today, we manually address the issues that come up. Sometimes we let the coding agent automatically respond to bugs to generate a PR. Sometimes an agent can identify which logs are most relevant.

But what if patterns emerge? This is the goal of Deterministic Third. Identify patterns in what the agent or LLM is consistently doing, define a spec for a feature or endpoint change, then propose the code change.

This could be a new API that deterministically answers a question users keep asking. It could be a query optimization for databases that are hit more frequently than was expected. It could be a user flow where users are trying to move through an application in ways it wasn't intended for.

LLM calls are expensive in cost and latency, and their responses remain probabilistic. If we can identify consistent patterns with our users or in our systems, then we can remove the LLM and build a deterministic path.

Observability and the Loop

I'm a huge fan of Datadog. In fact, they have some tooling that surfaces these patterns. (This isn't a shill; they just have a great product.)

Similar to tech debt cleanup, observability is more important than ever in the age of agentic development and products. We need to be able to see the results of the code that an agent produced. We need to be able to watch how our agents and LLMs are responding and interacting.

As we introduce more dynamic and personalized user experiences, marketing needs to be able to see and quantify effectiveness.

Observability remains vital to our architecture and needs to be a first-class citizen of any new system.

Observability also needs intentional user experience. We understand human-in-the-loop for our agents; the same applies to an agentic experience on top of any logging or tracking platform. Human in the loop should be possible at all points of interactions with systems and codebases.

How this applies

Let's take a simple example of a user checkout flow. A user selects a product and clicks checkout, but then often clicks back to the product page. The logs show that the user is then clicking around on other products or variations of the same product.

The agent monitoring the logs identifies the pattern and infers that the "add to cart" button is either taking the user directly to checkout before they are ready or the user experience encourages the user to click a "checkout" button.

Having identified the pattern, the agent spins up an environment to screenshot the flow and notices that a modal pops up after a user adds a product to the cart which has some content, but the only button is "checkout". The agent generates a quick requirements doc, creates a ticket, then calls a dev agent to add a "continue browsing" button to the modal.

Using the design system and existing components, the dev agent adds the button, runs unit tests and QA, then submits a PR for review.

Along the way, product can be notified about the identified pattern, product and engineering can see the requirements doc and ticket, then engineering is notified of the PR. A PR review agent can review the PR and merge the change depending on its confidence.

This is a simple example, but you can see how this could be expanded for unexpected user journeys, database queries that run long in certain situations, user experience frustrations when clicking a button multiple times, missing data points that would be helpful earlier in certain system states.

The Future PDLC/SDLC

As teams get used to introducing agents into their product development, the opportunities for rapid feature identification and deployment increase, and the application can begin to correct itself. In the not-so-distant future, sales, marketing, finance, or any team can set the goals for the business and allow the agents to suggest the features that would have the most impact, while considering the existing system and its metrics.

Teams that work on simplifying and optimizing their existing systems will be able to see the downstream effects. User testing and incremental roll outs become data points for the agent to optimize for conversion goals. Engineers can focus on the architecture and configurability of the system. Product can rapidly go from idea to PRD to "nearly deployable" POCs that can be user tested immediately.

Underlying platform, design systems, and internal tooling become the priority so that teams can get their work done efficiently.

Conclusion

Most companies will require focus on tech debt and observability layers across their stack to get here. The work is worthwhile, though, as teams begin to plan, build, and deploy with agents.

Deterministic Third should be a part of architecture decisions that are made moving forward. Most of what our users need to do with our applications is deterministic. Probabilistic flows help us identify the breadth of edge cases that we wouldn't be able to handle with traditional, manual analysis.

We're at a pivotal moment in how work gets done. What's different now is how fast that shift is moving, and our systems will need to adapt along with how we build.