Deterministic vs Non-Deterministic AI Workflows for Developers

I mean, Koston, you and I have been on this AI journey for a couple years now, and I feel like a lot of people around me have just kind of recently adopted these tools, and they're going through an experience that I went through as well, which is this idea that a markdown file is good enough.

Markdown is code now.

I can define how I want data piped out of one database into another.

Or I can tell an LLM to just look at this JSON file and extract the information out and make SQL queries to insert it into the database.

I don't need to spend all this time doing migration pipelines and stuff like that, writing a bunch of Python scripts through this stuff.

I just have my agent now.

I give it the clear instructions on what I expect out of the migration, what success looks like.

I define it clearly, but my data doesn't look quite right when I'm done.

And I think that that's really because something you and I have been bringing up a lot, which is like, why aren't people using agents to build the tool that does the thing? They're just using the agent to do the thing, right?

Yeah, yeah.

You can have all of your instructions in a markdown file and ask your agent to do some programmatic thing, but you don't really know what you're gonna get at the other end of that process.

On the other hand, you could have your agent build your process, you verify it, and then have a repeatable process that you can leverage for future iterations.

Yeah, I've been really trying to push people down this idea of like, you know, if you want to like prototype some type of migration workflow, I mean, you can do that with the LLM.

But whenever you're kind of like in a done spot, if like what you're trying to make needs to have like really repeatable results, which like for me, man, you know, a database migration pops into mind for me.

I know... other people have brought up some other really good examples but um i deal with these problems a lot where someone's like hey brandon i've got this old windows sql server database i need the data migrated out and into a nice modern postgresql database Um, I have seen some people taking approaches where they just snap two MCPs into their agent and just say to like migrate it.

Yeah.

I'm not surprised that the results are crazy and full of hallucinations.

Um, instead you could just like work back and forth with the NPC to like build out a script or maybe even you could start by sure maybe defining a skill, but I think you need to get to like building a script really quickly for these experiences where you need to have like a very predictable outcome, you know?

Yeah, absolutely.

You wouldn't want your teammates or team members or yourself to make direct queries to handle a migration.

You would put it all in a file so then it's repeatable and you can fix errors.

Same thing with the agent.

You want your AI to... to plan to build out the migration and then maybe execute on your behalf.

But it's something that you can verify and repeat.

So you would say you would probably lend more towards taking the agent and using it to, I mean, first formulate the plan.