AI reliability requires different SLOs

In this webinar clip, Alex Nauda, CTO of Nobl9, explains how keeping AI reliable means changing how you look at SLOs. Find out how to improve AI reliability with Gremlin → https://www.gremlin.com/solutions/improve-ai-reliability

Full Transcript:

There's a couple things that LLMs introduce that change how we have to manage these systems. These technologies are somewhat immature and they're changing really quickly, which means they remain immature.

The production hosting environments that we set up for these things have been remaining immature for some years. In a way, we wanna keep them on a shorter leash, but in a way, because they're sort of flexible systems and the users are using them in flexible ways, we have to give them a reasonable amount of leash, right?

But because of the immaturity of the technologies involved, I do find people want to give them a shorter leash. So they're doing things like really setting tight SLOs on the results that they get out of it. You know, looking at how many actual interactions, especially in chat systems, how many back and forth interactions they have on one topic, trying to identify certain situations or topics where it's clearly not working right. There's often a customer feedback with like a thumbs up, thumbs down in there. They're setting SLOs on that kind of interaction in order to really zero in on those places where they need some additional tuning.