Operations | Monitoring | ITSM | DevOps | Cloud

Best practices for scaling A/B testing with experiment protocols

Running one A/B test well is manageable. Scaling A/B testing to hundreds of experiments across dozens of teams is harder, because teams may make different decisions about metrics, randomization, rollout, and analysis. Experiment protocols in Datadog Experiments turn an approved methodology into a reusable configuration, giving experiments of the same type a consistent starting point. In this post, you’ll learn how to.

Monitor Oracle AI Database across clouds with Datadog

Organizations can use Oracle AI Database@AWS, Oracle AI Database@Azure, and Oracle AI Database@Google Cloud to run Oracle databases co-located with the cloud applications they support rather than managing the databases on premises. However, running applications in a cloud provider’s environment while hosting databases on Oracle Cloud Infrastructure (OCI) can lead to visibility gaps. When application performance degrades, identifying the source of the problem can be difficult.

Optimize AI costs with Agent Observability

AI agent costs accumulate along more dimensions than a model-call count. Spend is a product of both which models you call and how the agent is architected: how many turns the loop runs, how many tools it invokes, how much retrieved context it carries, and how much of that context is re-sent on every turn. Provider bills tell you how much you spent, but they’re keyed to API keys and models instead of the applications and workflow steps that generated the calls.

Build better SLOs with RUM and synthetic SLIs

Well-designed service level objectives (SLOs) help you track the benchmarks your app needs to meet to deliver a successful user experience (UX). To track performance against these SLOs, many SRE teams are turning to real user monitoring (RUM) and synthetic monitoring for their service level indicators (SLIs), as these forms of data provide more direct measurements of UX than backend performance metrics.

Introducing Scout 1.0: A new SRE model powering Bits AI with frontier performance at a fraction of the cost

Today, we’re introducing Scout 1.0, Datadog’s first AI model specialized for investigating production issues. Scout powers Bits Investigation, our AI SRE. We trained Scout to reduce inference cost so that investigations are cheap enough to run an order of magnitude more often and earlier in an incident’s lifecycle. Key highlights include: Close dialog.

Monitor the performance of critical steps in user journeys with RUM operations

Frontend and mobile applications depend on technical steps that users expect to work in every user journey: loading search results, signing in, saving a payment method, or processing a payment. Datadog Real User Monitoring (RUM) captures detailed events about these interactions, including views, actions, resources, and errors. But examining those events individually does not always provide a direct answer to more fundamental questions: Did a critical application step work, and how long did it take?

How Kubernetes DRA can improve GPU utilization and cut costs

Organizations are spending more on GPUs as they move AI workloads into production, and improving GPU utilization is a key priority. But Kubernetes’s legacy device plugin model can limit those improvements because it constrains how workloads request and consume GPU resources. Teams must configure GPU partitioning before workloads are scheduled. Resource requests cannot distinguish GPUs by capability, and a workload can reserve an entire GPU while using only part of its capacity.

How Datadog defines service criticality at scale

Datadog runs thousands of internal services and has been increasing global footprint year after year. For most of that growth, we treated services uniformly when making deployment and reliability decisions. At our current scale, that approach became expensive. A change destined for production rolls out to data centers sequentially, so deployment time grows with every data center we add.

Unify data across Datadog BYOC with cross-cluster queries for teams and AI agents

Organizations may need to keep data in specific regions or infrastructure they control, whether because of data residency requirements, internal policies, or transfer costs. Datadog Bring Your Own Cloud (BYOC) extends the Datadog platform into those environments. Each BYOC deployment runs as a Kubernetes cluster with its own object storage, keeping data close to its source.

Define user actions on your web app with visual labeling in Product Analytics

Adding or renaming a product event has traditionally meant creating an engineering ticket. Datadog Product Analytics uses the same SDKs and configuration as Real User Monitoring (RUM), and those SDKs autocapture actions such as clicks and taps. But an automatically generated action name describes the element rather than the user intent behind it.