Operations | Monitoring | ITSM | DevOps | Cloud

As organizations break down monolithic applications in favor of a more distributed, microservices-based architecture, they need to collect increasing amounts of metric data. But how do you summarize this data to provide insights at scale? Averages are simple to calculate but can be misleading, especially for increasingly complex and distributed environments that contain outlier values that skew the average.

Organizations in financial services, insurance, healthcare, and other regulated industries must meet stringent data residency, privacy, and regulatory requirements while maintaining full visibility into their systems. This becomes challenging when logs need to remain at rest in customers’ environments or specific regions, hindering teams’ ability to attain seamless observability and insight.

Being an on-call engineer is often overwhelming, requiring you to pivot between tickets, dashboards, runbooks, and different data sources as you try to separate legitimate incidents from unnecessary noise. Not only does the process of investigating irrelevant alerts take time away from remediating important issues, but it also compounds alert fatigue.

In Part 2 of this series, we looked at Snowflake’s built-in monitoring services for compute, query, and storage. In this post, we’ll demonstrate how Datadog complements and extends Snowflake’s existing monitoring and data visualization capabilities, enabling teams to get deeper visibility and extract more valuable insights from their Snowflake data.

In Part 1 of this series, we looked at how Snowflake enables users to easily store, process, analyze, and share high volumes of structured and semi-structured data, as well as key metrics for monitoring compute costs, storage, and datasets. In this post, we’ll walk through how to collect and analyze these metrics using Snowsight, Snowflake’s built-in web interface.

Snowflake is a self-managed data platform that enables users to easily store, process, analyze, and share high volumes of structured and semi-structured data. One of the most popular data platforms on the market, Snowflake has gained widespread adoption because it addresses a range of data challenges with a unified, scalable, and high-performance platform. Snowflake’s flexibility enables users to handle diverse workloads, such as data lake and data warehouse integration.

Monitoring cloud costs can be complex. When those costs span more than one cloud service provider (CSP) or SaaS provider, that complexity can make it difficult to understand your overall spending. Datadog Cloud Cost Management (CCM) enables teams to understand cloud costs, but each provider tags its cost data differently. Teams need to understand each provider’s unique cost data model before they can make sense of their costs in each cloud.

Google’s comprehensive AI offering includes Vertex AI, a cloud-based platform for building and deploying AI applications, AI Studio, a web platform for quickly prototyping and testing AI applications, and Gemini, their multimodal model. Gemini offers advanced capabilities in image, code, and text generation and can be used to implement chatbot assistants, perform complex data analysis, generate design assets, and more.

Etcd plays a critical role in your Kubernetes setup: it stores the ever-changing state of your cluster and its objects, and the API server uses this data to manage cluster resources. As your applications thrive and your Kubernetes clusters see more traffic, etcd handles an increasing amount of data. But etcd’s storage space is limited: the recommended maximum is 8 GiB, and a large and dynamic cluster can easily generate enough data to reach that limit.

Pinecone is a vector database that helps users build and deploy generative AI applications at scale. Whether using its serverless architecture or a hosted model, Pinecone allows users to store, search, and retrieve the most meaningful information from their company data with each query, sending only the necessary context to Large Language Models (LLMs). By providing the ability to search and retrieve contextual data, Pinecone enables you to reduce LLM hallucinations and enhance data security.