eG Enterprise adds Advanced Performance Monitoring of Snowflake

Dowlath Sherief

Published on: May 11, 2023
Last updated on: August 28, 2024

I’m delighted to share that version 7.2 of eG Enterprise has introduced support for performance monitoring of Snowflake databases. eG Enterprise’s integration with Snowflake enables complete visibility into the Snowflake architecture and operations, alongside the performance and costs of any dependent cloud hosted infrastructures such as AWS or Azure.

What is Snowflake?

Snowflake is a cloud-based data warehousing platform that provides data storage, analysis, and reporting. It is designed to be highly scalable, secure, and highly available, and supports a wide range of data types and sources.

Snowflake’s cost-per-usage model is revolutionary in the cloud DB ecosystem and good tools to optimize performance and costs including those associated with cloud platform, storage and K8s dependencies are essential.

Snowflake uses a unique architecture that separates storage and computation, allowing for high levels of performance and scalability. It also provides a SQL-based interface for querying data, making it easy for analysts and data scientists to work with. Key features are:

A fully managed SaaS (software as a service) that provides a single platform for data warehousing, data lakes, data engineering, data science, secure sharing and consumption of real-time / shared data.
Out-of-the-box separates storage and compute, has on-the-fly scalable compute, data sharing, data cloning, and third-party tools support.
Enables data storage, processing, and analytic solutions that are faster, easier to use, and far more flexible than traditional offerings.
A Snowflake database is where an organization’s uploaded structured and semi-structured data sets are held for processing and analysis. Snowflake automatically manages all parts of the data storage process, including organization, structure, metadata, file size, compression, and statistics.

Figure 1: One-click dashboard templates allow you to monitor Snowflake in seconds with eG Enterprise v7.2

There are many customer case studies available from Snowflake, see: Case Study (snowflake.com). These offer very good insights into the type of use cases and services the Snowflake technologies and services can enable, both pricing and technological considerations are usually covered.

What is the Snowflake Architecture?

Snowflake is designed through three main components:

Cloud services: Snowflake uses ANSI SQL for cloud services empowering users to optimize their data. It eliminates the need for manual data warehouse management and tuning.
Query processing (compute): The compute layer of Snowflake is made up of virtual cloud data warehouses that let you analyze data. Each Snowflake virtual warehouse is an independent cluster and they do not compete for computing resources nor affect the performance of each other — which means workload concurrency is never a problem.
Database storage: The database storage layer holds all data loaded into Snowflake, including structured and semi structured data. Snowflake automatically manages all parts of the data storage process, including organization, structure, metadata, file size, compression, and statistics.

Image shows the snowflake database architecture

Figure 2: The Snowflake architecture

See: Key Concepts & Architecture — Snowflake Documentation for more details.

Key to Snowflake’s simplicity and scalability is its decoupled storage and compute model, which interact seamlessly for the end-user and makes scaling simple.

Consumption based Pricing of Snowflake

A huge number of organizations and businesses now build applications and services based on pay-as-you-use consumption models and subscriptions. So, using products such as Snowflake on the backend makes sense as costs will track end user demand and can be directly passed on to end-users. See Consumption-based Pricing: Ensuring Every Customer’s Success | Blog (snowflake.com) and Why Consumption-Based Apps Thrive on Snowflake – Blog for more information.

Adding support to monitor Snowflake allows us to better support our customers who leverage our APM support for technologies such as Java, .NET, Node.js and PHP to deliver consumption-based services and apps.

eG Enterprise Performance and Availability Monitoring for Snowflake

Leveraging AIOps technologies eG Enterprise offers an observability platform to help you use Snowflake at scale and assure performance, availability and cost optimizations. With eG Enterprise you can:

Quickly pinpoint costly and slow queries and drill into precise execution details to address bottlenecks
Optimize the storage requirements by looking at the usage patterns. Identifying spikes in storage used and large files being loaded inefficiently
Monitor the credit usage and take necessary actions where cost can be optimized
Detect any misconfiguration and security threats and proactively resolve them
Monitor the performance of the data warehouse and understand if the warehouse can satisfy user requirements. Predict when to resize the warehouse
Know if replication is enabled, and track lag between instances to get early insights into problems with replication
Get alerts out-of-the-box from AIOps driven anomaly detection to rapidly detect deviations from historical patterns or fluctuations of key resources such as storage
Use rich dashboards, topology maps and reports to break down communication silos across the organization

The eG Enterprise Snowflake Solution Page covers details of the features available, please see: Snowflake Monitoring and Performance Management.

Get Detailed Cloud and Snowflake Usage and Cost Insights in a Single Monitoring Platform

eG Innovations supports over 500+ Technology stacks, including most database technologies and public clouds such as Microsoft Azure and Amazon AWS. eG Enterprise allows you to go beyond Snowflake monitoring and cover the end-to-end user experience including the apps themselves that depend on Snowflake, underlying cloud infrastructure, networking – monitoring Snowflake in context, with eG Enterprise you can also:

Track and compare the performance and costs of different database technologies
Evaluate the overall costs of your Snowflake and Cloud Provider usage – see the combined costs of running Snowflake on AWS or Azure
Proactively monitor user access and authentication technologies including Active Directory, Azure AD and AWS access
Monitor technologies used in conjunction with Snowflake such as Kafka, Docker, SAP, Node.js and more. Support for over 500+ technology stacks, see: End-to-End Monitoring: Applications, Cloud, Containers
Monitor Cloud services such as Kubernetes (including Azure AKS or AWS EKS), Networking and Load Balancing, and detect cloud outages and issues in the same console as your Snowflake stack
Troubleshoot, optimize and capacity plan Cloud storage services such as AWS S3 or Azure Blob Storage. Metric thresholds, dynamic baselining, and alerting are configured out-of-the-box without the need for manual intervention
Leverage a comprehensive suite of APM (Application Performance Monitoring) tools for observability on applications or services dependent on your Snowflake usage
Use synthetic and RUM (Real User Monitoring) tools including protocol, full session and web application simulators allow the performance and behavior of Snowflake to be tested within the full context of applications and services depending on it
Transfer licenses across the technologies you monitor, with eG Enterprise you can monitor your on-prem Microsoft SQL Servers deployments before migration to Snowflake and continue to use the same product transferring after migration transferring licenses to avoid shelf-ware
Take advantage of a single interface with broad support for all major Database technologies allows you to monitor and benchmark Snowflake alongside other DB technologies to ensure you get the best cost and performance balance for your data workloads

To learn more about eG Enterprise support for Snowflake, please visit: Snowflake Monitoring and Performance Management.

eG Enterprise is an Observability solution for Modern IT. Monitor digital workspaces,
web applications, SaaS services, cloud and containers from a single pane of glass.

Free Trial See the platform

eG Enterprise is an Observability solution for Modern IT. Monitor digital workspaces,
web applications, SaaS services, cloud and containers from a single pane of glass.

Free Trial See the platform