Skip to main content

Streamline Your Workflow: Send Snowflake Alerts to Slack

snowflake

Do you know integrating Snowflake and Slack can make your life as a data engineer much easier? 

Here's why:

  • Real-time error catching and debugging: Instead of constantly checking logs for errors, you can set up Snowflake to automatically ping you in a Slack channel when something goes wrong. This is like having a dedicated assistant who watches for errors in your code and immediately lets you know so you can fix them faster. This is achieved through the use of webhooks, which are essentially automated HTTP requests that Snowflake sends to Slack when triggered by an event.
  • Keep everyone in the loop: Slack integration also means you can keep your entire team informed about the status of data pipelines and other processes. You can configure Snowflake to send notifications to a shared channel whenever a pipeline completes, fails, or encounters an issue. This keeps everyone on the same page and avoids unnecessary status update meetings.

This integration turns Slack into a powerful hub for managing your data workflow, making your life as a data engineer simpler and more productive.

How to Integrate Snowflake and Slack

Using Snowflake’s NOTIFICATION INTEGRATION function and Slack’s Webhooks feature it is possible to send messages to Slack from Snowflake.
This method can be used to send messages to Microsoft Teams and PagerDuty as well.

If you are not familiar with Slack, you can watch this video to get a quick overview of Slack.

Webhook is a http request. It is API call sometimes called reverse API or Push API. The defining feature of webhook is that the client instructs the server to callback (send http request to) the client when some event happens. In our case, the client, slack app, instructs the Snowflake to call (make a http request to an url endpoint) when some kind of event happens.

It is more about point of view: from which side you look at the process. From the Snowflake side, it is a regular API call, making a post request to Slack. But from the Slack side it is webhook (reverse API call) i.e, asking Snowflake to callback when an event occurs at a specific endpoint.


Steps

On Slack side

  • Create a channel for testing purposes (optional)
  • Create Slack app
  • Integrate app to the channel
  • Get app’s webhook URL

  1. Creating a channel for testing purposes (optional)

    1. Login to Slack, From Sidebar on the left click on “Add channels” button

      add channel

    2. Enter channel name

      enter channel name

    3. Choose channel visibility

      channel visibility

      public channels are visible to everyone within the workspace.
      private channels are visible to people who are invited.

    4. Click on the “Create” button

  2. Creating Slack App

    1. Open Slack Apps’s Dashboard page

    2. Click on “Create App” button

      Create app image

    3. Select “From Scratch” option

      From Scrach

    4. Enter App name and choose Workspace from drop-down

      Enter Aoo name and choose workspace

    5. Click “Create App” button

  3. Integrating the app to the channel

    1. On the App settings page, click “Incoming webhooks” link on the left sidebar

      incoming webhooks

      If you had closed the tab, you can find your app from Slack app dashboard

    2. Toggle on “Activate Incoming Webhooks”

      Activate incoming webhook

    3. Go to the bottom and click “Add New Webhook to Workspace” button

      Add New Webhook to Workspace

    4. Choose the channel you want to connect the app to and click “Allow” button

      select channel


      After Integration on the channel you can see the new message confirming it

      app_integrated

  4. Getting App’s webhook URL

    On App’s settings page Incoming Webhooks section go to bottom and copy “Webhook Url”, we will use it soon.

    Copy Webhook Url

    Note: App integration to a channel creates a unique Webhook Url for each channel。
    What it means is that if you want to send messages to a different Slack channel from Snowflake, that channel should also be connected to the app and uniquely generated Webhook URL for that channel should be used on the Snowflake side.

On Snowflake Side

  1. Storing Webhook Url’s token as a secret

    Necessary privileges：

    • Create secret
    • Read secret
    • Use secret

    Webhook Url looks like this： https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX.
    We will store the token that comes after https://hooks.slack.com/services/ part as a secret on Snowflake.


    CREATE OR REPLACE SECRET my_slack_webhook_secret
TYPE = GENERIC_STRING
SECRET_STRING = 'T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX';

    create secret


  2. Creating NOTIFICATION INTEGRATION

    Necessary privileges：

    • CREATE INTEGRATION
    • USAGE On INTEGRATION

    CREATE OR REPLACE NOTIFICATION INTEGRATION my_slack_webhook_int
TYPE=WEBHOOK
ENABLED=TRUE
WEBHOOK_URL='https://hooks.slack.com/services/SNOWFLAKE_WEBHOOK_SECRET'
WEBHOOK_SECRET=my_db.my_schema.my_slack_webhook_secret
WEBHOOK_BODY_TEMPLATE='{"text": "SNOWFLAKE_WEBHOOK_MESSAGE"}'
WEBHOOK_HEADERS=('Content-Type'='application/json');

    SNOWFLAKE_WEBHOOK_SECRET and SNOWFLAKE_WEBHOOK_MESSAGE are placeholders for a secret (in our case Webhook url token) and a message.


  3. Calling NOTIFICATION INTEGRATION function

    CALL SYSTEM$SEND_SNOWFLAKE_NOTIFICATION(
    SNOWFLAKE.NOTIFICATION.TEXT_PLAIN(
        SNOWFLAKE.NOTIFICATION.SANITIZE_WEBHOOK_CONTENT('Hello Slack!')
    ),
    SNOWFLAKE.NOTIFICATION.INTEGRATION('my_slack_webhook_int')
);

    SNOWFLAKE.NOTIFICATION.SANITIZE_WEBHOOK_CONTENT removes placeholders (for example, the SNOWFLAKE_WEBHOOK_SECRET placeholder, which specifies a secret) from the body of a notification message to be sent.
    If this function is not used and the message body contains a placeholder for a secret, then when a message is sent to a Slack webhook, a message containing a secret will be exposed to the Slack channel.

    The rest are set functions to be used with NOTIFICATION INTEGRATION to send a message.

Result

final_result


If calling 4 functions and stored procedures every time to send a message to a Slack is bothersome, a wrapper stored procedure can be used to simplify the process.

CREATE OR REPLACE PROCEDURE post_on_slack(message)
RETURNS FLOAT
LANGUAGE SQL
AS
BEGIN
    CALL SYSTEM$SEND_SNOWFLAKE_NOTIFICATION(
        SNOWFLAKE.NOTIFICATION.TEXT_PLAIN(
            SNOWFLAKE.NOTIFICATION.SANITIZE_WEBHOOK_CONTENT(message)
        ),
        SNOWFLAKE.NOTIFICATION.INTEGRATION('my_slack_webhook_int')
    );
END;

Call post_on_slack('Hello Slack!')
Labels:

Comments

Post a Comment

Popular posts from this blog

脱初心者！ Git ワークフローを理解して開発効率アップ

Git – チーム開発に必須のバージョン管理システムですが、その真価を発揮するにはワークフローの理解が欠かせません。 色々な人は Git の使い方を良く知っていますが、Git を仕事やワークフローに統合する方法を余り良く知らない人もいます。本記事では、Git をワークフローに組み込むことで、開発プロセスがどのように効率化され、チーム全体のパフォーマンスが向上するのかを解説します。Centralized Workflow から Forking Workflow まで、代表的な 9 つのワークフローの特徴を分かりやすく紹介します。それぞれのメリット・デメリット、そして最適なユースケースを理解することで、あなたのプロジェクトに最適なワークフローを選択し、開発をスムーズに進めましょう！ Centralized Workflow Feature branching/GitHub Flow Trunk Based Flow Git Feature Flow Git Flow Enhanced Git Flow One Flow GitLab Flow Forking Workflow 分かりやすくするために、同じコンセプトを説明するに一つ以上の図を使った場合があります。 Centralized Workflow 説明： 集中化ワークフローではプロジェクトにおけるすべての変更の単一の入力箇所として中央リポジトリを使用します。デフォルトの開発用ブランチは main と呼ばれ、すべての変更がこのブランチにコミットされます。 集中化ワークフローでは main 以外のブランチは不要です。チームメンバー全員がひとつのブランチで作業し、変更を直接中央リポジトリにプッシュします。 メリット： SVN のような集中型バージョン管理システムから移行する小規模チームに最適。 デメリット： お互いのコードが邪魔になり (お互いの変更を上書きするように)、プロダクション環境にバグをい入れる可能性が高くて、複数のメンバいるチームでこのフローを使いにくい。 地図： graph TD; A[Central Repository] -->|Clone| B1[Developer A's Local Repo] A --...
Post a Comment
Read more

From Generic to Genius: Fine-tuning LLMs for Superior Accuracy in Snowflake

TL;DR: Cortex Fine-tuning is a fully managed service that lets you fine-tune popular LLMs using your data, all within Snowflake. While large language models (LLMs) are revolutionizing various fields, their "out-of-the-box" capabilities might not always align perfectly with your specific needs. This is where the power of fine-tuning comes into play. As it will be explained in this article, this feature empowers you to take a base LLM and customize it to excel in your particular domain. Here's the brief summary of why you might want to leverage Snowflake's fine-tuning capabilities: Unlocking Domain Expertise : Pre-trained LLMs are trained on massive, general datasets. Fine-tuning allows you to build upon this foundation and train the LLM further using data specific to your field, such as legal documents, medical records, or financial data. This empowers the LLM to understand complex terminology and patterns unique to your domain, leading to more accurate a...
Post a Comment
Read more

How Wendy’s Successfully Penetrated the Japanese Market After Long Struggles

Wendy’s had long struggled to penetrate the Japanese market. Initially the Daiei Group tried to bring Wendy’s to Japan but failed. The next owner of Wendy’s’ Japanese franchise, Zensho Holdings Co. also failed miserably. However, Japanese-American entrepreneur Ernest M. Higa seems to have managed to do the task. This article will discuss the challenges Wendy’s faced when entering the Japanese market, how Ernie Higa addressed those challenges, macro environmental factors that impacted the success of the brand in Japan, future threats the Japanese fast food market is facing , and potential solutions. The prior challenges that Wendy’s faced when they entered the Japanese market There is no one-size-fits-all formula in business, especially when Japan is involved in the conversation. According to Japanese-American entrepreneur Ernie Higa, even if a company has a good product and good pricing, penetrating the Japanese market is more difficult compared to the US’s market. Foreign e...
Post a Comment
Read more