Skip to main content

Streamline Your Workflow: Send Snowflake Alerts to Slack

snowflake

Do you know integrating Snowflake and Slack can make your life as a data engineer much easier? 

Here's why:

  • Real-time error catching and debugging: Instead of constantly checking logs for errors, you can set up Snowflake to automatically ping you in a Slack channel when something goes wrong. This is like having a dedicated assistant who watches for errors in your code and immediately lets you know so you can fix them faster. This is achieved through the use of webhooks, which are essentially automated HTTP requests that Snowflake sends to Slack when triggered by an event.
  • Keep everyone in the loop: Slack integration also means you can keep your entire team informed about the status of data pipelines and other processes. You can configure Snowflake to send notifications to a shared channel whenever a pipeline completes, fails, or encounters an issue. This keeps everyone on the same page and avoids unnecessary status update meetings.

This integration turns Slack into a powerful hub for managing your data workflow, making your life as a data engineer simpler and more productive.

How to Integrate Snowflake and Slack

Using Snowflake’s NOTIFICATION INTEGRATION function and Slack’s Webhooks feature it is possible to send messages to Slack from Snowflake.
This method can be used to send messages to Microsoft Teams and PagerDuty as well.

If you are not familiar with Slack, you can watch this video to get a quick overview of Slack.

Webhook is a http request. It is API call sometimes called reverse API or Push API. The defining feature of webhook is that the client instructs the server to callback (send http request to) the client when some event happens. In our case, the client, slack app, instructs the Snowflake to call (make a http request to an url endpoint) when some kind of event happens.

It is more about point of view: from which side you look at the process. From the Snowflake side, it is a regular API call, making a post request to Slack. But from the Slack side it is webhook (reverse API call) i.e, asking Snowflake to callback when an event occurs at a specific endpoint.


Steps

On Slack side

  • Create a channel for testing purposes (optional)
  • Create Slack app
  • Integrate app to the channel
  • Get app’s webhook URL

  1. Creating a channel for testing purposes (optional)

    1. Login to Slack, From Sidebar on the left click on “Add channels” button

      add channel

    2. Enter channel name

      enter channel name

    3. Choose channel visibility

      channel visibility

      public channels are visible to everyone within the workspace.
      private channels are visible to people who are invited.

    4. Click on the “Create” button

  2. Creating Slack App

    1. Open Slack Apps’s Dashboard page

    2. Click on “Create App” button

      Create app image

    3. Select “From Scratch” option

      From Scrach

    4. Enter App name and choose Workspace from drop-down

      Enter Aoo name and choose workspace

    5. Click “Create App” button

  3. Integrating the app to the channel

    1. On the App settings page, click “Incoming webhooks” link on the left sidebar

      incoming webhooks

      If you had closed the tab, you can find your app from Slack app dashboard

    2. Toggle on “Activate Incoming Webhooks”

      Activate incoming webhook

    3. Go to the bottom and click “Add New Webhook to Workspace” button

      Add New Webhook to Workspace

    4. Choose the channel you want to connect the app to and click “Allow” button

      select channel


      After Integration on the channel you can see the new message confirming it

      app_integrated

  4. Getting App’s webhook URL

    On App’s settings page Incoming Webhooks section go to bottom and copy “Webhook Url”, we will use it soon.

    Copy Webhook Url

    Note: App integration to a channel creates a unique Webhook Url for each channel。
    What it means is that if you want to send messages to a different Slack channel from Snowflake, that channel should also be connected to the app and uniquely generated Webhook URL for that channel should be used on the Snowflake side.

On Snowflake Side

  1. Storing Webhook Url’s token as a secret

    Necessary privileges：

    • Create secret
    • Read secret
    • Use secret

    Webhook Url looks like this： https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX.
    We will store the token that comes after https://hooks.slack.com/services/ part as a secret on Snowflake.


    CREATE OR REPLACE SECRET my_slack_webhook_secret
TYPE = GENERIC_STRING
SECRET_STRING = 'T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX';

    create secret


  2. Creating NOTIFICATION INTEGRATION

    Necessary privileges：

    • CREATE INTEGRATION
    • USAGE On INTEGRATION

    CREATE OR REPLACE NOTIFICATION INTEGRATION my_slack_webhook_int
TYPE=WEBHOOK
ENABLED=TRUE
WEBHOOK_URL='https://hooks.slack.com/services/SNOWFLAKE_WEBHOOK_SECRET'
WEBHOOK_SECRET=my_db.my_schema.my_slack_webhook_secret
WEBHOOK_BODY_TEMPLATE='{"text": "SNOWFLAKE_WEBHOOK_MESSAGE"}'
WEBHOOK_HEADERS=('Content-Type'='application/json');

    SNOWFLAKE_WEBHOOK_SECRET and SNOWFLAKE_WEBHOOK_MESSAGE are placeholders for a secret (in our case Webhook url token) and a message.


  3. Calling NOTIFICATION INTEGRATION function

    CALL SYSTEM$SEND_SNOWFLAKE_NOTIFICATION(
    SNOWFLAKE.NOTIFICATION.TEXT_PLAIN(
        SNOWFLAKE.NOTIFICATION.SANITIZE_WEBHOOK_CONTENT('Hello Slack!')
    ),
    SNOWFLAKE.NOTIFICATION.INTEGRATION('my_slack_webhook_int')
);

    SNOWFLAKE.NOTIFICATION.SANITIZE_WEBHOOK_CONTENT removes placeholders (for example, the SNOWFLAKE_WEBHOOK_SECRET placeholder, which specifies a secret) from the body of a notification message to be sent.
    If this function is not used and the message body contains a placeholder for a secret, then when a message is sent to a Slack webhook, a message containing a secret will be exposed to the Slack channel.

    The rest are set functions to be used with NOTIFICATION INTEGRATION to send a message.

Result

final_result


If calling 4 functions and stored procedures every time to send a message to a Slack is bothersome, a wrapper stored procedure can be used to simplify the process.

CREATE OR REPLACE PROCEDURE post_on_slack(message)
RETURNS FLOAT
LANGUAGE SQL
AS
BEGIN
    CALL SYSTEM$SEND_SNOWFLAKE_NOTIFICATION(
        SNOWFLAKE.NOTIFICATION.TEXT_PLAIN(
            SNOWFLAKE.NOTIFICATION.SANITIZE_WEBHOOK_CONTENT(message)
        ),
        SNOWFLAKE.NOTIFICATION.INTEGRATION('my_slack_webhook_int')
    );
END;

Call post_on_slack('Hello Slack!')
Labels:

Comments

Post a Comment

Popular posts from this blog

脱初心者！ Git ワークフローを理解して開発効率アップ

Git – チーム開発に必須のバージョン管理システムですが、その真価を発揮するにはワークフローの理解が欠かせません。 色々な人は Git の使い方を良く知っていますが、Git を仕事やワークフローに統合する方法を余り良く知らない人もいます。本記事では、Git をワークフローに組み込むことで、開発プロセスがどのように効率化され、チーム全体のパフォーマンスが向上するのかを解説します。Centralized Workflow から Forking Workflow まで、代表的な 9 つのワークフローの特徴を分かりやすく紹介します。それぞれのメリット・デメリット、そして最適なユースケースを理解することで、あなたのプロジェクトに最適なワークフローを選択し、開発をスムーズに進めましょう！ Centralized Workflow Feature branching/GitHub Flow Trunk Based Flow Git Feature Flow Git Flow Enhanced Git Flow One Flow GitLab Flow Forking Workflow 分かりやすくするために、同じコンセプトを説明するに一つ以上の図を使った場合があります。 Centralized Workflow 説明： 集中化ワークフローではプロジェクトにおけるすべての変更の単一の入力箇所として中央リポジトリを使用します。デフォルトの開発用ブランチは main と呼ばれ、すべての変更がこのブランチにコミットされます。 集中化ワークフローでは main 以外のブランチは不要です。チームメンバー全員がひとつのブランチで作業し、変更を直接中央リポジトリにプッシュします。 メリット： SVN のような集中型バージョン管理システムから移行する小規模チームに最適。 デメリット： お互いのコードが邪魔になり (お互いの変更を上書きするように)、プロダクション環境にバグをい入れる可能性が高くて、複数のメンバいるチームでこのフローを使いにくい。 地図： graph TD; A[Central Repository] -->|Clone| B1[Developer A's Local Repo] A --...
Post a Comment
Read more

Introduction to SQLFluff: How to make your SQL code clean and error-free

Image by Jake Aldridge from Pixabay You know oftentimes, the cause of runtime or compile errors and hours of debugging agony is all due to simply a missing semicolon. Have you ever had such experience? If you had, you are not alone. There are two ways to avoid these unfortunate situations: either become a perfect developer who never makes mistakes, or use helpful tools such as linters that can catch these errors early on. I am nowhere near being a perfect developer who never makes a mistake. In fact, I'm probably the opposite of a perfect developer, so even if I wanted to, I wouldn’t be able to teach you how to become a perfect developer. But what I can teach you is using linters. A Wikipedia defines a linter as a "static code analysis tool used to flag programming errors, bugs, stylistic errors and suspicious constructs." If you're not convinced yet on using linters, consider this scenario: in a large project with multiple members, different people tend to ...
Post a Comment
Read more

WinMerge のセットアップと使う方

WinMerge は、Windows 用のオープン ソースの差分およびマージ ツールです。WinMerge は、フォルダーとファイルの両方を比較し、違いを理解して扱いやすい視覚的なテキスト形式で表示します。この記事でWinMerge のセットアップと使う方を教えます。 source: https://winmerge.org WinMerge をダウンロード WinMerge のウェブサイト に行って、「WinMerge-2.16.44-x64-Setup.exe」ボタンを押し、WinMerge 2.16 をダウンロードしてください。 WinMerge をインストール ダウンロードされたソフトウェアをクリックし、ポップアップ画面で「Next」を押してください 「Languages」部分をスクロールダウンし、「Japanese menus and dialogs」を選択し、「Next」ボタンを押してください ターミナル等からも WinMerge をアクセス出来ようにする為に「Add WinMerge folder to your system path」オプションを選択し、希望によって他のオプション選択してください 「Enable Explorer context menu Integration」オプションを選択したら、フォルダ／ファイルを右キリックし、コンテクストメニューから WinMerge を開くようになります。 「Install」ボタンを押し、「Next」ボタンを押し、その後、「Finish」ボタンを押してください 言語を日本語にする もし WinMerge の言語が日本語じゃなくて、英語なら、「Edit」タブから「Options」を押してください。 ポップアップ画面で右側の下にある「Languages」と言うドロップダウンメニューから日本語を選択し、「OK」ボタンを押してください WinMerge を使う方 「ファイル」タッブから「開く」を押し 参照ボタンを押し、比較したいフォルダ・ファイルを指定 比較したいフォルダを指定する方法: ポップアップ画面から対象のフォルダーを選択し、「Open」を押してくだい 何も選択しないで、「Open」を押してくだい 右側下にある「比較」ボタンを押し ...
Post a Comment
Read more