> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aieev.com/llms.txt
> Use this file to discover all available pages before exploring further.

# External Channels

> Send notifications to Slack, Discord, and webhooks

Send console notifications to the Slack or Discord channels your team uses, or to a server you run (webhooks). Choose which notifications each channel receives, and pick channels on [log alert](https://docs.aieev.com/docs/air-container/log-alerts.md) rules.

## Supported Channels

| Type | URL | Notes |
| :- | :- | :- |
| **Slack** | `https://hooks.slack.com/services/…` | A Slack [incoming webhook](https://api.slack.com/messaging/webhooks) URL |
| **Discord** | `https://discord.com/api/webhooks/…` | A Discord channel [webhook](https://support.discord.com/hc/articles/228383668) URL |
| **Webhook** | Any `https://` URL reachable from the internet | Signed the [Standard Webhooks](https://www.standardwebhooks.com) way. See [Receiving Webhooks](#receiving-webhooks). |

* Every channel URL must use `https://`.
* An organization can connect up to 10 channels. Channels that are off count toward the limit.
* Once saved, a URL is never shown in full again — only the start and the last few characters.

## Connect a Channel

1. Open **Notifications** in the console and select the **External channels** tab. You can also get there from the **External channels** card in organization settings.
2. Click **Connect channel**.
3. Paste the channel URL. Slack and Discord URLs select their type automatically. **How to get a URL** next to the field explains how to create one.
4. Choose the **Notifications to send** and the options.
5. Click **Send test message** to make sure messages arrive, then save.

The **Nickname** tells channels apart in the console. If you leave it blank, Slack and Discord channels are named after their type, and webhooks after the URL's host name.

When you create a webhook channel, its **webhook signing secret** is shown only once. Store it on the receiving server — the dialog closes only after you copy it.

## Choose What to Send

| Option | What is sent |
| :- | :- |
| **Failures and warnings** (default, recommended) | Container deploy failed, Container unhealthy, Rented resource expiring soon, Low balance warning, Credits expiring soon, Containers suspended (out of credit) |
| **All** | Every notification that can be sent. Billing notifications are left out unless you are the Organization Owner. |
| **Choose** | Pick the notifications you want, by area. |

> [!NOTE]
>
> A channel stores the exact list you picked. Notification types added later are not added to existing channels automatically.

### Notifications You Can Send

| Area | Notifications |
| :- | :- |
| **Container** | Deployed, deploy failed, unhealthy, autoscaling applied, waitlisted GPU capacity available, replica limit increase approved |
| **Rentals** | Rental completed, rented resource started, expiring soon, expired |
| **Credit** | Low balance warning, credits expiring soon, containers suspended (out of credit) |
| **Billing** (Owner only) | Credit topped up, auto top-up succeeded, recurring payment credited, payment failed, auto top-up failed, payment refunded, invoice overdue |
| **Members** | Member joined, member left |

These notifications are never sent to external channels:

* **Log alerts**: Sent only to the channels picked on each rule, regardless of channel subscriptions.
* **Notifications just for you**: Request received and request updates, replies to your feedback.
* **Service announcements** and **External channel turned off** notifications.

### Only Specific Projects

Turn on **Only send container notifications from specific projects** to send container notifications from the projects you pick only. Credit, rental, and member notifications are sent regardless of project.

## Message Options

* **Message language**: Korean or English. It defaults to the organization's region (English for the US, Korean otherwise) and is stored per channel, so changing the organization's region later does not change existing channels. Times in messages are shown in KST for Korean and UTC for English.
* **Mention @here on failure alerts**: Adds `@here` to failure (red) alerts in Slack and Discord channels. Off by default.
* **Include log lines and deployment failure reasons**: Includes the matching lines of a log alert and the cause of a deploy failure in the message. On by default. When off, only the title and the console link are sent — turn it off if your logs may contain sensitive values.

## What Messages Look Like

In Slack and Discord, each notification arrives as a single card.

* The card color shows severity — failure (red), warning (amber), success (green), info (gray).
* Below the title and text, the matching log lines of a log alert (up to 5) or the cause of a deploy failure appear in a code block.
* Links such as **View container**, **View logs**, and **View billing** take you to the right console page.
* The footer reads `AirCloud · {organization name}`.

## Receiving Webhooks

### Request

AirCloud sends a `POST` request to the channel URL for each notification.

| Header | Value |
| :- | :- |
| `Content-Type` | `application/json` |
| `User-Agent` | `AirCloud-Notifications/1.0` |
| `webhook-id` | The delivery ID. It stays the same across automatic retries and manual resends. |
| `webhook-timestamp` | When the request was sent (Unix seconds). It changes on every attempt. |
| `webhook-signature` | The signature, in the form `v1,<signature>` |

### Body

```json theme={null}
{
  "id": "6f1c2b0e-2d4a-4a57-9c3e-0b8f0d6a1e42",
  "type": "endpoint.deploy_failed",
  "version": 1,
  "created_at": "2026-11-03T05:12:33.123456+00:00",
  "severity": "error",
  "organization": { "id": "org-xxxxxxxxxxxx", "name": "Acme" },
  "title": "Container 'my-llm' deployment failed",
  "text": null,
  "url": "https://…/console/projects/…/container/…",
  "data": { "endpoint_id": "…", "endpoint_name": "my-llm", "project_id": "…" },
  "detail": "RuntimeError: …"
}
```

| Field | Description |
| :- | :- |
| `id` | The notification ID. It is the same on every channel that receives the notification. |
| `type` | The notification type, for example `endpoint.deploy_failed`, `endpoint.log_alert`, or `balance.low_balance`. Test messages use `integration.test`. |
| `version` | The body format version. Always `1` for now. |
| `created_at` | When the notification was created (ISO 8601, UTC) |
| `severity` | One of `error`, `warning`, `success`, `info` |
| `organization` | The organization's ID and name |
| `title`, `text` | The title and text, in the channel's message language. `text` is `null` when there is none. |
| `url` | The relevant console page, or `null` |
| `data` | Extra details. Fields differ by notification type and may change, so prefer `title`, `text`, and `url`. |
| `detail` | The original failure reason for deploy failed and unhealthy notifications. `null` for other notifications, and on channels with **Include log lines and deployment failure reasons** off. |

### Verify the Signature

Webhooks are signed according to [Standard Webhooks](https://www.standardwebhooks.com). The signature is the HMAC-SHA256 of `{webhook-id}.{webhook-timestamp}.{body}` with your signing secret, base64-encoded and prefixed with `v1,`. The key is the base64 string after `whsec_`, decoded.

* Verify against the **raw request bytes**. Parsing and re-serializing the JSON breaks the signature.
* Reject requests whose `webhook-timestamp` is too old (for example, more than 5 minutes) to prevent replay attacks.
* The [official libraries](https://github.com/standard-webhooks/standard-webhooks) for each language verify signatures for you.

```python Python theme={null}
import base64
import hashlib
import hmac
import time

TOLERANCE_SECONDS = 5 * 60


def verify(secret: str, headers: dict, body: bytes) -> bool:
    msg_id = headers["webhook-id"]
    timestamp = headers["webhook-timestamp"]
    signature = headers["webhook-signature"]

    if abs(time.time() - int(timestamp)) > TOLERANCE_SECONDS:
        return False

    key = base64.b64decode(secret.removeprefix("whsec_"))
    signed = f"{msg_id}.{timestamp}.".encode() + body
    expected = base64.b64encode(hmac.new(key, signed, hashlib.sha256).digest()).decode()

    for item in signature.split(" "):
        version, _, value = item.partition(",")
        if version == "v1" and hmac.compare_digest(value, expected):
            return True
    return False
```

> [!NOTE]
>
> A test message sent from the connect dialog before saving has no signing secret yet, so it arrives without a `webhook-signature` header. Test messages from a saved channel are signed.

### Responses and Retries

* **Success**: Any 2xx response counts as delivered. Respond within **10 seconds**. Redirects are not followed.
* **Retries**: If a delivery fails or gets no response, it is retried after 30 seconds, 2 minutes, 10 minutes, 30 minutes, 1 hour, and 2 hours (up to 7 attempts including the first, about 4 hours in total). Deliveries older than 6 hours are no longer sent.
* **Retry-After**: A `Retry-After` header is honored (up to 30 minutes).
* **Deduplication**: The same notification can arrive more than once. Deduplicate on `webhook-id`.

| Response | Slack and Discord | Webhook |
| :- | :- | :- |
| 2xx | Delivered | Delivered |
| 429, 5xx, timeout, connection error | Retried | Retried |
| Other 4xx | Failed without retry | Retried |
| Channel deleted or no permission (see below) | Channel turned off | Turned off on 410 |

## Delivery History and Resending

Click a channel to open its details: the on/off switch, test messages, and recent deliveries.

* **Recent deliveries**: Shows the latest 50, and history is kept for 30 days. Each is **Pending**, **Sent**, **Failed**, or **Skipped**.
* **Retry**: Only failed deliveries can be resent. A resend starts the retry cycle over, with the same `webhook-id`.
* **Test messages** are not recorded in the history and never include `@here`.

## When a Channel Is Turned Off Automatically

AirCloud turns a channel off when it can no longer deliver to it.

| Situation | What to do |
| :- | :- |
| The Slack or Discord channel or the webhook was deleted (Slack 404 or 410, Discord 404, webhook 410) | Update the URL, then turn the channel back on. |
| No permission to post to the channel (Slack or Discord 401 or 403) | Create a new webhook, update the URL, then turn the channel back on. |
| The URL resolves to a private network address | Use a URL that is reachable from the internet. |
| Deliveries failed for more than a day | Check the receiving side, then turn the channel back on. |

* When a channel is turned off, the whole organization gets an **External channel turned off** notification, and the Organization Owner gets an email.
* Pending deliveries at that moment become **Skipped**. Notifications that occur while a channel is off are not sent later, even after you turn it back on.
* To turn it back on, use the switch in the channel's details. Changing the URL alone does not turn a channel back on.
* Turning a channel off or deleting it yourself sends no notification.

## Regenerate the Signing Secret

**Regenerate signing secret** in a webhook channel's details issues a new secret. **The current secret stops working immediately** — there is no grace period. Signature checks fail until the receiving server uses the new secret, so update it right away. Pending retries are signed with the new secret too.

## Permissions

| Action | Member | Owner |
| :- | :-: | :-: |
| Connect, edit, turn on or off, and disconnect channels; regenerate the signing secret; send test messages; view delivery history | ✅ | ✅ |
| Attach channels to log alert rules | ✅ | ✅ |
| Resend failed deliveries (except billing notifications) | ✅ | ✅ |
| Choose billing notifications, resend billing notifications | — | ✅ |
| Change the URL of, turn off, delete, or regenerate the secret of a channel that receives billing notifications | — | ✅ |

## URL Restrictions

* Only `https://` URLs are allowed, and URLs cannot contain a user name or password.
* Internal names such as `localhost`, `.local`, and `.internal`, and private, loopback, and link-local IP addresses are not allowed.
* The address is checked again on every delivery. If it resolves to a private address, the delivery is not sent and the channel is turned off.

## Disconnect a Channel

**Disconnect** in a channel's details deletes the channel. Its delivery history is deleted too, and it is removed from any log alert rules that used it.

## Related Documentation

> [card] [Notifications](https://docs.aieev.com/docs/console/notifications.md)
>
> See what you can be notified about and choose console, desktop, and email delivery.

> [card] [Log Alerts](https://docs.aieev.com/docs/air-container/log-alerts.md)
>
> Alert on container log conditions and send them to external channels.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.