Skip to content

Getting started with Observability

This guide walks you through creating an Observability workspace and forwarding metrics with Prometheus remote write. The same workspace also provides Loki endpoints for logs — see Logs after metrics are flowing.

Info

Observability workspaces are currently in Private Beta. Access may be limited — contact support for early access.

General availability is planned for August 2026.

Prerequisites

  • A Thalassa Cloud organisation and project
  • IAM permission to create observability workspaces (observability:WorkspaceFullAccess or equivalent)
  • A Prometheus instance, Prometheus Operator, or OpenTelemetry Collector that can remote-write metrics
  • A service account (recommended) for production ingest, with Prometheus remote-write permissions and OIDC credentials

Step 1: Create a workspace

Create in the console

  1. Open the Thalassa Cloud Console.
  2. Navigate to ObservabilityWorkspaces.
  3. Click Create Workspace.
  4. Configure:
    • Name — for example production-observability
    • Region — usually the same region as your workloads
    • Retention — days to retain metrics and logs (optional; regional default if omitted)
  5. Click Create and wait until status is Ready.

Create via API

curl -X POST "https://api.thalassa.cloud/v1/observability/workspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Project-Identity: $PROJECT_ID" \
  -d '{
    "name": "production-observability",
    "regionIdentity": "nl-01",
    "retentionDays": 30
  }'

Creating a workspace enables both metrics and logs backends. Endpoints appear on the resource when status is ready.

Step 2: Collect endpoint URLs and credentials

Workspace endpoints

On the workspace Overview, copy:

EndpointPurpose
Remote write URLPrometheus / Agent metrics ingest
Prometheus query URLPromQL queries (Grafana, scripts)
Alerting URLRuler / Alertmanager base (when using tools)
Logs push URLLoki push API
Logs query URLLogQL queries

URLs look like:

https://prometheus.<region>.thalassa.cloud/workspace/obsw-…/api/v1/push
https://loki.<region>.thalassa.cloud/workspace/obsw-…/loki/api/v1/push

The workspace ID is already in the path — use the URL as provided.

Authentication

Use a service account for collectors:

  1. Create a service account under IAMService Accounts.
  2. Bind policies that allow workspace data-plane access — at minimum Prometheus remote write for metrics ingest (for example prometheus:RemoteWriteFullAccess or prometheus:FullAccess), scoped so the principal can use this workspace.
  3. Create OIDC client credentials for the service account.
  4. Ensure the token includes the required scopes (for example observability:prometheus:remote_write).

Personal access tokens work for testing but are not recommended for production agents.

Step 3: Configure Prometheus remote write

prometheus.yml

Replace the URL with your workspace remote write URL from the console:

global:
  scrape_interval: 15s
  external_labels:
    cluster: production
    region: nl-01

scrape_configs:
  - job_name: "kubernetes-pods"
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true

remote_write:
  - url: https://prometheus.<region>.thalassa.cloud/workspace/obsw-<id>/api/v1/push
    oauth2:
      client_id: <service-account-client-id>
      client_secret: <service-account-client-secret>
      token_url: https://api.thalassa.cloud/oidc/token
    queue_config:
      max_samples_per_send: 1000
      batch_send_deadline: 5s
      max_retries: 3

Reload Prometheus

# systemd
sudo systemctl reload prometheus

# or API
curl -X POST http://localhost:9090/-/reload

For Prometheus Operator and advanced queue/relabel settings, see Metrics remote write.

Step 4: Verify ingest and query

Check the agent

  1. Open your Prometheus UI → StatusRuntime (remote write section).
  2. Confirm the remote write endpoint is up and samples are sent without sustained errors.

Query from the gateway

curl -H "Authorization: Bearer $TOKEN" \
  "https://prometheus.<region>.thalassa.cloud/workspace/obsw-<id>/api/v1/query?query=up"

Query in the console

  1. Open the workspace.
  2. Open the Metrics view.
  3. Run up and confirm series from your collector.

Step 5: Optional — alerts and logs

  • Create alert or recording rules from the workspace Rules tab, or with mimirtool.
  • Configure Alertmanager receivers on the Alerts tab.
  • Send logs to the workspace push URL — see Logs.

Next steps