Files

T

Hanzo Dev cbb18f4cc3 refactor: complete clickhouse->datastore debrand — zero 'clickhouse' in .go (packages/dirs/types/aliases renamed to datastore); datastore-go/v2 driver; sentry intact; hanzoai/sqlite. (3 deploy/integration dirs reference the real upstream ClickHouse server engine — flagged, not our API)

2026-07-12 18:54:57 -07:00

10 KiB

Raw Blame History

Integration Tests

O11y uses integration tests to verify that different components work together correctly in a real environment. These tests run against actual services (Datastore, PostgreSQL, O11y, Zeus mock, Keycloak, etc.) spun up as containers, so suites exercise the same code paths production does.

How to set up the integration test environment?

Prerequisites

Before running integration tests, ensure you have the following installed:

Python 3.13+
uv
Docker (for containerized services)

Initial Setup

Navigate to the shared tests project:

cd tests

Install dependencies using uv:

uv sync

NOTE: the build backend could throw an error while installing psycopg2, please see https://www.psycopg.org/docs/install.html#build-prerequisites

Starting the Test Environment

To spin up all the containers necessary for writing integration tests and keep them running:

make py-test-setup

Under the hood this runs, from tests/:

uv run pytest --basetemp=./tmp/ -vv --reuse integration/bootstrap/setup.py::test_setup

This command will:

Start all required services (Datastore, PostgreSQL, Zookeeper, O11y, Zeus mock, gateway mock)
Register an admin user
Keep containers running via the --reuse flag

Stopping the Test Environment

When you're done writing integration tests, clean up the environment:

make py-test-teardown

Which runs:

uv run pytest --basetemp=./tmp/ -vv --teardown integration/bootstrap/setup.py::test_teardown

This destroys the running integration test setup and cleans up resources.

Understanding the Integration Test Framework

Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. WireMock is used to spin up test doubles of external services (Zeus cloud API, gateway, etc.).

Why Python/pytest? It's expressive, low-boilerplate, and has powerful fixture capabilities that make integration testing straightforward. Extensive libraries for HTTP requests, JSON handling, and data analysis (numpy) make it easier to test APIs and verify data.
Why testcontainers? They let us spin up isolated dependencies that match our production environment without complex setup.
Why WireMock? Well maintained, documented, and extensible.

tests/
├── conftest.py              # pytest_plugins registration
├── pyproject.toml
├── uv.lock
├── fixtures/                # shared fixture library (flat package)
│   ├── __init__.py
│   ├── auth.py              # admin/editor/viewer users, tokens, license
│   ├── datastore.py
│   ├── http.py              # WireMock helpers
│   ├── keycloak.py          # IdP container
│   ├── postgres.py
│   ├── o11y.py            # O11y-backend container
│   ├── sql.py
│   ├── types.py
│   └── ...                  # logs, metrics, traces, alerts, dashboards, ...
├── integration/
│   ├── bootstrap/
│   │   └── setup.py         # test_setup / test_teardown
│   ├── testdata/            # JSON / JSONL / YAML inputs per suite
│   └── tests/               # one directory per feature area
│       ├── alerts/
│       │   ├── 01_*.py      # numbered suite files
│       │   └── conftest.py  # optional suite-local fixtures
│       ├── auditquerier/
│       ├── cloudintegrations/
│       ├── dashboard/
│       ├── passwordauthn/
│       ├── querier/
│       └── ...
└── e2e/                     # Playwright suite (see docs/contributing/e2e.md)

Each test suite follows these principles:

Organization: Suites live under tests/integration/tests/ in self-contained packages. Shared fixtures live in the top-level tests/fixtures/ package so the e2e tree can reuse them.
Execution Order: Files are prefixed with two-digit numbers (01_, 02_, 03_) to ensure sequential execution when tests depend on ordering.
Time Constraints: Each suite should complete in under 10 minutes (setup takes ~4 mins).

Test Suite Design

Test suites should target functional domains or subsystems within O11y. When designing a test suite, consider these principles:

Functional Cohesion: Group tests around a specific capability or service boundary
Data Flow: Follow the path of data through related components
Change Patterns: Components frequently modified together should be tested together

The exact boundaries for suites are intentionally flexible, allowing contributors to define logical groupings based on their domain knowledge. Current suites cover alerts, audit querier, callback authn, cloud integrations, dashboards, ingestion keys, logs pipelines, password authn, preferences, querier, raw export data, roles, root user, service accounts, and TTL.

How to write an integration test?

Now start writing an integration test. Create a new file tests/integration/tests/bootstrap/01_version.py and paste the following:

import requests

from fixtures import types
from fixtures.logger import setup_logger

logger = setup_logger(__name__)


def test_version(o11y: types.O11y) -> None:
    response = requests.get(
        o11y.self.host_configs["8080"].get("/api/v1/version"),
        timeout=2,
    )
    logger.info(response)

We have written a simple test which calls the version endpoint of the O11y backend. To run just this function, run the following command:

cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse \
  integration/tests/bootstrap/01_version.py::test_version

Note: The --reuse flag is used to reuse the environment if it is already running. Always use this flag when writing and running integration tests. Without it the environment is destroyed and recreated every run.

Here's another example of how to write a more comprehensive integration test:

from http import HTTPStatus
import requests
from fixtures import types
from fixtures.logger import setup_logger

logger = setup_logger(__name__)


def test_user_registration(o11y: types.O11y) -> None:
    """Test user registration functionality."""
    response = requests.post(
        o11y.self.host_configs["8080"].get("/api/v1/register"),
        json={
            "name": "testuser",
            "orgId": "",
            "orgName": "test.org",
            "email": "test@example.com",
            "password": "password123Z$",
        },
        timeout=2,
    )

    assert response.status_code == HTTPStatus.OK
    assert response.json()["setupCompleted"] is True

Test inputs (JSON fixtures, expected payloads) go under tests/integration/testdata/<suite>/ and are loaded via fixtures.fs.get_testdata_file_path.

How to run integration tests?

Running All Tests

make py-test