Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ relevant section below for more details.
- `INFLUXDB_TOKEN`: If you want to send API events to InfluxDB, specify this write token.
- `INFLUXDB_URL`: The URL for your InfluxDB database.
- `INFLUXDB_ORG`: The organisation string for your InfluxDB API call.
- `EXPERIMENTATION_ENABLED`: Set to `true` once the experimentation infra is deployed. Experiments and warehouse
connections stay unavailable until it is. Defaults to `false`.
- `EXPERIMENTATION_ENABLED`: Set to `true` to turn on experiments and warehouse connections. Defaults to `false`. See
[Self-Hosting and Open Source](/experimentation/self-hosting).
- `WAREHOUSE_CREDENTIALS_SECRET`: Secret used to encrypt warehouse connection credentials. Warehouse delivery must use
the same value. Changing it makes saved credentials unreadable.
- `GA_TABLE_ID`: GA table ID (view) to query when looking for organisation usage.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,8 @@ The task processor itself is documented [here](/deployment-self-hosting/scaling-
### Experimentation

The chart can run [experimentation](/experimentation). You bring your own Kafka and data warehouse, and each
environment's events are delivered to the warehouse connected to it. The chart then:
environment's events are delivered to the warehouse connected to it. To evaluate experimentation without deploying
this, see [Self-Hosting and Open Source](/experimentation/self-hosting). The chart then:

- Runs the event ingestion service, which receives SDK events and writes them to a Kafka topic.
- Runs warehouse delivery, which reads that topic and inserts each environment's events into its warehouse.
Expand Down
38 changes: 20 additions & 18 deletions docs/docs/experimentation/connect-a-warehouse.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
title: Connect a Warehouse
sidebar_label: Connect a Warehouse
sidebar_position: 2
description:
Choose where experiment events are stored, the managed Flagsmith warehouse or your own ClickHouse or Databricks.
description: Choose where experiment events are stored, from the managed Flagsmith warehouse to your own database.
---

import CodeBlock from '@theme/CodeBlock'; import eventsSql from '../../../experimentation/clickhouse/events.sql?raw';
Expand All @@ -19,15 +18,14 @@ warehouse connection, and it must be in place before you can create experiments.

:::note Self-hosted

The managed Flagsmith warehouse is only available on Flagsmith Cloud. When self-hosting, connect your own warehouse as
described below, and deploy event ingestion and warehouse delivery, for example with the
[Helm chart](/deployment-self-hosting/hosting-guides/kubernetes-openshift#experimentation).
The managed Flagsmith warehouse is only available on Flagsmith Cloud. When self-hosting, see
[Self-Hosting and Open Source](/experimentation/self-hosting).

:::

:::note Plans

On Flagsmith Cloud, connecting your own ClickHouse or Databricks requires a
On Flagsmith Cloud, connecting your own ClickHouse, Databricks or PostgreSQL requires a
Comment thread
Zaimwa9 marked this conversation as resolved.
Resolved
[Scale-Up or Enterprise subscription](https://www.flagsmith.com/pricing). Self-hosted Flagsmith has no plan restriction.

:::
Expand All @@ -36,7 +34,7 @@ On Flagsmith Cloud, connecting your own ClickHouse or Databricks requires a

The **Flagsmith** warehouse is managed and hosted by Flagsmith. There is nothing to provision or configure.

1. Go to **Environment Settings > Warehouse**.
1. Go to **Environment Settings › Warehouse**.
2. Select **Flagsmith** and click **Enable**, then confirm.

![The Warehouse tab with the Flagsmith managed warehouse selected](/img/experimentation/warehouse-tab.png)
Expand All @@ -56,17 +54,16 @@ Warehouse tab open.
Once connected, the warehouse card shows the total number of events received and the number of unique event names, which
is useful to verify that instrumentation data is arriving.

<!-- Screenshot: connected warehouse card with event stats -->

### Connection statuses

The current status is shown on the warehouse card in **Environment Settings > Warehouse**.
The current status is shown on the warehouse card in **Environment Settings › Warehouse**.

- **Created**: the connection exists but no events have been received yet.
- **Pending Connection**: waiting for the first event to arrive.
- **Connected**: events have been received; you are ready to run experiments.
- **Errored**: the card shows the reason. For Databricks, see
[Troubleshooting](/experimentation/connect-databricks#troubleshooting); otherwise [contact support](/support/).
- **Errored**: the card shows the reason. See Troubleshooting for
[Databricks](/experimentation/connect-databricks#troubleshooting) or
[PostgreSQL](/experimentation/connect-postgresql#troubleshooting); otherwise [contact support](/support/).

## Bring your own ClickHouse

Expand All @@ -75,15 +72,15 @@ be reachable from Flagsmith: on Flagsmith Cloud, over the internet; when self-ho
warehouse delivery.

1. Configure your instance by running the following against it, using an administrative user (the statements require
`CREATE USER`, `GRANT`, `CREATE DATABASE` and `CREATE TABLE` privileges). Replace `<USER>` and `<CHANGE_ME_PASSWORD>`
with your own values. This is a one-off setup step; the same SQL is available to copy from the Flagsmith dashboard.
`CREATE USER`, `GRANT`, `CREATE DATABASE` and `CREATE TABLE` privileges). Replace `<USER>` and `<YOUR_PASSWORD>` with
your own values. This is a one-off setup step; the same SQL is available to copy from the Flagsmith dashboard.

<CodeBlock language="sql">{eventsSql}</CodeBlock>

```sql
-- Create a dedicated user for Flagsmith
CREATE USER IF NOT EXISTS <USER>
IDENTIFIED WITH sha256_password BY '<CHANGE_ME_PASSWORD>';
IDENTIFIED WITH sha256_password BY '<YOUR_PASSWORD>';

-- Allow it to write and read experiment events
GRANT SELECT, INSERT ON flagsmith_exp.events TO <USER>;
Expand All @@ -92,12 +89,12 @@ warehouse delivery.
GRANT SHOW TABLES, SHOW COLUMNS ON flagsmith_exp.* TO <USER>;
```

2. Go to **Environment Settings > Warehouse**, select **ClickHouse** and enter your connection details: host, HTTPS port
2. Go to **Environment Settings › Warehouse**, select **ClickHouse** and enter your connection details: host, HTTPS port
(8443 by default), database (`flagsmith_exp` by default), and the username and password of the user you created in
step 1.
3. Click **Test connection**. Once the test succeeds, save the connection.
4. Click **Send your first event** on the connection card to verify events are arriving in your warehouse; the button
disappears once the first event is received.
4. On Flagsmith Cloud, click **Send your first event** on the connection card to verify events are arriving in your
warehouse; the button disappears once the first event is received.

Connections use the ClickHouse HTTP(S) interface, with TLS enabled by default.

Expand All @@ -108,6 +105,11 @@ Experiment results and exposures are computed from your ClickHouse instance.
You can also store experiment events in your own **Databricks** workspace, and Flagsmith computes results there. See
[Connect Databricks](/experimentation/connect-databricks) for setup and troubleshooting.

## Bring your own PostgreSQL

You can store experiment events in your own **PostgreSQL** database, or a PostgreSQL-compatible service, and Flagsmith
computes results there. See [Connect PostgreSQL](/experimentation/connect-postgresql) for setup and troubleshooting.

## Coming soon

Bring your own warehouse: connections for **Snowflake** and **BigQuery**.
4 changes: 2 additions & 2 deletions docs/docs/experimentation/connect-databricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ storage works; you don't need an external location.

If the test fails, you can save anyway and test again later, but events won't be delivered until it succeeds.

5. The connection shows **Connected** once the test succeeds. To verify events reach your table, click **Send your first
event** on the connection card; the button disappears once the first event arrives.
5. The connection shows **Connected** once the test succeeds. On Flagsmith Cloud, to verify events reach your table,
click **Send your first event** on the connection card; the button disappears once the first event arrives.

![A connected Databricks warehouse card showing the last event received, total events and unique event names](/img/experimentation/warehouse-databricks-connected.png)

Expand Down
129 changes: 129 additions & 0 deletions docs/docs/experimentation/connect-postgresql.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
---
title: Connect PostgreSQL
sidebar_label: Connect PostgreSQL
sidebar_position: 2.7
description: Store and query experiment events in your own PostgreSQL database.
---

:::info Beta

Experimentation is in beta.

:::

Flagsmith writes experiment events to a table in your PostgreSQL database and computes experiment results by querying it
there. Each environment has one [warehouse connection](/experimentation/connect-a-warehouse).

Results queries slow down as the events table grows. See [Performance and table growth](#performance-and-table-growth).

## Before you start

- On Flagsmith Cloud, a [Scale-Up or Enterprise subscription](https://www.flagsmith.com/pricing).
- A database reachable from Flagsmith: on Flagsmith Cloud, over the internet; when self-hosting, from your Flagsmith API
and warehouse delivery. On Flagsmith Cloud, servers restricted by IP allowlists aren't currently supported.
- A superuser, or a role with `CREATEROLE` and `CREATE` on the database, to run the setup SQL.
- When self-hosting, the event ingestion service, Kafka and warehouse delivery. See
[Self-Hosting and Open Source](/experimentation/self-hosting).

PostgreSQL-compatible services such as Amazon RDS and Aurora, Cloud SQL, AlloyDB, Supabase and Neon also work.

## 1. Create the events table and user

Run this once against your database. Replace `<USER>` and `<YOUR_PASSWORD>` with your own values. The same SQL is
available to copy from the Flagsmith dashboard.

```sql
-- Create the schema and events table
CREATE SCHEMA IF NOT EXISTS flagsmith_exp;
CREATE TABLE IF NOT EXISTS flagsmith_exp.events (
environment_key TEXT NOT NULL,
event TEXT NOT NULL,
feature_name TEXT,
timestamp TIMESTAMPTZ NOT NULL,
collected_at TIMESTAMPTZ,
identifier TEXT NOT NULL,
value TEXT,
traits TEXT,
metadata TEXT,
sdk_language TEXT,
sdk_version TEXT
);
CREATE INDEX IF NOT EXISTS events_env_event_feature_ts ON flagsmith_exp.events (environment_key, event, feature_name, timestamp);
CREATE INDEX IF NOT EXISTS events_env_event_identifier_ts ON flagsmith_exp.events (environment_key, event, identifier, timestamp);

-- Create a dedicated user for Flagsmith
CREATE USER <USER> PASSWORD '<YOUR_PASSWORD>';

-- Allow it to read and write experiment events
GRANT USAGE ON SCHEMA flagsmith_exp TO <USER>;
GRANT SELECT, INSERT ON flagsmith_exp.events TO <USER>;
Comment thread
Zaimwa9 marked this conversation as resolved.
Resolved
```

To use a schema other than `flagsmith_exp`, use the same name in the script and in Flagsmith. Schema names may contain
only lowercase letters, numbers and underscores, and can't start with a number.

Flagsmith doesn't create or alter the table, so keep exactly these columns and types.

## 2. Connect in Flagsmith

1. Go to **Environment Settings › Warehouse** and select **PostgreSQL**.
2. Fill in the connection details:
- **Host**: a hostname or IP address only, without a scheme, path or port.
- **Name this warehouse**: any name, e.g. `Production PostgreSQL`.
- **Port**: `5432` by default.
- **Database**: the database you ran the script in.
- **Schema**: `flagsmith_exp`, unless you changed it.
- **Username** and **Password**: the user you created in step 1. Usernames are lowercase unless you quoted the name
in `CREATE USER`.
- **Secure connection (TLS)**: on by default. It encrypts the connection but doesn't verify the server's certificate.
You can only turn it off when self-hosting.
3. Click **Test connection**. Once it succeeds, click **Save and continue**. The test checks that the user can use the
schema, and that the events table exists, has the expected columns and allows `SELECT` and `INSERT`. If the test
can't connect, you can save anyway and test again later, but events won't be delivered until it succeeds. If
Flagsmith rejects a field, such as the host or schema, fix it before saving.
4. On Flagsmith Cloud, to verify events reach your table, click **Send your first event** on the connection card; the
button disappears once the first event arrives.

![The PostgreSQL connection form with host, name, port, database, schema, username, password and TLS fields](/img/experimentation/warehouse-postgres-form.png)

![A connected PostgreSQL warehouse card showing the last event received, total events, unique event names and the database setup SQL](/img/experimentation/warehouse-postgres-connected.png)

## Manage the connection

### Change the password

Change the user's password on your server first. Then edit the connection, enter the new password, click **Test
connection**, then click **Save changes**. Leaving the password blank keeps the current one.

### Change server

The host can't be changed. To use a different PostgreSQL database, disconnect and create a new connection.

## Performance and table growth

Results queries scan the events table, so they slow down as it grows. If results become too slow, use
[ClickHouse](/experimentation/connect-a-warehouse#bring-your-own-clickhouse) or
[Databricks](/experimentation/connect-databricks), which are built for this kind of query.

Flagsmith doesn't delete old events or partition the table. Monitor its size and manage retention yourself.

## Troubleshooting

Errors show under a form field, below **Test connection**, or on the connection card with status **Errored** when event
delivery fails.

| Message | Where | What to do |
| ---------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Enter a hostname or IP address, without a scheme or port. | Form | Remove any scheme (`postgresql://`), path, user or port from **Host**. |
| Port must be a number between 1 and 65535. | Form | Fix **Port**. |
| Use lowercase letters, numbers and underscores, not starting with a number. | Form | Fix **Schema**, and use the same name in the setup SQL. |
| Host must not target internal or private network addresses. | Test, card | On Flagsmith Cloud, use a host with a public address. |
| Authentication failed. | Test, card | Check **Username** and **Password**, and that your server's `pg_hba.conf` or firewall rules allow the connection. |
| Database does not exist. | Test, card | Check **Database**. |
| Events table not found in the configured database. Run the setup SQL to create it. | Test, card | Run the script in [step 1](#1-create-the-events-table-and-user) with the same schema as the connection. |
| Permission denied on the events table. The user needs SELECT and INSERT on it. | Test, card | Run the `GRANT` statements in step 1 again for the connection's user. |
| The events table's columns don't match the setup SQL. | Test, card | Make the table's columns and types match step 1 exactly. |
| The query timed out. | Test, card | Check the server's load. If the events table is large, see [Performance and table growth](#performance-and-table-growth). |
| Could not connect to the host. | Test, card | Check **Host** and **Port**, that the server accepts connections from Flagsmith, that the user has `CONNECT` on the database and, with **Secure connection (TLS)** on, that it supports SSL. |
| The PostgreSQL server rejected the request. | Test, card | Check the server's logs for the rejected connection or query. |
| Connection failed. | Test, card | [Contact support](/support/). |
6 changes: 4 additions & 2 deletions docs/docs/experimentation/run-an-experiment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,8 @@ Enable events when you create the client, with the environment key you already u
client-side key and server-side keys.

When self-hosting, also point the SDK at your Flagsmith API and your event ingestion service, as in the commented lines
below. Otherwise the SDK sends events to Flagsmith Cloud. To deploy the event ingestion service, see
[Experimentation in the Helm chart](/deployment-self-hosting/hosting-guides/kubernetes-openshift#experimentation).
below. Otherwise the SDK sends events to Flagsmith Cloud. If you use the Flagsmith database, set the events URL to your
Flagsmith API URL ending in `/api/` instead. See [Self-Hosting and Open Source](/experimentation/self-hosting).

The SDKs buffer events and send them in batches, by default every 10 seconds. Send what is left before your application
exits, as shown in each tab.
Expand Down Expand Up @@ -599,6 +599,8 @@ The SDKs skip events silently rather than throw. If the **Exposures** panel stay
- The identity is inside the experiment's rollout. Identities outside it are not enrolled and record no exposure.
- Buffered events were sent before your application exited. See [Initialise](#initialise).
- Event names match your metrics exactly, including case.
- Event timestamps are within the accepted window. Events older than 10 days or more than 24 hours in the future are
rejected, and a batch over 2 MiB is rejected whole.

To see why an event was skipped, turn on your SDK's debug logging.

Expand Down
Loading
Loading