Repository navigation
docs(experimentation): document PostgreSQL warehouses #8873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
771bbd9
docs(experimentation): document PostgreSQL warehouses
Zaimwa9 21d9624
Merge remote-tracking branch 'origin/main' into docs/postgres-warehouse
Zaimwa9 7550a40
fix(experimentation): require CREATE on the database for the PostgreS…
Zaimwa9 2e8dbd5
fix(experimentation): rename the setup SQL password placeholder to YO…
Zaimwa9 5b6923c
docs(experimentation): limit Send your first event to Flagsmith Cloud…
Zaimwa9 b89effd
docs(experimentation): add PostgreSQL warehouse screenshots
Zaimwa9 6bd0b3d
docs(experimentation): remove the connected warehouse card screenshot…
Zaimwa9 663b145
docs(experimentation): fold the Flagsmith database into Connect a War…
Zaimwa9 535d89e
docs(experimentation): trim the Flagsmith database snippet and duplic…
Zaimwa9 ff1ae87
docs(experimentation): move the Flagsmith database how-to to the self…
Zaimwa9 6cc79f8
docs(experimentation): point the Helm guide to self-hosting for evalu…
Zaimwa9 49ddaed
docs(experimentation): address PostgreSQL warehouse review comments
Zaimwa9 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| --- | ||
| title: Connect PostgreSQL | ||
| sidebar_label: Connect PostgreSQL | ||
| sidebar_position: 2.7 | ||
| description: Store and query experiment events in your own PostgreSQL database. | ||
| --- | ||
|
|
||
| :::info Beta | ||
|
|
||
| Experimentation is in beta. | ||
|
|
||
| ::: | ||
|
|
||
| Flagsmith writes experiment events to a table in your PostgreSQL database and computes experiment results by querying it | ||
| there. Each environment has one [warehouse connection](/experimentation/connect-a-warehouse). | ||
|
|
||
| Results queries slow down as the events table grows. See [Performance and table growth](#performance-and-table-growth). | ||
|
|
||
| ## Before you start | ||
|
|
||
| - On Flagsmith Cloud, a [Scale-Up or Enterprise subscription](https://www.flagsmith.com/pricing). | ||
| - A database reachable from Flagsmith: on Flagsmith Cloud, over the internet; when self-hosting, from your Flagsmith API | ||
| and warehouse delivery. On Flagsmith Cloud, servers restricted by IP allowlists aren't currently supported. | ||
| - A superuser, or a role with `CREATEROLE` and `CREATE` on the database, to run the setup SQL. | ||
| - When self-hosting, the event ingestion service, Kafka and warehouse delivery. See | ||
| [Self-Hosting and Open Source](/experimentation/self-hosting). | ||
|
|
||
| PostgreSQL-compatible services such as Amazon RDS and Aurora, Cloud SQL, AlloyDB, Supabase and Neon also work. | ||
|
|
||
| ## 1. Create the events table and user | ||
|
|
||
| Run this once against your database. Replace `<USER>` and `<YOUR_PASSWORD>` with your own values. The same SQL is | ||
| available to copy from the Flagsmith dashboard. | ||
|
|
||
| ```sql | ||
| -- Create the schema and events table | ||
| CREATE SCHEMA IF NOT EXISTS flagsmith_exp; | ||
| CREATE TABLE IF NOT EXISTS flagsmith_exp.events ( | ||
| environment_key TEXT NOT NULL, | ||
| event TEXT NOT NULL, | ||
| feature_name TEXT, | ||
| timestamp TIMESTAMPTZ NOT NULL, | ||
| collected_at TIMESTAMPTZ, | ||
| identifier TEXT NOT NULL, | ||
| value TEXT, | ||
| traits TEXT, | ||
| metadata TEXT, | ||
| sdk_language TEXT, | ||
| sdk_version TEXT | ||
| ); | ||
| CREATE INDEX IF NOT EXISTS events_env_event_feature_ts ON flagsmith_exp.events (environment_key, event, feature_name, timestamp); | ||
| CREATE INDEX IF NOT EXISTS events_env_event_identifier_ts ON flagsmith_exp.events (environment_key, event, identifier, timestamp); | ||
|
|
||
| -- Create a dedicated user for Flagsmith | ||
| CREATE USER <USER> PASSWORD '<YOUR_PASSWORD>'; | ||
|
|
||
| -- Allow it to read and write experiment events | ||
| GRANT USAGE ON SCHEMA flagsmith_exp TO <USER>; | ||
| GRANT SELECT, INSERT ON flagsmith_exp.events TO <USER>; | ||
|
Zaimwa9 marked this conversation as resolved.
Resolved
|
||
| ``` | ||
|
|
||
| To use a schema other than `flagsmith_exp`, use the same name in the script and in Flagsmith. Schema names may contain | ||
| only lowercase letters, numbers and underscores, and can't start with a number. | ||
|
|
||
| Flagsmith doesn't create or alter the table, so keep exactly these columns and types. | ||
|
|
||
| ## 2. Connect in Flagsmith | ||
|
|
||
| 1. Go to **Environment Settings › Warehouse** and select **PostgreSQL**. | ||
| 2. Fill in the connection details: | ||
| - **Host**: a hostname or IP address only, without a scheme, path or port. | ||
| - **Name this warehouse**: any name, e.g. `Production PostgreSQL`. | ||
| - **Port**: `5432` by default. | ||
| - **Database**: the database you ran the script in. | ||
| - **Schema**: `flagsmith_exp`, unless you changed it. | ||
| - **Username** and **Password**: the user you created in step 1. Usernames are lowercase unless you quoted the name | ||
| in `CREATE USER`. | ||
| - **Secure connection (TLS)**: on by default. It encrypts the connection but doesn't verify the server's certificate. | ||
| You can only turn it off when self-hosting. | ||
| 3. Click **Test connection**. Once it succeeds, click **Save and continue**. The test checks that the user can use the | ||
| schema, and that the events table exists, has the expected columns and allows `SELECT` and `INSERT`. If the test | ||
| can't connect, you can save anyway and test again later, but events won't be delivered until it succeeds. If | ||
| Flagsmith rejects a field, such as the host or schema, fix it before saving. | ||
| 4. On Flagsmith Cloud, to verify events reach your table, click **Send your first event** on the connection card; the | ||
| button disappears once the first event arrives. | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| ## Manage the connection | ||
|
|
||
| ### Change the password | ||
|
|
||
| Change the user's password on your server first. Then edit the connection, enter the new password, click **Test | ||
| connection**, then click **Save changes**. Leaving the password blank keeps the current one. | ||
|
|
||
| ### Change server | ||
|
|
||
| The host can't be changed. To use a different PostgreSQL database, disconnect and create a new connection. | ||
|
|
||
| ## Performance and table growth | ||
|
|
||
| Results queries scan the events table, so they slow down as it grows. If results become too slow, use | ||
| [ClickHouse](/experimentation/connect-a-warehouse#bring-your-own-clickhouse) or | ||
| [Databricks](/experimentation/connect-databricks), which are built for this kind of query. | ||
|
|
||
| Flagsmith doesn't delete old events or partition the table. Monitor its size and manage retention yourself. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| Errors show under a form field, below **Test connection**, or on the connection card with status **Errored** when event | ||
| delivery fails. | ||
|
|
||
| | Message | Where | What to do | | ||
| | ---------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | Enter a hostname or IP address, without a scheme or port. | Form | Remove any scheme (`postgresql://`), path, user or port from **Host**. | | ||
| | Port must be a number between 1 and 65535. | Form | Fix **Port**. | | ||
| | Use lowercase letters, numbers and underscores, not starting with a number. | Form | Fix **Schema**, and use the same name in the setup SQL. | | ||
| | Host must not target internal or private network addresses. | Test, card | On Flagsmith Cloud, use a host with a public address. | | ||
| | Authentication failed. | Test, card | Check **Username** and **Password**, and that your server's `pg_hba.conf` or firewall rules allow the connection. | | ||
| | Database does not exist. | Test, card | Check **Database**. | | ||
| | Events table not found in the configured database. Run the setup SQL to create it. | Test, card | Run the script in [step 1](#1-create-the-events-table-and-user) with the same schema as the connection. | | ||
| | Permission denied on the events table. The user needs SELECT and INSERT on it. | Test, card | Run the `GRANT` statements in step 1 again for the connection's user. | | ||
| | The events table's columns don't match the setup SQL. | Test, card | Make the table's columns and types match step 1 exactly. | | ||
| | The query timed out. | Test, card | Check the server's load. If the events table is large, see [Performance and table growth](#performance-and-table-growth). | | ||
| | Could not connect to the host. | Test, card | Check **Host** and **Port**, that the server accepts connections from Flagsmith, that the user has `CONNECT` on the database and, with **Secure connection (TLS)** on, that it supports SSL. | | ||
| | The PostgreSQL server rejected the request. | Test, card | Check the server's logs for the rejected connection or query. | | ||
| | Connection failed. | Test, card | [Contact support](/support/). | | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.