Skip to content

[LFXV2-2254] feat(lfx_one/scripts): add rename-project-slug Go migration script #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andrest50 wants to merge 10 commits into
base: main
Choose a base branch
from
+1,008 −1
Open

[LFXV2-2254] feat(lfx_one/scripts): add rename-project-slug Go migration script #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
20b0d67
feat(lfx_one/scripts): add rename-project-slug Go migration script (L…
andrest50 Jun 17, 2026
69dea8e
chore(rename-project-slug): remove unused opensearch flags
andrest50 Jun 18, 2026
a615444
chore(rename-project-slug): add pre-built binaries for darwin and linux
andrest50 Jun 18, 2026
82a8005
chore(rename-project-slug): log progress every 1000 records instead o…
andrest50 Jun 18, 2026
95593b6
fix(rename-project-slug): bump golang.org/x/crypto to v0.53.0
andrest50 Jun 18, 2026
69ebe91
fix(review): address PR #44 review feedback
andrest50 Jun 18, 2026
86b3e25
docs(rename-project-slug): replace real project slugs with placeholde…
andrest50 Jun 18, 2026
f603bcf
fix(review): address coderabbitai review comments on PR #44
andrest50 Jun 18, 2026
39f9f13
fix(review): address PR #44 review feedback (iteration 3)
andrest50 Jun 18, 2026
225da11
fix(review): address PR #44 review feedback (iteration 4)
andrest50 Jun 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
.claude/
.vscode/
node_modules/

lfx_one/scripts/*/bin/*.test
165 changes: 165 additions & 0 deletions lfx_one/scripts/rename-project-slug/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# rename-project-slug

A standalone Go migration script that renames a project slug across the two LFX V2
data stores that do not self-heal during a Core Services slug migration:

- **OpenSearch** — the `resources` index (dry-run audits the match count, apply
runs a painless `_update_by_query` that rewrites every slug occurrence).
- **NATS JetStream KV** — the source-of-truth buckets for V2 services (dry-run
logs what would change, apply writes each updated record with optimistic-lock
retries).

## Prerequisites

Network access (direct or via `kubectl port-forward`) to OpenSearch and NATS.

## Running the pre-built binary

Pre-built binaries are included in `bin/` — no Go toolchain required:

```bash
# macOS Apple Silicon (M1/M2/M3)
./bin/rename-project-slug-darwin-arm64 <old-slug> <new-slug> [flags]

# macOS Intel
./bin/rename-project-slug-darwin-amd64 <old-slug> <new-slug> [flags]

# Linux x86-64
./bin/rename-project-slug-linux-amd64 <old-slug> <new-slug> [flags]
```

## Building from source

Requires Go 1.25+.

```bash
go build -o bin/rename-project-slug .
```

Or run directly without building:

```bash
go run . <old-slug> <new-slug> [flags]
```

## Usage

```text
go run . <old-slug> <new-slug> [flags]
```

### Required arguments

| Argument | Description |
|----------|-------------|
| `<old-slug>` | The current project slug to replace |
| `<new-slug>` | The new project slug to set |

Alternatively, use `--old-slug` / `--new-slug` flags.

### Flags

| Flag | Default | Description |
|------|---------|-------------|
| `--target` | `both` | Stores to migrate: `opensearch`, `nats`, or `both` |
| `--dry-run` | `true` | Preview changes without writing; pass `--dry-run=false` to apply |
| `--debug` | `false` | Enable debug-level structured logging |

#### OpenSearch flags

| Flag / Env var | Default | Description |
|----------------|---------|-------------|
| `--opensearch-url` / `OPENSEARCH_URL` | `http://localhost:9200` | OpenSearch base URL |

#### NATS flags

| Flag / Env var | Default | Description |
|----------------|---------|-------------|
| `--nats-url` / `NATS_URL` | `nats://localhost:4222` | NATS server URL |
| `--nats-buckets` | see below | Comma-separated KV bucket names to migrate |
| `--concurrency` | `50` | Max concurrent record updates per bucket |

Default NATS buckets: `committee-members,committees,committee-settings,projects,project-settings`

> **Note on the `projects` bucket:** the real project-service is not always locally
> available. Before running `--dry-run=false` with this bucket, verify the JSON slug
> field name in the project-service. If the bucket does not exist the script logs a
> warning and continues.

## Examples

```bash
# Audit (dry-run, both stores) — safe to run any time
go run . old-slug new-slug

# Audit OpenSearch only
OPENSEARCH_URL=http://localhost:9200 go run . old-slug new-slug --target=opensearch

# Apply OpenSearch changes
OPENSEARCH_URL=http://localhost:9200 \
go run . old-slug new-slug --target=opensearch --dry-run=false

# Apply NATS KV changes
NATS_URL=nats://localhost:4222 \
go run . old-slug new-slug --target=nats --dry-run=false --concurrency=20

# Apply both stores
OPENSEARCH_URL=http://localhost:9200 \
NATS_URL=nats://localhost:4222 \
go run . old-slug new-slug --dry-run=false

# Restrict NATS migration to specific buckets
go run . old-slug new-slug \
--target=nats --dry-run=false \
--nats-buckets=committee-members,committees
```

## Port-forwarding (typical staging workflow)

```bash
# Terminal 1 — OpenSearch
kubectl port-forward -n lfx-v2 svc/opensearch 9200:9200

# Terminal 2 — NATS
kubectl port-forward -n lfx-v2 svc/nats 4222:4222

# Terminal 3 — run the script
OPENSEARCH_URL=http://localhost:9200 \
NATS_URL=nats://localhost:4222 \
go run . <old-slug> <new-slug> --dry-run=false
```

## What the OpenSearch migration rewrites

For every document in the `resources` index that matches on any of these fields:

- `data.project_slug` equals `<old-slug>`
- `data.slug` equals `<old-slug>`
- `tags` contains `project_slug:<old-slug>`
- `object_ref` equals `project:<old-slug>`
- `parent_refs` contains `project:<old-slug>`

The painless script rewrites all matched fields to `<new-slug>` and marks the
document as a noop if nothing changed (`conflicts: proceed`).

## What the NATS KV migration rewrites

Per-bucket field mapping (which JSON field holds the project slug):

| Bucket | Field rewritten |
|--------|----------------|
| `committee-members` | `project_slug` |
| `committees` | `project_slug` |
| `committee-settings` | `project_slug` |
| `projects` | `slug` |
| `project-settings` | `project_slug` |
| _(any other bucket)_ | `project_slug` (default) |

Index/alias keys (prefixed `lookup/` or `slug/`) are skipped. Each record update
uses an optimistic-lock retry (3 attempts with backoff).

## Running tests

```bash
go test ./...
```
Comment thread
andrest50 marked this conversation as resolved.
Binary file added lfx_one/scripts/rename-project-slug/bin/rename-project-slug-darwin-amd64
View file
Open in desktop
Binary file not shown.
Binary file added lfx_one/scripts/rename-project-slug/bin/rename-project-slug-darwin-arm64
View file
Open in desktop
Binary file not shown.
Binary file added lfx_one/scripts/rename-project-slug/bin/rename-project-slug-linux-amd64
View file
Open in desktop
Binary file not shown.
20 changes: 20 additions & 0 deletions lfx_one/scripts/rename-project-slug/go.mod
View file
Open in desktop
Comment thread
andrest50 marked this conversation as resolved.
Comment thread
andrest50 marked this conversation as resolved.
Comment thread
andrest50 marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
// Copyright The Linux Foundation and each contributor to LFX.
// SPDX-License-Identifier: MIT

module github.com/linuxfoundation/lfx-engineering/lfx_one/scripts/rename-project-slug

go 1.25.0
Comment thread
andrest50 marked this conversation as resolved.

require (
github.com/nats-io/nats.go v1.43.0
github.com/opensearch-project/opensearch-go/v2 v2.3.0
golang.org/x/sync v0.20.0
)

require (
github.com/klauspost/compress v1.18.0 // indirect
github.com/nats-io/nkeys v0.4.11 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.53.0 // indirect
golang.org/x/sys v0.46.0 // indirect
)
81 changes: 81 additions & 0 deletions lfx_one/scripts/rename-project-slug/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
github.com/aws/aws-sdk-go v1.44.263/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI=
github.com/aws/aws-sdk-go-v2 v1.18.0/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw=
github.com/aws/aws-sdk-go-v2/config v1.18.25/go.mod h1:dZnYpD5wTW/dQF0rRNLVypB396zWCcPiBIvdvSWHEg4=
github.com/aws/aws-sdk-go-v2/credentials v1.13.24/go.mod h1:jYPYi99wUOPIFi0rhiOvXeSEReVOzBqFNOX5bXYoG2o=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.3/go.mod h1:4Q0UFP0YJf0NrsEuEYHpM9fTSEVnD16Z3uyEF7J9JGM=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.33/go.mod h1:7i0PF1ME/2eUPFcjkVIwq+DOygHEoK92t5cDqNgYbIw=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.27/go.mod h1:UrHnn3QV/d0pBZ6QBAEQcqFLf8FAzLmoUfPVIueOvoM=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.34/go.mod h1:Etz2dj6UHYuw+Xw830KfzCfWGMzqvUTCjUj5b76GVDc=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.27/go.mod h1:EOwBD4J4S5qYszS5/3DpkejfuK+Z5/1uzICfPaZLtqw=
github.com/aws/aws-sdk-go-v2/service/sso v1.12.10/go.mod h1:ouy2P4z6sJN70fR3ka3wD3Ro3KezSxU6eKGQI2+2fjI=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.10/go.mod h1:AFvkxc8xfBe8XA+5St5XIHHrQQtkxqrRincx4hmMHOk=
github.com/aws/aws-sdk-go-v2/service/sts v1.19.0/go.mod h1:BgQOMsg8av8jset59jelyPW7NoZcZXLVpDsXunGDrk8=
github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/nats-io/nats.go v1.43.0 h1:uRFZ2FEoRvP64+UUhaTokyS18XBCR/xM2vQZKO4i8ug=
github.com/nats-io/nats.go v1.43.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=
github.com/nats-io/nkeys v0.4.11 h1:q44qGV008kYd9W1b1nEBkNzvnWxtRSQ7A8BoqRrcfa0=
github.com/nats-io/nkeys v0.4.11/go.mod h1:szDimtgmfOi9n25JpfIdGw12tZFYXqhGxjhVxsatHVE=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/opensearch-project/opensearch-go/v2 v2.3.0 h1:nQIEMr+A92CkhHrZgUhcfsrZjibvB3APXf2a1VwCmMQ=
github.com/opensearch-project/opensearch-go/v2 v2.3.0/go.mod h1:8LDr9FCgUTVoT+5ESjc2+iaZuldqE+23Iq0r1XeNue8=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
Loading
Loading