Compare commits

..
145 Commits
Author SHA1 Message Date
zeekayandClaude Opus 4.8 f90c751b10 refactor(datastore): back the datastore migration driver with native hanzo-ds/go, drop clickhouse-go
The datastore:// migration driver is now a self-contained golang-migrate
database.Driver that opens through github.com/hanzo-ds/go (the native
ClickHouse-wire driver registered as the database/sql name "datastore").
The ClickHouse-specific migration logic (multi-statement exec,
schema_migrations DDL, cluster/engine config, Version/SetVersion/Drop/Lock)
is ported directly into database/datastore, so it no longer embeds or
imports the upstream database/clickhouse driver or
github.com/ClickHouse/clickhouse-go.

- Delete database/clickhouse (driver + test + examples) and
  internal/cli/build_clickhouse.go.
- internal/cli/build_datastore.go drops the clickhouse-go blank import.
- go.mod: drop github.com/ClickHouse/clickhouse-go, add
  github.com/hanzo-ds/go v1.0.1; tidy. Zero clickhouse-go in go.mod,
  go.sum, and the module/dependency graph.
- Port the driver test + example migrations under database/datastore
  (integration test requires a live datastore/clickhouse-wire backend).
- Update README, release workflow build tags, bug report template.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 18:40:56 -07:00
z c7a0a74aef docs(brand): add hero banner 2026-06-28 20:17:58 -07:00
z 50117bac35 chore(brand): dynamic hero banner 2026-06-28 20:17:56 -07:00
d9c67dc417 ci: run on self-hosted ARC pool (hanzo-build-linux-amd64/deploy), not GitHub-hosted (#1)
Co-authored-by: zeekay <z@hanzo.ai>
2026-06-19 20:36:45 -07:00
Antje WorringandClaude Opus 4.8 a5075e2de1 docs: tidy LLM.md indexes; CLAUDE.md -> LLM.md symlink convention
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:05:12 -07:00
Hanzo AI 9477979a55 deps: bump Go to 1.26.4 across go.mod, Dockerfiles, GH Actions
Workspace-wide sync. luxfi/node already shipped on 1.26.4 in v1.30.6
(commit 121aca1fa9); this is the cross-repo catch-up.
2026-06-07 12:10:59 -07:00
Hanzo AI f4abb4bba7 Merge remote-tracking branch 'upstream/master' 2026-05-19 21:54:56 -07:00
Hanzo AI 3461b6bfd7 go.mod: bump go directive to 1.26.3 (security advisory) 2026-05-12 21:25:26 -07:00
Hanzo AI 602c5edb72 chore: symlink AGENTS.md and CLAUDE.md to LLM.md
Canonical project context lives in LLM.md. Symlinks ensure
agentic coding tools (Claude Code, Cursor, etc.) find context
automatically regardless of which filename they look for.
2026-04-01 14:12:04 -07:00
Hanzo Dev a7da767fa3 chore: bump Go 1.26.1 2026-03-22 14:10:48 -07:00
Hanzo Dev b55e3ea06d docs: add LLM.md project guide 2026-03-11 10:31:22 -07:00
Dale HuiandGitHub 2bd822b3aa Merge pull request #1361 from golang-migrate/dependabot/go_modules/go_modules-6b971a9d7e
Bump go.opentelemetry.io/otel/sdk from 1.36.0 to 1.40.0 in the go_modules group across 1 directory
2026-03-11 00:50:10 -07:00
Zach Kelling 460d57267a fix: tar binary as 'migrate' not 'migrate.linux-amd64' to match upstream 2026-03-03 10:16:42 -08:00
Zach Kelling 1e4b96537d Add release workflow for CI-built binaries
Builds migrate CLI for linux/darwin (amd64/arm64) with datastore
driver support and publishes GitHub Release on tag push.
2026-03-03 09:24:50 -08:00
Zach Kelling 837dda3cc9 Add datastore database driver for Hanzo Datastore
Registers "datastore://" as a URL scheme that delegates to the
ClickHouse driver. This allows golang-migrate to natively connect
to Hanzo Datastore instances without URL rewriting in scripts.
2026-03-03 09:15:21 -08:00
dependabot[bot]andGitHub 1620c56290 Bump go.opentelemetry.io/otel/sdk
Bumps the go_modules group with 1 update in the / directory: [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go).


Updates `go.opentelemetry.io/otel/sdk` from 1.36.0 to 1.40.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.36.0...v1.40.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.40.0
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-01 20:47:32 +00:00
Dale HuiandGitHub 257fa847d6 Merge pull request #1320 from achanda/upgrade-pgxv5
Upgrade pgx v5 to latest version
2025-12-14 15:16:48 -08:00
001b70ac7d ci: update golangci-lint to v2 (#1341)
* ci: update golangci-lint to v2

update method:
```
golangci-lint migrate
golangci-lint run --fix
```

and fix the last ones manually

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix tests

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-14 15:14:46 -08:00
leonklingeleandGitHub 89e308c340 chore: remove dependency on "hashicorp/go-multierror" (#1322)
* feat: remove dependency on "hashicorp/go-multierror"

It seems the project has been unmainted for quite some time already,
see for example https://github.com/hashicorp/go-multierror/issues/97
and https://github.com/hashicorp/go-multierror/issues/98.
This also removes an uneccessary dependency which slims down the go.mod.

Go as of version 1.20 supports appending errors by specifying
multiple "%w" formatters in the "fmt.Errorf"[^0] function.

[^0]: https://pkg.go.dev/fmt#Errorf

* feat: use errors.Join to combine multiple errors

As an update to the previous commit 59f084ea3be2e7ec9d912c1a311402b1e4c40df6,
use "errors.Join"[^0] instead of multiple "%w" formatters for "fmt.Errorf".

[^0]: https://pkg.go.dev/errors#Join

* chore: bring back unused util.go file as removing it is a breaking change
2025-11-29 11:37:42 -08:00
Abhishek Chanda fed8fe9f0a Upgrade pgx v5 to latest version 2025-11-27 20:13:25 -06:00
Dale HuiandGitHub 472ef2e75c Merge pull request #1336 from golang-migrate/dependabot/go_modules/golang.org/x/crypto-0.45.0
Bump golang.org/x/crypto from 0.43.0 to 0.45.0
2025-11-27 00:56:21 -08:00
dependabot[bot]andGitHub 8d7625987a Bump golang.org/x/crypto from 0.43.0 to 0.45.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.43.0 to 0.45.0.
- [Commits](https://github.com/golang/crypto/compare/v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-27 08:05:11 +00:00
9f9df7c897 chore: Update cloud.google.com/go/spanner version (#1330)
* chore: Update cloud.google.com/go/spanner version

* Run `go mod tidy`

---------

Co-authored-by: Dale Hui <dhui@users.noreply.github.com>
2025-11-27 00:01:15 -08:00
Dale HuiandGitHub a371c8ea2b Merge pull request #1304 from iamonah/fix/clarify-databaseName-meaning
Rename databaseName to databaseDriverName
2025-11-26 22:52:08 -08:00
Dale HuiandGitHub 43cc3b3ea5 Merge pull request #1325 from HaraldNordgren/linter_issues
fix linter issues
2025-11-26 22:38:12 -08:00
Dale HuiandGitHub f939a89925 Merge pull request #1335 from golang-migrate/dependabot/go_modules/golang.org/x/crypto-0.45.0
Bump golang.org/x/crypto from 0.36.0 to 0.45.0
2025-11-26 22:30:24 -08:00
dependabot[bot]andGitHub 6dd86e0ded Bump golang.org/x/crypto from 0.36.0 to 0.45.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.36.0 to 0.45.0.
- [Commits](https://github.com/golang/crypto/compare/v0.36.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-20 01:45:37 +00:00
Dale HuiandGitHub 70e6d6d574 Merge pull request #1333 from matsoob/updateGoInBaseImage
Update docker image to use go 1.25, addressing vulnerabilities
2025-11-18 23:04:52 -08:00
Dale HuiandGitHub a51d0da644 Merge pull request #1334 from golang-migrate/dependabot/go_modules/github.com/dvsekhvalnov/jose2go-1.7.0
Bump github.com/dvsekhvalnov/jose2go from 1.6.0 to 1.7.0
2025-11-18 23:01:27 -08:00
dependabot[bot]andGitHub 3f583f0b61 Bump github.com/dvsekhvalnov/jose2go from 1.6.0 to 1.7.0
Bumps [github.com/dvsekhvalnov/jose2go](https://github.com/dvsekhvalnov/jose2go) from 1.6.0 to 1.7.0.
- [Commits](https://github.com/dvsekhvalnov/jose2go/compare/v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/dvsekhvalnov/jose2go
  dependency-version: 1.7.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-14 21:46:48 +00:00
matsoob ce3cb727b6 Update docker image 2025-11-13 23:35:48 +00:00
Harald Nordgren 1ee8337800 revert GenerateAdvisoryLockID 2025-11-11 09:42:53 +01:00
Harald Nordgren 952d08e30b fix linter issues 2025-11-09 16:54:45 +01:00
Dale HuiandGitHub 39ab04d659 Update goreleaser Go version from 1.24.x to 1.25.x 2025-11-08 21:48:01 -08:00
Dale HuiandGitHub 8b9c5f7712 Merge pull request #1310 from dhui/update_go
Add support for Go 1.25 and drop support for 1.23
2025-08-23 20:27:53 -07:00
Dale Hui b4ec9bccb1 Add support for Go 1.25 and drop support for 1.23 2025-08-23 13:42:01 -07:00
ed4bdd4614 Ensure bufferWriter is always closed in Migration.Buffer and propagate close errors (#1308)
* Ensure bufferWriter is always closed and error is returned in Buffer()

* Ensure bufferWriter is always closed and error is returned in Buffer()

* addressed review comments. used named error and joined error to return final error

* Fixed defer block

---------

Co-authored-by: Chandra kant <chandra.kant@cohesity.com>
2025-08-23 13:12:17 -07:00
Roman SharkovandGitHub 8945e853c4 Merge pull request #1303 from romshark/master
refactor: Remove go.uber.org/atomic in favor of std sync/atomic
2025-08-20 16:35:25 -07:00
Dale HuiandGitHub 7108d806dd Merge pull request #1309 from dhui/dktest_v0.4.6
Update dktest to v0.4.6 for docker vuln fix
2025-08-19 16:24:09 -07:00
Dale HuiandGitHub 682016f04c Merge pull request #1277 from sandhilt/doc/change-apt-key-to-gpg
Update apt-key to gpg
2025-08-19 14:22:16 -07:00
Dale Hui f3e6b5a737 Replace usage of deprecated docker types
This is a backwards incompatible change,
but there don't seem to be any public imports:
https://pkg.go.dev/github.com/golang-migrate/migrate/v4@v4.18.3/testing?tab=importedby
2025-08-19 14:03:55 -07:00
Dale Hui 0a17402aa2 Update dktest to v0.4.6 for docker vuln fix 2025-08-19 13:27:40 -07:00
Dale HuiandGitHub 5eee0c8030 Merge pull request #1299 from golang-migrate/dependabot/go_modules/golang.org/x/oauth2-0.27.0
Bump golang.org/x/oauth2 from 0.18.0 to 0.27.0
2025-08-19 13:15:26 -07:00
0xOnah 30690d7df3 Rename databaseName to databaseDriverName 2025-08-03 21:30:31 +01:00
dependabot[bot]andGitHub 642a24d61b Bump golang.org/x/oauth2 from 0.18.0 to 0.27.0
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-version: 0.27.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-18 18:07:02 +00:00
sandhilt 5300864bba fix(docs): update Linux installation instructions for GPG key handling 2025-05-13 15:05:40 -03:00
Dale HuiandGitHub 278833935c Merge pull request #1186 from urbim/bugfix/sqlserverLockFix
Fixed sqlserver not actually getting a lock if lock is already set
2025-04-25 13:14:27 -07:00
Urban Malc 6b9f24bf7f Fixed sqlserver not actually getting a lock if lock is already set 2025-04-24 12:14:21 +02:00
Dale HuiandGitHub 9023d66a0b Merge pull request #1244 from alexandear-org/chore-redundant-build-tags
Remove redundant build tags
2025-04-17 15:21:39 -07:00
Dale HuiandGitHub 10494908af Merge pull request #1179 from lunfel/master
Update migrate -help output for the readme file
2025-04-17 15:17:59 -07:00
Dale Hui 033835ac1b Update to dktest v0.4.5 2025-04-17 14:58:46 -07:00
Dale HuiandGitHub 329152ed62 Merge pull request #1196 from Rambatino/patch-1
fix: typo limited not limitted
2025-04-17 13:59:24 -07:00
Oleksandr Redko 34c2b4ad9c Remove redundant build tags 2025-04-17 23:57:28 +03:00
Dale HuiandGitHub a3b7633246 Merge pull request #1259 from golang-migrate/dependabot/go_modules/golang.org/x/net-0.38.0
Bump golang.org/x/net from 0.33.0 to 0.38.0
2025-04-17 00:12:46 -07:00
dependabot[bot]andGitHub 5b97c921ed Bump golang.org/x/net from 0.33.0 to 0.38.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.33.0 to 0.38.0.
- [Commits](https://github.com/golang/net/compare/v0.33.0...v0.38.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.38.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-17 02:40:34 +00:00
Dale HuiandGitHub 3c3ce91f29 Merge pull request #1258 from golang-migrate/dependabot/go_modules/golang.org/x/crypto-0.35.0
Bump golang.org/x/crypto from 0.31.0 to 0.35.0
2025-04-16 19:39:14 -07:00
dependabot[bot]andGitHub f37ef795db Bump golang.org/x/crypto from 0.31.0 to 0.35.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.31.0 to 0.35.0.
- [Commits](https://github.com/golang/crypto/compare/v0.31.0...v0.35.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.35.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-16 22:36:30 +00:00
Dale HuiandGitHub 1af841d787 Merge pull request #1260 from dhui/update_go
Update supported Go versions and README
2025-04-16 15:35:09 -07:00
Dale Hui 7269490309 Update golangci-lint version used in GitHub Actions 2025-04-16 15:12:33 -07:00
Dale Hui fccd197e40 Mention CLI install instructions in main README
Fixes:
* https://github.com/golang-migrate/migrate/issues/973
* https://github.com/golang-migrate/migrate/issues/1020
2025-04-16 14:43:17 -07:00
Dale Hui e6d84f6f80 Drop support for Go 1.22 and add support for Go 1.24
Drop `toolchain` module directive to favor default/auto behavior
See also: https://github.com/golang-migrate/migrate/pull/1183
2025-04-16 14:38:52 -07:00
Dale HuiandGitHub 146bfa05b3 Merge pull request #1241 from hazzik/refactor/pkg/errors
refactor: replace github.com/pkg/errors with stdlib
2025-04-16 14:04:34 -07:00
Dale HuiandGitHub 604248c97e Merge pull request #1250 from golang-migrate/dependabot/go_modules/github.com/golang-jwt/jwt/v4-4.5.2
Bump github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2
2025-03-23 16:54:14 -07:00
dependabot[bot]andGitHub a4d0a1b8d9 Bump github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.1 to 4.5.2.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.5.1...v4.5.2)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-21 22:40:35 +00:00
Alex Zaytsev 60d73be14e refactor: replace github.com/pkg/errors with stdlib 2025-03-10 13:54:11 +10:00
Jochen SchalandaandGitHub 36d17ba2ec tests: fix various tests (#1209)
* Remove old versions of MongoDB

Removed:

| Release     | Release Date  | End of Life Date |
|-------------|---------------|------------------|
| MongoDB 3.4 | November 2016 | January 2020     |
| MongoDB 3.6 | November 2017 | April 2021       |
| MongoDB 4.0 | June 2018     | April 2022       |
| MongoDB 4.2 | August 2019   | April 2023       |

Added:

| Release     | Release Date  | End of Life Date |
|-------------|---------------|------------------|
| MongoDB 5.0 | July 2021     | October 2024     |
| MongoDB 6.0 | July 2022     | July 2025        |
| MongoDB 7.0 | August 2023   | August 2026      |
| MongoDB 8.0 | October 2024  | TBD              |

https://www.mongodb.com/legal/support-policy/lifecycles

* test(postgres): run tests with PostgreSQL 17

Add PostgreSQL 17, remove PostgreSQL 12 (EOL since November 21, 2024).

https://www.postgresql.org/support/versioning/

* test(cockroachdb): remove old versions of CockroachDB in tests

Removed:

|       Release       |                Released                |              Maintenance Support             |              Assistance Support              |         Latest        |
|:-------------------:|:--------------------------------------:|:--------------------------------------------:|:--------------------------------------------:|:---------------------:|
| 2.1                 | 6 years ago (30 Oct 2018)              | Ended 5 years ago (30 Oct 2019)              | Ended 4 years and 7 months ago (30 Apr 2020) | 2.1.11 (22 Jan 2020)  |
| 2.0                 | 6 years and 8 months ago (04 Apr 2018) | Ended 5 years and 8 months ago (04 Apr 2019) | Ended 5 years ago (04 Oct 2019)              | 2.0.7 (28 Nov 2018)   |
| 1.1                 | 7 years ago (12 Oct 2017)              | Ended 6 years ago (12 Oct 2018)              | Ended 5 years and 8 months ago (12 Apr 2019) | 1.1.9 (01 Oct 2018)   |
| 1.0                 | 7 years ago (10 May 2017)              | Ended 6 years ago (10 May 2018)              | Ended 6 years ago (10 Nov 2018)              | 1.0.7 (11 Feb 2018)   |

Added:

|       Release       |                Released                |              Maintenance Support             |              Assistance Support              |         Latest        |
|:-------------------:|:--------------------------------------:|:--------------------------------------------:|:--------------------------------------------:|:---------------------:|
| 24.3                | 3 weeks ago (18 Nov 2024)              | Ends in 11 months (18 Nov 2025)              | Ends in 1 year and 5 months (18 May 2026)    | 24.3.0 (20 Nov 2024)  |
| 24.2                | 3 months and 4 weeks ago (12 Aug 2024) | Ends in 2 months (12 Feb 2025)               | Ends in 2 months (12 Feb 2025)               | 24.2.5 (11 Nov 2024)  |
| 24.1 (Upcoming LTS) | 6 months and 3 weeks ago (20 May 2024) | Ends in 5 months (20 May 2025)               | Ends in 11 months (20 Nov 2025)              | 24.1.7 (11 Nov 2024)  |
| 23.2 (Upcoming LTS) | 10 months ago (05 Feb 2024)            | Ends in 1 month and 4 weeks (05 Feb 2025)    | Ends in 7 months and 4 weeks (05 Aug 2025)   | 23.2.16 (13 Nov 2024) |

* test(firebird): remove old versions of Firebird in tests

Removed: Firebird 2.5, see https://firebirdsql.org/en/discontinued-versions/

Added:
- Firebird 4.0 (https://firebirdsql.org/en/firebird-4-0-5/)
- Firebird 5.0 (https://firebirdsql.org/en/firebird-5-0-1/)

* test(clickhouse): run ClickHouse tests with official LTS image (24.8)
2025-02-19 16:19:12 -08:00
Dale HuiandGitHub 24a6476044 Merge pull request #1225 from stensonb/patch-1
Update FAQ.md - typo
2025-02-19 16:06:49 -08:00
Bryan StensonandGitHub a8680333c3 Update FAQ.md - typo 2025-02-01 23:26:02 -08:00
Dale HuiandGitHub d477553dcd Merge pull request #1195 from golang-migrate/dependabot/go_modules/github.com/golang-jwt/jwt/v4-4.5.1
Bump github.com/golang-jwt/jwt/v4 from 4.4.2 to 4.5.1
2025-01-26 17:53:05 -08:00
dependabot[bot]andGitHub e145cded4a Bump github.com/golang-jwt/jwt/v4 from 4.4.2 to 4.5.1
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.4.2 to 4.5.1.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v4.4.2...v4.5.1)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-27 01:06:51 +00:00
Dale Hui 7651c8a37e linter fixes 2025-01-26 17:04:55 -08:00
Dale Hui e5a152b9f6 Drop support for Azure SQL Edge
Will be retired on 9/30/2025 and CI tests using the docker image have been flakey
https://azure.microsoft.com/en-us/updates?id=azure-sql-edge-retirement
2025-01-26 16:43:31 -08:00
Dale HuiandGitHub 12c619ee47 Fix CI (#1222)
* Run go mod tidy

* Fix Github linting action

Changes:
* checkout before setting up go
* pin golangci-lint version

* Update supported db versions

* Use start-single-node instead of start for cockroachdb
2025-01-26 15:46:11 -08:00
Dale Hui bc06922b4d Update dktest from v0.4.3 to v0.4.4 2025-01-25 21:05:33 -08:00
Mark RamotowskiandGitHub 8b091919d4 fix: typo limited not limitted 2024-11-28 09:28:14 +00:00
Mathieu Tanguay c5137c417c Update migrate -help output for the readme file 2024-10-02 10:41:33 -04:00
Dale HuiandGitHub c378583d78 Merge pull request #1162 from rselbach/rselbach/no-sensitive-info
Don't output sensitive information on error
2024-09-23 10:45:56 -07:00
Roberto Selbach e22d0128cc Don't output sensitive information on error 2024-09-12 12:12:34 -04:00
Dale Hui 555501f7cd Update dktest from v0.4.2 to v0.4.3 2024-09-08 14:02:51 -07:00
Dale HuiandGitHub d5eb594426 Merge pull request #1101 from adityassharma-ss/patch-1
CORRECTION DONE near m.Step(2) if you want to explicitly set the number of migrations to run
2024-09-08 11:42:13 -07:00
Dale HuiandGitHub 67c71f9a9d Merge pull request #1152 from golang-migrate/dependabot/go_modules/google.golang.org/grpc-1.64.1
Bump google.golang.org/grpc from 1.64.0 to 1.64.1
2024-09-08 11:34:22 -07:00
Dale HuiandGitHub 1e02b42a7a Merge pull request #1119 from muzammilar/add-noble-numbat
Support .deb package for Noble Numbat (24.04)
2024-09-07 23:07:01 -07:00
dependabot[bot]andGitHub 5b6f623dec Bump google.golang.org/grpc from 1.64.0 to 1.64.1
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.64.0 to 1.64.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.64.0...v1.64.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-08 06:04:50 +00:00
Dale HuiandGitHub ddd7fa61da Merge pull request #1127 from joschi/tests-mysql-9
test(mysql): run tests with MySQL 8.0, 8.4, and 9.0
2024-09-07 22:59:58 -07:00
Dale HuiandGitHub 7e8f6be051 Merge pull request #1126 from joschi/tests-postgres-16
test(postgres): run tests with PostgreSQL 16
2024-09-07 22:59:25 -07:00
Dale HuiandGitHub 65a3bd56cc Merge pull request #1153 from joschi/fix-docker-imports
fix(tests): fix Docker imports
2024-09-07 22:58:32 -07:00
Dale HuiandGitHub e40e64c4ac Merge pull request #1133 from joschi/go-1.23
build: use Go 1.23 to build the project
2024-09-07 22:51:54 -07:00
Jochen Schalanda 6ceb5a9467 fix(tests): fix Docker imports 2024-09-07 22:50:55 +02:00
Jochen Schalanda 0444ec8531 test(postgres): run tests with PostgreSQL 16 2024-09-07 21:47:48 +02:00
Jochen Schalanda f4d986b9d9 test(mysql): run tests with MySQL 8.0, 8.4, and 9.0 2024-09-07 21:47:28 +02:00
Jochen Schalanda ee93fd4aff build: use Go 1.23 to build the project
https://tip.golang.org/doc/go1.23
2024-09-07 21:45:48 +02:00
Dale Hui d28e549a68 Update dktest from v0.4.1 to v0.4.2 2024-09-07 00:01:37 -07:00
Jochen SchalandaandGitHub 897be38496 chore: remove deprecated rand.Seed() in testing.docker (#1149)
* chore: remove deprecated `rand.Seed()` in testing.docker

* Use math/rand/v2

* Remove Go 1.21

Go 1.21 doesn't support `math/rand/v2`.
2024-09-06 23:46:24 -07:00
Dale Hui 1b1c6f6d14 Use newer math/rand/v2 2024-09-06 23:32:35 -07:00
Dale HuiandGitHub f44379d6f1 Merge pull request #1151 from dhui/fix-redshift-tests
Fix redshift tests
2024-09-06 00:41:25 -07:00
Dale Hui e1a7e31b1d Use forked version of postgres 8 to avoid Docker image format version issues.
See: https://docs.docker.com/engine/deprecated/#pushing-and-pulling-with-image-manifest-v2-schema-1

Forked images created by pulling the original image and pushing to the fork. For more details, see: https://github.com/golang-migrate/migrate/pull/1143#issuecomment-2328069381
2024-09-05 23:28:26 -07:00
Dale Hui 899a244399 Fix redshift tests
Newer versions of postgres require a password
2024-09-05 23:26:56 -07:00
Muzammil Abdul Rehman 41e389a078 Support .deb package for Noble Numbat (24.04) 2024-07-17 12:56:34 -07:00
Dale HuiandGitHub 2477f639fd Merge pull request #1098 from gjabell/master
Upgrade go-sqlite3 to v1.14.22
2024-06-10 12:46:13 -07:00
Dale HuiandGitHub 1585b870f8 Merge pull request #1103 from Shion1305/shion/issue-1102
🧑‍💻 improve error message for invalid source, database, resolves: #1102
2024-06-10 12:34:31 -07:00
Dale HuiandGitHub b16220c938 Merge pull request #1105 from Shion1305/shion/issue-1104
#1104 Fix golanglint-ci lint config & lint errors
2024-06-10 12:28:15 -07:00
Shion Ichikawa 34594af0ab 🚨 golangci-lint: fix for errcheck lint errors 2024-06-05 23:40:12 +09:00
Shion Ichikawa 1735cb3920 🚨 golangci-lint: disable interfacer linter 2024-06-05 23:39:14 +09:00
Shion Ichikawa cabb2db836 🧑‍💻 improve error message for invalid source, database, resolves: #1102 2024-06-05 22:44:03 +09:00
Aditya SharmaandGitHub da72428fa0 Update README.md
if you want to explicitly set the number of migrations to run, the README.md needs a correction, there should "Steps()" and not Step()
2024-06-04 19:25:09 +05:30
Galen Abell c81eaf7918 Upgrade go-sqlite3 to v1.14.22 2024-05-31 17:30:06 +02:00
Dale HuiandGitHub 04b36ebaac Merge pull request #1070 from certifaction/mysql-safe-update
Make MySQL SetVersion compatible with sql_safe_update
2024-04-29 21:59:22 -07:00
Dale HuiandGitHub 45e4f1ef2f Merge pull request #1074 from golang-migrate/dependabot/go_modules/golang.org/x/net-0.23.0
Bump golang.org/x/net from 0.21.0 to 0.23.0
2024-04-29 21:55:07 -07:00
dependabot[bot]andGitHub 859d92bd77 Bump golang.org/x/net from 0.21.0 to 0.23.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-19 13:19:09 +00:00
Mateusz Nowotyński e428ecebea Make MySQL SetVersion compatible with sql_safe_update 2024-04-19 09:34:12 +02:00
Dale HuiandGitHub 0c456c49a6 Merge pull request #1068 from goodfirm/master
chore: fix some typos in comments
2024-04-16 23:16:25 -07:00
Dale Hui f100226758 Update dktest from v0.4.0 to v0.4.1 to fix docker vulnerability 2024-04-16 23:03:36 -07:00
Dale HuiandGitHub c52377528f Merge pull request #1055 from golang-migrate/dependabot/go_modules/github.com/jackc/pgx/v5-5.5.4
Bump github.com/jackc/pgx/v5 from 5.3.1 to 5.5.4
2024-04-16 22:57:30 -07:00
dependabot[bot]andGitHub a78d1abfa8 Bump github.com/jackc/pgx/v5 from 5.3.1 to 5.5.4
Bumps [github.com/jackc/pgx/v5](https://github.com/jackc/pgx) from 5.3.1 to 5.5.4.
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.3.1...v5.5.4)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-16 20:43:15 +00:00
Dale HuiandGitHub 837776f0c7 Merge pull request #1054 from golang-migrate/dependabot/go_modules/google.golang.org/protobuf-1.33.0
Bump google.golang.org/protobuf from 1.31.0 to 1.33.0
2024-04-16 13:38:31 -07:00
Dale HuiandGitHub 128b6509cc Merge pull request #1050 from golang-migrate/dependabot/go_modules/github.com/jackc/pgx/v4-4.18.2
Bump github.com/jackc/pgx/v4 from 4.18.1 to 4.18.2
2024-04-16 13:38:07 -07:00
dependabot[bot]andGitHub 2e0872f3e8 Bump google.golang.org/protobuf from 1.31.0 to 1.33.0
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-16 16:19:02 +00:00
dependabot[bot]andGitHub d1df97b2d1 Bump github.com/jackc/pgx/v4 from 4.18.1 to 4.18.2
Bumps [github.com/jackc/pgx/v4](https://github.com/jackc/pgx) from 4.18.1 to 4.18.2.
- [Changelog](https://github.com/jackc/pgx/blob/v4.18.2/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v4.18.1...v4.18.2)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-16 16:18:25 +00:00
Tim RühsenandGitHub 1a002d0e9e Set golangci-lint to 1.54.2 (latest is broken) (#1046)
* Set golangci-lint to 1.54.2 (latest is broken)

* Testing golangci-lint latest (1.57.1)
2024-04-15 23:54:05 -07:00
Dale HuiandGitHub 2c5df8742b Merge pull request #1072 from dhui/dktesting-cleanup
Delete db docker images after tests complete for a few large db docker images
2024-04-15 23:44:37 -07:00
Dale Hui f4950c144e Fallback to dktest.DefaultCleanupTimeout if the dktest.Options doesn't have one specified 2024-04-15 22:57:23 -07:00
Dale Hui b1d02e222f Cleanup sqlserver images after tests run 2024-04-15 22:42:34 -07:00
Dale Hui 49cac86ab2 Cleanup mongodb images after tests run 2024-04-15 00:35:46 -07:00
Dale Hui 06614d9cf5 Cleanup yugabytedb images after tests run 2024-04-15 00:25:40 -07:00
Dale Hui 1b707a76f4 Cleanup cassandra images after tests run 2024-04-15 00:18:13 -07:00
Dale Hui 2884a8e851 Cleanup postgres images after tests run 2024-04-15 00:17:36 -07:00
Dale Hui 4bc6777f19 Add dktesting.Cleanup() method 2024-04-15 00:09:23 -07:00
goodfirm 9d70a397e4 chore: fix some typos in comments
Signed-off-by: goodfirm <fanyishang@yeah.net>
2024-04-10 14:36:41 +08:00
Dale Hui e9133367d7 Drop support for Go 1.20 and add support for Go 1.22 2024-03-24 22:35:06 -07:00
Dale HuiandGitHub 87ba13c398 Merge pull request #1061 from occupyhabit/master
chore: fix some typos
2024-03-24 22:31:39 -07:00
Dale Hui ff8a961068 Update yugabyte test images 2024-03-24 22:30:38 -07:00
occupyhabit 7f85f9cf64 chore: fix some typos
Signed-off-by: occupyhabit <wangmengjiao@outlook.com>
2024-03-23 21:49:28 +08:00
Dale HuiandGitHub a5dc5d1efc Merge pull request #1025 from otoolep/patch-1
rqlite is spelled with all lowercase
2024-01-10 20:56:10 -08:00
Philip O'TooleandGitHub 94b8fa5946 rqlite is spelled with all lowercase
I'm the creator of rqlite, so wanted to fix the title. See https://rqlite.io/
2024-01-05 15:34:45 -05:00
Dale HuiandGitHub 0d41589774 Merge pull request #1022 from golang-migrate/dependabot/go_modules/github.com/dvsekhvalnov/jose2go-1.6.0
Bump github.com/dvsekhvalnov/jose2go from 1.5.0 to 1.6.0
2024-01-02 12:48:02 -08:00
dependabot[bot]andGitHub a860f0cbd4 Bump github.com/dvsekhvalnov/jose2go from 1.5.0 to 1.6.0
Bumps [github.com/dvsekhvalnov/jose2go](https://github.com/dvsekhvalnov/jose2go) from 1.5.0 to 1.6.0.
- [Commits](https://github.com/dvsekhvalnov/jose2go/compare/v1.5...v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/dvsekhvalnov/jose2go
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-31 22:22:36 +00:00
Dale HuiandGitHub 516f038f30 Merge pull request #840 from alex-kuck/fix/839/sqlserver_version_table
[sqlserver] Ensure version table in provided schema
2023-12-31 13:59:38 -08:00
Alex Kucksdorf 0350a00606 [sqlserver] Always access version table with explicit schema 2023-12-25 10:29:27 +01:00
Alex Kucksdorf 814769373f [sqlserver] Ensure version table in provided schema
Closes #839
2023-12-25 10:29:26 +01:00
Dale HuiandGitHub 0815e2d770 Merge pull request #992 from fortnox-andreas/master
Add support for locking table in pgx-driver
2023-12-20 14:13:36 -08:00
Dale Hui 5aa4670cbb Fix GoReleaser deprecations
https://goreleaser.com/deprecations/#-rm-dist
https://goreleaser.com/deprecations/#sourcerlcp
2023-12-20 13:12:07 -08:00
Dale Hui d63a5c2c28 Only test against YugabyteDB LTS releases 2023-12-20 12:49:33 -08:00
Tommy Karlsson 091ad5db99 Quote locktable from config in queries
Defer rollback of transactions
2023-12-20 16:10:29 +01:00
Tommy Karlsson bead4a952b Added documentation and test for lock strategy 2023-09-20 14:27:26 +02:00
Andreas Emilsson 129922aae3 Added support for pgx locking table
In order to support running migrations through PgBouncer which does not
support advisory locks.
2023-09-20 09:59:55 +02:00
108 changed files with 2494 additions and 917 deletions
+1 -1
View File
@@ -25,7 +25,7 @@ e.g. s3, github, go-bindata, gcs, file
Obtained by running: `migrate -help`
**Loaded Database Drivers**
e.g. spanner, stub, clickhouse, cockroachdb, crdb-postgres, postgres, postgresql, pgx, redshift, cassandra, cockroach, mysql
e.g. spanner, stub, datastore, cockroachdb, crdb-postgres, postgres, postgresql, pgx, redshift, cassandra, cockroach, mysql
Obtained by running: `migrate -help`
**Go Version**
+9
View File
@@ -0,0 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="640" viewBox="0 0 1280 640" role="img" aria-label="migrate">
<rect width="1280" height="640" fill="#0A0A0A"/>
<svg x="96" y="215" width="210" height="210" viewBox="0 0 67 67"><path d="M22.21 67V44.6369H0V67H22.21Z" fill="#fff"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z" fill="#fff"/><path d="M22.21 0H0V22.3184H22.21V0Z" fill="#fff"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z" fill="#fff"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z" fill="#fff"/></svg>
<text x="378" y="276" font-family="Inter,system-ui,-apple-system,sans-serif" font-size="78" font-weight="800" letter-spacing="-2" fill="#ffffff">migrate</text>
<text x="378" y="322" font-family="Inter,system-ui,sans-serif" font-size="30" fill="#ffffff" opacity=".66">Database migrations. CLI and Golang library.</text>
<rect x="378" y="338" width="806" height="3" rx="1.5" fill="#ffffff" opacity=".9"/>
<text x="378" y="390" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">github.com/hanzoai</text>
<text x="1184" y="390" text-anchor="end" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">hanzo.ai</text>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+14 -11
View File
@@ -7,22 +7,22 @@ on:
jobs:
lint:
name: lint
runs-on: ubuntu-latest
runs-on: hanzo-build-linux-amd64
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.21.x"
- uses: actions/checkout@v4
go-version: "1.26.4.x"
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v9
with:
version: latest
version: v2.6.2
test:
runs-on: ubuntu-latest
runs-on: hanzo-build-linux-amd64
strategy:
matrix:
go: ["1.20.x", "1.21.x"]
go: ["1.24.x", "1.25.x"]
steps:
- uses: actions/checkout@v4
@@ -43,7 +43,7 @@ jobs:
check-coverage:
name: Check coverage
needs: [test]
runs-on: ubuntu-latest
runs-on: hanzo-build-linux-amd64
steps:
- uses: shogo82148/actions-goveralls@v1
with:
@@ -52,7 +52,7 @@ jobs:
goreleaser:
name: Release a new version
needs: [lint, test]
runs-on: ubuntu-latest
runs-on: hanzo-build-linux-amd64
environment: GoReleaser
# This job only runs when
# 1. When the previous `lint` and `test` jobs has completed successfully
@@ -68,7 +68,7 @@ jobs:
ruby-version: 2.7
- uses: actions/setup-go@v5
with:
go-version: "1.21.x"
go-version: "1.26.4.x"
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
@@ -83,7 +83,7 @@ jobs:
- uses: goreleaser/goreleaser-action@v5
with:
version: latest
args: release --rm-dist
args: release --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -97,6 +97,9 @@ jobs:
- run: package_cloud push golang-migrate/migrate/ubuntu/jammy dist/migrate.linux-amd64.deb
env:
PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
- run: package_cloud push golang-migrate/migrate/ubuntu/noble dist/migrate.linux-amd64.deb
env:
PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
- run: package_cloud push golang-migrate/migrate/debian/buster dist/migrate.linux-amd64.deb
env:
PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+52
View File
@@ -0,0 +1,52 @@
name: Release
on:
push:
tags:
- 'v*'
permissions:
contents: write
packages: write
jobs:
release:
runs-on: hanzo-build-linux-amd64
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.26.4'
- name: Build CLI binaries
run: |
VERSION=${GITHUB_REF_NAME#v}
mkdir -p cli/build
for pair in "linux/amd64" "linux/arm64" "darwin/amd64" "darwin/arm64"; do
OS=${pair%/*}
ARCH=${pair#*/}
OUTPUT="cli/build/migrate.${OS}-${ARCH}"
echo "Building $OUTPUT..."
CGO_ENABLED=0 GOOS=$OS GOARCH=$ARCH \
go build -a -o "$OUTPUT" \
-ldflags="-s -w -X main.Version=$VERSION" \
-tags 'datastore file' \
./cmd/migrate
ln -f "$OUTPUT" cli/build/migrate
tar czf "${OUTPUT}.tar.gz" -C cli/build migrate
done
cd cli/build && sha256sum *.tar.gz > sha256sum.txt
- name: Create GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release create ${{ github.ref_name }} \
cli/build/*.tar.gz \
cli/build/sha256sum.txt \
--title "${{ github.ref_name }}" \
--generate-notes \
--latest
+1
View File
@@ -8,3 +8,4 @@ vendor/
.vscode/
.idea
dist/
migrate
+29 -18
View File
@@ -1,26 +1,37 @@
run:
# timeout for analysis, e.g. 30s, 5m, default is 1m
timeout: 5m
version: "2"
linters:
enable:
#- golint
- interfacer
- unconvert
#- dupl
- goconst
- gofmt
- misspell
- unparam
- nakedret
- prealloc
#- gosec
linters-settings:
misspell:
locale: US
- revive
- unconvert
- unparam
settings:
misspell:
locale: US
revive:
rules:
- name: redundant-build-tag
exclusions:
generated: lax
rules:
- path: (.+)\.go$
text: G104
paths:
- third_party$
- builtin$
- examples$
issues:
max-same-issues: 0
max-issues-per-linter: 0
exclude-use-default: false
exclude:
# gosec: Duplicated errcheck checks
- G104
max-same-issues: 0
formatters:
enable:
- gofmt
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
-1
View File
@@ -87,7 +87,6 @@ release:
prerelease: auto
source:
enabled: true
rlcp: true
format: zip
changelog:
skip: false
Symlink
+1
View File
@@ -0,0 +1 @@
LLM.md
Symlink
+1
View File
@@ -0,0 +1 @@
LLM.md
+2 -2
View File
@@ -1,4 +1,4 @@
FROM golang:1.21-alpine3.19 AS builder
FROM golang:1.26.4-alpine3.21 AS builder
ARG VERSION
RUN apk add --no-cache git gcc musl-dev make
@@ -15,7 +15,7 @@ COPY . ./
RUN make build-docker
FROM alpine:3.19
FROM alpine:3.21
RUN apk add --no-cache ca-certificates
+1 -1
View File
@@ -65,7 +65,7 @@
and then "force" the expected version.
#### What happens if two programs try and update the database at the same time?
Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations at the same time
Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations on
the same database at the same time. For example, the MySQL driver uses the `GET_LOCK` function, while the Postgres driver uses
the `pg_advisory_lock` function.
+39
View File
@@ -0,0 +1,39 @@
# Hanzo Golang Migrate
## Overview
Go module: github.com/golang-migrate/migrate/v4
## Tech Stack
- **Language**: Go
## Build & Run
```bash
go build ./...
go test ./...
```
## Structure
```
golang-migrate/
CONTRIBUTING.md
Dockerfile
Dockerfile.circleci
Dockerfile.github-actions
FAQ.md
GETTING_STARTED.md
LICENSE
MIGRATIONS.md
Makefile
README.md
SECURITY.md
cli/
cmd/
database/
dktesting/
```
## Key Files
- `README.md` -- Project documentation
- `go.mod` -- Go module definition
- `Makefile` -- Build automation
- `Dockerfile` -- Container build
+1 -1
View File
@@ -1,5 +1,5 @@
SOURCE ?= file go_bindata github github_ee bitbucket aws_s3 google_cloud_storage godoc_vfs gitlab
DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb yugabytedb clickhouse mongodb sqlserver firebird neo4j pgx pgx5 rqlite
DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb yugabytedb datastore mongodb sqlserver firebird neo4j pgx pgx5 rqlite
DATABASE_TEST ?= $(DATABASE) sqlite sqlite3 sqlcipher
VERSION ?= $(shell git describe --tags 2>/dev/null | cut -c 2-)
TEST_FLAGS ?=
+8 -6
View File
@@ -1,9 +1,11 @@
<p align="center"><img src=".github/hero.svg" alt="migrate" width="880"></p>
[![GitHub Workflow Status (branch)](https://img.shields.io/github/actions/workflow/status/golang-migrate/migrate/ci.yaml?branch=master)](https://github.com/golang-migrate/migrate/actions/workflows/ci.yaml?query=branch%3Amaster)
[![GoDoc](https://pkg.go.dev/badge/github.com/golang-migrate/migrate)](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)
[![Coverage Status](https://img.shields.io/coveralls/github/golang-migrate/migrate/master.svg)](https://coveralls.io/github/golang-migrate/migrate?branch=master)
[![packagecloud.io](https://img.shields.io/badge/deb-packagecloud.io-844fec.svg)](https://packagecloud.io/golang-migrate/migrate?filter=debs)
[![Docker Pulls](https://img.shields.io/docker/pulls/migrate/migrate.svg)](https://hub.docker.com/r/migrate/migrate/)
![Supported Go Versions](https://img.shields.io/badge/Go-1.20%2C%201.21-lightgrey.svg)
![Supported Go Versions](https://img.shields.io/badge/Go-1.24%2C%201.25-lightgrey.svg)
[![GitHub Release](https://img.shields.io/github/release/golang-migrate/migrate.svg)](https://github.com/golang-migrate/migrate/releases)
[![Go Report Card](https://goreportcard.com/badge/github.com/golang-migrate/migrate/v4)](https://goreportcard.com/report/github.com/golang-migrate/migrate/v4)
@@ -40,10 +42,10 @@ Database drivers run migrations. [Add a new database?](database/driver.go)
* [Google Cloud Spanner](database/spanner)
* [CockroachDB](database/cockroachdb)
* [YugabyteDB](database/yugabytedb)
* [ClickHouse](database/clickhouse)
* [Datastore](database/datastore)
* [Firebird](database/firebird)
* [MS SQL Server](database/sqlserver)
* [RQLite](database/rqlite)
* [rqlite](database/rqlite)
### Database URLs
@@ -87,7 +89,7 @@ Source drivers read migrations from local or remote sources. [Add a new source?]
* Handles ctrl+c (SIGINT) gracefully.
* No config search paths, no config files, no magic ENV var injections.
__[CLI Documentation](cmd/migrate)__
[CLI Documentation](cmd/migrate) (includes CLI install instructions)
### Basic usage
@@ -145,7 +147,7 @@ func main() {
m, err := migrate.NewWithDatabaseInstance(
"file:///migrations",
"postgres", driver)
m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run
m.Up() // or m.Steps(2) if you want to explicitly set the number of migrations to run
}
```
@@ -174,7 +176,7 @@ Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-file
## Coming from another db migration tool?
Check out [migradaptor](https://github.com/musinit/migradaptor/).
*Note: migradaptor is not affliated or supported by this project*
*Note: migradaptor is not affiliated or supported by this project*
## Versions
+12 -8
View File
@@ -27,8 +27,8 @@ $ scoop install migrate
### Linux (*.deb package)
```bash
$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add -
$ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
$ curl -fsSL https://packagecloud.io/golang-migrate/migrate/gpgkey | sudo gpg --dearmor -o /etc/apt/keyrings/migrate.gpg
$ echo "deb [signed-by=/etc/apt/keyrings/migrate.gpg] https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
$ apt-get update
$ apt-get install -y migrate
```
@@ -86,14 +86,18 @@ Options:
-help Print usage
Commands:
create [-ext E] [-dir D] [-seq] [-digits N] [-format] NAME
Create a set of timestamped up/down migrations titled NAME, in directory D with extension E.
Use -seq option to generate sequential up/down migrations with N digits.
Use -format option to specify a Go time format string.
create [-ext E] [-dir D] [-seq] [-digits N] [-format] [-tz] NAME
Create a set of timestamped up/down migrations titled NAME, in directory D with extension E.
Use -seq option to generate sequential up/down migrations with N digits.
Use -format option to specify a Go time format string. Note: migrations with the same time cause "duplicate migration version" error.
Use -tz option to specify the timezone that will be used when generating non-sequential migrations (defaults: UTC).
goto V Migrate to version V
up [N] Apply all or N up migrations
down [N] Apply all or N down migrations
drop Drop everything inside database
down [N] [-all] Apply all or N down migrations
Use -all to apply all down migrations
drop [-f] Drop everything inside database
Use -f to bypass confirmation
force V Set version V but don't run migration (ignores dirty state)
version Print current migration version
```
+5 -11
View File
@@ -7,14 +7,12 @@ import (
nurl "net/url"
"strconv"
"strings"
"sync/atomic"
"time"
"go.uber.org/atomic"
"github.com/gocql/gocql"
"github.com/golang-migrate/migrate/v4/database"
"github.com/golang-migrate/migrate/v4/database/multistmt"
"github.com/hashicorp/go-multierror"
)
func init() {
@@ -199,14 +197,14 @@ func (c *Cassandra) Close() error {
}
func (c *Cassandra) Lock() error {
if !c.isLocked.CAS(false, true) {
if !c.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (c *Cassandra) Unlock() error {
if !c.isLocked.CAS(true, false) {
if !c.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -316,11 +314,7 @@ func (c *Cassandra) ensureVersionTable() (err error) {
defer func() {
if e := c.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -342,7 +336,7 @@ func parseConsistency(consistencyStr string) (consistency gocql.Consistency, err
var ok bool
err, ok = r.(error)
if !ok {
err = fmt.Errorf("Failed to parse consistency \"%s\": %v", consistencyStr, r)
err = fmt.Errorf("failed to parse consistency \"%s\": %v", consistencyStr, r)
}
}
}()
+15 -1
View File
@@ -61,6 +61,20 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
}
func Test(t *testing.T) {
t.Run("test", test)
t.Run("testMigrate", testMigrate)
t.Cleanup(func() {
for _, spec := range specs {
t.Log("Cleaning up ", spec.ImageName)
if err := spec.Cleanup(); err != nil {
t.Error("Error removing ", spec.ImageName, "error:", err)
}
}
})
}
func test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.Port(9042)
if err != nil {
@@ -81,7 +95,7 @@ func Test(t *testing.T) {
})
}
func TestMigrate(t *testing.T) {
func testMigrate(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.Port(9042)
if err != nil {
-26
View File
@@ -1,26 +0,0 @@
# ClickHouse
`clickhouse://host:port?username=user&password=password&database=clicks&x-multi-statement=true`
| URL Query | Description |
|------------|-------------|
| `x-migrations-table`| Name of the migrations table |
| `x-migrations-table-engine`| Engine to use for the migrations table, defaults to TinyLog |
| `x-cluster-name` | Name of cluster for creating `schema_migrations` table cluster wide |
| `database` | The name of the database to connect to |
| `username` | The user to sign in as |
| `password` | The user's password |
| `host` | The host to connect to. |
| `port` | The port to bind to. |
| `x-multi-statement` | false | Enable multiple statements to be ran in a single migration (See note below) |
## Notes
* The Clickhouse driver does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
* This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
* The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
* Using the default TinyLog table engine for the schema_versions table prevents backing up the table if using the [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) tool. If backing up the database with make sure the migrations are run with `x-migrations-table-engine=MergeTree`.
* Clickhouse cluster mode is not officially supported, since it's not tested right now, but you can try enabling `schema_migrations` table replication by specifying a `x-cluster-name`:
* When `x-cluster-name` is specified, `x-migrations-table-engine` also should be specified. See the docs regarding [replicated table engines](https://clickhouse.tech/docs/en/engines/table-engines/mergetree-family/replication/#table_engines-replication).
* When `x-cluster-name` is specified, only the `schema_migrations` table is replicated across the cluster. You still need to write your migrations so that the application tables are replicated within the cluster.
* If you want to create database inside the migration, you should know, that table which will manage migrations `schema-migrations table` will be in `default` table, so you can't use `USE <database_name>` inside migration. In this case you may not specify the database in the connection string (example you can find [here](examples/migrations/003_create_database.up.sql))
+1 -1
View File
@@ -15,7 +15,7 @@ CREATE USER IF NOT EXISTS cockroach;
GRANT ALL ON DATABASE example TO cockroach;
```
When using Migrate CLI we need to pass to database URL. Let's export it to a variable for convienience:
When using Migrate CLI we need to pass to database URL. Let's export it to a variable for convenience:
```
export COCKROACHDB_URL='cockroachdb://cockroach:@localhost:26257/example?sslmode=disable'
```
+5 -9
View File
@@ -3,18 +3,18 @@ package cockroachdb
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"regexp"
"strconv"
"sync/atomic"
"github.com/cockroachdb/cockroach-go/v2/crdb"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
"github.com/lib/pq"
"go.uber.org/atomic"
)
func init() {
@@ -161,7 +161,7 @@ func (c *CockroachDb) Lock() error {
}
defer func() {
if errClose := rows.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -276,7 +276,7 @@ func (c *CockroachDb) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -318,11 +318,7 @@ func (c *CockroachDb) ensureVersionTable() (err error) {
defer func() {
if e := c.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+6 -6
View File
@@ -26,13 +26,13 @@ import (
const defaultPort = 26257
var (
opts = dktest.Options{Cmd: []string{"start", "--insecure"}, PortRequired: true, ReadyFunc: isReady}
// Released versions: https://www.cockroachlabs.com/docs/releases/
opts = dktest.Options{Cmd: []string{"start-single-node", "--insecure"}, PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.cockroachlabs.com/docs/releases/release-support-policy#supported-versions
specs = []dktesting.ContainerSpec{
{ImageName: "cockroachdb/cockroach:v1.0.7", Options: opts},
{ImageName: "cockroachdb/cockroach:v1.1.9", Options: opts},
{ImageName: "cockroachdb/cockroach:v2.0.7", Options: opts},
{ImageName: "cockroachdb/cockroach:v2.1.3", Options: opts},
{ImageName: "cockroachdb/cockroach:latest-v24.3", Options: opts},
{ImageName: "cockroachdb/cockroach:latest-v24.1", Options: opts},
{ImageName: "cockroachdb/cockroach:latest-v23.2", Options: opts},
{ImageName: "cockroachdb/cockroach:latest-v23.1", Options: opts},
}
)
+31
View File
@@ -0,0 +1,31 @@
# Datastore
`datastore://host:port?username=user&password=password&database=clicks&x-multi-statement=true`
Hanzo Datastore is a ClickHouse-wire-compatible database engine. This driver
connects through the native `github.com/hanzo-ds/go` driver, which registers the
`database/sql` driver name `datastore`.
| URL Query | Description |
|------------|-------------|
| `x-migrations-table`| Name of the migrations table |
| `x-migrations-table-engine`| Engine to use for the migrations table, defaults to TinyLog |
| `x-cluster-name` | Name of cluster for creating `schema_migrations` table cluster wide |
| `x-multi-statement` | Enable multiple statements to be ran in a single migration (See note below) |
| `x-multi-statement-max-size` | Maximum size of a single migration in bytes when `x-multi-statement` is set (defaults to 10 MB) |
| `database` | The name of the database to connect to |
| `username` | The user to sign in as |
| `password` | The user's password |
| `host` | The host to connect to. |
| `port` | The port to bind to. |
## Notes
* The Datastore driver does not natively support executing multiple statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
* This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
* The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
* Using the default TinyLog table engine for the `schema_migrations` table prevents backing up the table. If you need to back up the database, run the migrations with `x-migrations-table-engine=MergeTree`.
* Datastore cluster mode is not officially supported, but you can try enabling `schema_migrations` table replication by specifying a `x-cluster-name`:
* When `x-cluster-name` is specified, `x-migrations-table-engine` also should be specified.
* When `x-cluster-name` is specified, only the `schema_migrations` table is replicated across the cluster. You still need to write your migrations so that the application tables are replicated within the cluster.
* If you want to create a database inside a migration, note that the `schema_migrations` table will live in the `default` database, so you cannot use `USE <database_name>` inside a migration. In this case you may omit the database in the connection string (example [here](examples/migrations/003_create_database.up.sql)).
@@ -1,25 +1,43 @@
package clickhouse
// Package datastore implements the golang-migrate database.Driver interface for
// Hanzo Datastore (a ClickHouse-wire-compatible database engine). It registers
// the "datastore://" URL scheme and connects through the native
// github.com/hanzo-ds/go driver, which registers the database/sql driver name
// "datastore".
//
// Usage:
//
// import _ "github.com/golang-migrate/migrate/v4/database/datastore"
//
// Then use a URL like:
//
// datastore://host:9000?username=default&database=default&x-multi-statement=true
package datastore
import (
"database/sql"
"errors"
"fmt"
"io"
"net/url"
"strconv"
"strings"
"sync/atomic"
"time"
"go.uber.org/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/golang-migrate/migrate/v4/database/multistmt"
"github.com/hashicorp/go-multierror"
// Registers the "datastore" database/sql driver (ClickHouse-wire native).
_ "github.com/hanzo-ds/go"
)
var (
multiStmtDelimiter = []byte(";")
// DriverName is the database/sql driver registered by github.com/hanzo-ds/go.
DriverName = "datastore"
DefaultMigrationsTable = "schema_migrations"
DefaultMigrationsTableEngine = "TinyLog"
DefaultMultiStatementMaxSize = 10 * 1 << 20 // 10 MB
@@ -37,9 +55,11 @@ type Config struct {
}
func init() {
database.Register("clickhouse", &ClickHouse{})
database.Register("datastore", &Datastore{})
}
// WithInstance creates a Datastore driver from an existing *sql.DB opened
// against the "datastore" driver.
func WithInstance(conn *sql.DB, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
@@ -49,32 +69,34 @@ func WithInstance(conn *sql.DB, config *Config) (database.Driver, error) {
return nil, err
}
ch := &ClickHouse{
ds := &Datastore{
conn: conn,
config: config,
}
if err := ch.init(); err != nil {
if err := ds.init(); err != nil {
return nil, err
}
return ch, nil
return ds, nil
}
type ClickHouse struct {
// Datastore is a golang-migrate database.Driver backed by the native
// github.com/hanzo-ds/go ClickHouse-wire driver.
type Datastore struct {
conn *sql.DB
config *Config
isLocked atomic.Bool
}
func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
func (ds *Datastore) Open(dsn string) (database.Driver, error) {
purl, err := url.Parse(dsn)
if err != nil {
return nil, err
}
q := migrate.FilterCustomQuery(purl)
q.Scheme = "tcp"
conn, err := sql.Open("clickhouse", q.String())
q.Scheme = DriverName
conn, err := sql.Open(DriverName, q.String())
if err != nil {
return nil, err
}
@@ -92,7 +114,7 @@ func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
migrationsTableEngine = s
}
ch = &ClickHouse{
ds = &Datastore{
conn: conn,
config: &Config{
MigrationsTable: purl.Query().Get("x-migrations-table"),
@@ -104,44 +126,44 @@ func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
},
}
if err := ch.init(); err != nil {
if err := ds.init(); err != nil {
return nil, err
}
return ch, nil
return ds, nil
}
func (ch *ClickHouse) init() error {
if len(ch.config.DatabaseName) == 0 {
if err := ch.conn.QueryRow("SELECT currentDatabase()").Scan(&ch.config.DatabaseName); err != nil {
func (ds *Datastore) init() error {
if len(ds.config.DatabaseName) == 0 {
if err := ds.conn.QueryRow("SELECT currentDatabase()").Scan(&ds.config.DatabaseName); err != nil {
return err
}
}
if len(ch.config.MigrationsTable) == 0 {
ch.config.MigrationsTable = DefaultMigrationsTable
if len(ds.config.MigrationsTable) == 0 {
ds.config.MigrationsTable = DefaultMigrationsTable
}
if ch.config.MultiStatementMaxSize <= 0 {
ch.config.MultiStatementMaxSize = DefaultMultiStatementMaxSize
if ds.config.MultiStatementMaxSize <= 0 {
ds.config.MultiStatementMaxSize = DefaultMultiStatementMaxSize
}
if len(ch.config.MigrationsTableEngine) == 0 {
ch.config.MigrationsTableEngine = DefaultMigrationsTableEngine
if len(ds.config.MigrationsTableEngine) == 0 {
ds.config.MigrationsTableEngine = DefaultMigrationsTableEngine
}
return ch.ensureVersionTable()
return ds.ensureVersionTable()
}
func (ch *ClickHouse) Run(r io.Reader) error {
if ch.config.MultiStatementEnabled {
func (ds *Datastore) Run(r io.Reader) error {
if ds.config.MultiStatementEnabled {
var err error
if e := multistmt.Parse(r, multiStmtDelimiter, ch.config.MultiStatementMaxSize, func(m []byte) bool {
if e := multistmt.Parse(r, multiStmtDelimiter, ds.config.MultiStatementMaxSize, func(m []byte) bool {
tq := strings.TrimSpace(string(m))
if tq == "" {
return true
}
if _, e := ch.conn.Exec(string(m)); e != nil {
if _, e := ds.conn.Exec(string(m)); e != nil {
err = database.Error{OrigErr: e, Err: "migration failed", Query: m}
return false
}
@@ -157,19 +179,20 @@ func (ch *ClickHouse) Run(r io.Reader) error {
return err
}
if _, err := ch.conn.Exec(string(migration)); err != nil {
if _, err := ds.conn.Exec(string(migration)); err != nil {
return database.Error{OrigErr: err, Err: "migration failed", Query: migration}
}
return nil
}
func (ch *ClickHouse) Version() (int, bool, error) {
func (ds *Datastore) Version() (int, bool, error) {
var (
version int
dirty uint8
query = "SELECT version, dirty FROM `" + ch.config.MigrationsTable + "` ORDER BY sequence DESC LIMIT 1"
query = "SELECT version, dirty FROM `" + ds.config.MigrationsTable + "` ORDER BY sequence DESC LIMIT 1"
)
if err := ch.conn.QueryRow(query).Scan(&version, &dirty); err != nil {
if err := ds.conn.QueryRow(query).Scan(&version, &dirty); err != nil {
if err == sql.ErrNoRows {
return database.NilVersion, false, nil
}
@@ -178,7 +201,7 @@ func (ch *ClickHouse) Version() (int, bool, error) {
return version, dirty == 1, nil
}
func (ch *ClickHouse) SetVersion(version int, dirty bool) error {
func (ds *Datastore) SetVersion(version int, dirty bool) error {
var (
bool = func(v bool) uint8 {
if v {
@@ -186,13 +209,13 @@ func (ch *ClickHouse) SetVersion(version int, dirty bool) error {
}
return 0
}
tx, err = ch.conn.Begin()
tx, err = ds.conn.Begin()
)
if err != nil {
return err
}
query := "INSERT INTO " + ch.config.MigrationsTable + " (version, dirty, sequence) VALUES (?, ?, ?)"
query := "INSERT INTO " + ds.config.MigrationsTable + " (version, dirty, sequence) VALUES (?, ?, ?)"
if _, err := tx.Exec(query, version, bool(dirty), time.Now().UnixNano()); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -202,28 +225,24 @@ func (ch *ClickHouse) SetVersion(version int, dirty bool) error {
// ensureVersionTable checks if versions table exists and, if not, creates it.
// Note that this function locks the database, which deviates from the usual
// convention of "caller locks" in the ClickHouse type.
func (ch *ClickHouse) ensureVersionTable() (err error) {
if err = ch.Lock(); err != nil {
// convention of "caller locks" in the Datastore type.
func (ds *Datastore) ensureVersionTable() (err error) {
if err = ds.Lock(); err != nil {
return err
}
defer func() {
if e := ch.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
if e := ds.Unlock(); e != nil {
err = errors.Join(err, e)
}
}()
var (
table string
query = "SHOW TABLES FROM " + quoteIdentifier(ch.config.DatabaseName) + " LIKE '" + ch.config.MigrationsTable + "'"
query = "SHOW TABLES FROM " + quoteIdentifier(ds.config.DatabaseName) + " LIKE '" + ds.config.MigrationsTable + "'"
)
// check if migration table exists
if err := ch.conn.QueryRow(query).Scan(&table); err != nil {
if err := ds.conn.QueryRow(query).Scan(&table); err != nil {
if err != sql.ErrNoRows {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -232,42 +251,42 @@ func (ch *ClickHouse) ensureVersionTable() (err error) {
}
// if not, create the empty migration table
if len(ch.config.ClusterName) > 0 {
if len(ds.config.ClusterName) > 0 {
query = fmt.Sprintf(`
CREATE TABLE %s ON CLUSTER %s (
version Int64,
dirty UInt8,
sequence UInt64
) Engine=%s`, ch.config.MigrationsTable, ch.config.ClusterName, ch.config.MigrationsTableEngine)
) Engine=%s`, ds.config.MigrationsTable, ds.config.ClusterName, ds.config.MigrationsTableEngine)
} else {
query = fmt.Sprintf(`
CREATE TABLE %s (
version Int64,
dirty UInt8,
sequence UInt64
) Engine=%s`, ch.config.MigrationsTable, ch.config.MigrationsTableEngine)
) Engine=%s`, ds.config.MigrationsTable, ds.config.MigrationsTableEngine)
}
if strings.HasSuffix(ch.config.MigrationsTableEngine, "Tree") {
if strings.HasSuffix(ds.config.MigrationsTableEngine, "Tree") {
query = fmt.Sprintf(`%s ORDER BY sequence`, query)
}
if _, err := ch.conn.Exec(query); err != nil {
if _, err := ds.conn.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
func (ch *ClickHouse) Drop() (err error) {
query := "SHOW TABLES FROM " + quoteIdentifier(ch.config.DatabaseName)
tables, err := ch.conn.Query(query)
func (ds *Datastore) Drop() (err error) {
query := "SHOW TABLES FROM " + quoteIdentifier(ds.config.DatabaseName)
tables, err := ds.conn.Query(query)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -277,9 +296,9 @@ func (ch *ClickHouse) Drop() (err error) {
return err
}
query = "DROP TABLE IF EXISTS " + quoteIdentifier(ch.config.DatabaseName) + "." + quoteIdentifier(table)
query = "DROP TABLE IF EXISTS " + quoteIdentifier(ds.config.DatabaseName) + "." + quoteIdentifier(table)
if _, err := ch.conn.Exec(query); err != nil {
if _, err := ds.conn.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
@@ -290,21 +309,23 @@ func (ch *ClickHouse) Drop() (err error) {
return nil
}
func (ch *ClickHouse) Lock() error {
if !ch.isLocked.CAS(false, true) {
func (ds *Datastore) Lock() error {
if !ds.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (ch *ClickHouse) Unlock() error {
if !ch.isLocked.CAS(true, false) {
func (ds *Datastore) Unlock() error {
if !ds.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
}
func (ch *ClickHouse) Close() error { return ch.conn.Close() }
func (ds *Datastore) Close() error { return ds.conn.Close() }
// Copied from lib/pq implementation: https://github.com/lib/pq/blob/v1.9.0/conn.go#L1611
func quoteIdentifier(name string) string {
@@ -312,5 +333,5 @@ func quoteIdentifier(name string) string {
if end > -1 {
name = name[:end]
}
return `"` + strings.Replace(name, `"`, `""`, -1) + `"`
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
}
@@ -1,4 +1,4 @@
package clickhouse_test
package datastore_test
import (
"context"
@@ -8,13 +8,15 @@ import (
"log"
"testing"
_ "github.com/ClickHouse/clickhouse-go"
"github.com/dhui/dktest"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database/clickhouse"
"github.com/golang-migrate/migrate/v4/database/datastore"
dt "github.com/golang-migrate/migrate/v4/database/testing"
"github.com/golang-migrate/migrate/v4/dktesting"
_ "github.com/golang-migrate/migrate/v4/source/file"
// Registers the "datastore" database/sql driver (ClickHouse-wire native).
_ "github.com/hanzo-ds/go"
)
const defaultPort = 9000
@@ -26,19 +28,19 @@ var (
PortRequired: true, ReadyFunc: isReady,
}
specs = []dktesting.ContainerSpec{
{ImageName: "yandex/clickhouse-server:21.3", Options: opts},
{ImageName: "clickhouse:24.8", Options: opts},
}
)
func clickhouseConnectionString(host, port, engine string) string {
func datastoreConnectionString(host, port, engine string) string {
if engine != "" {
return fmt.Sprintf(
"clickhouse://%v:%v?username=user&password=password&database=db&x-multi-statement=true&x-migrations-table-engine=%v&debug=false",
"datastore://%v:%v?username=user&password=password&database=db&x-multi-statement=true&x-migrations-table-engine=%v&debug=false",
host, port, engine)
}
return fmt.Sprintf(
"clickhouse://%v:%v?username=user&password=password&database=db&x-multi-statement=true&debug=false",
"datastore://%v:%v?username=user&password=password&database=db&x-multi-statement=true&debug=false",
host, port)
}
@@ -48,7 +50,7 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
return false
}
db, err := sql.Open("clickhouse", clickhouseConnectionString(ip, port, ""))
db, err := sql.Open("datastore", datastoreConnectionString(ip, port, ""))
if err != nil {
log.Println("open error", err)
@@ -90,8 +92,8 @@ func testSimple(t *testing.T, engine string) {
t.Fatal(err)
}
addr := clickhouseConnectionString(ip, port, engine)
p := &clickhouse.ClickHouse{}
addr := datastoreConnectionString(ip, port, engine)
p := &datastore.Datastore{}
d, err := p.Open(addr)
if err != nil {
t.Fatal(err)
@@ -113,12 +115,12 @@ func testSimpleWithInstanceDefaultConfigValues(t *testing.T) {
t.Fatal(err)
}
addr := clickhouseConnectionString(ip, port, "")
conn, err := sql.Open("clickhouse", addr)
addr := datastoreConnectionString(ip, port, "")
conn, err := sql.Open("datastore", addr)
if err != nil {
t.Fatal(err)
}
d, err := clickhouse.WithInstance(conn, &clickhouse.Config{})
d, err := datastore.WithInstance(conn, &datastore.Config{})
if err != nil {
_ = conn.Close()
t.Fatal(err)
@@ -140,8 +142,8 @@ func testMigrate(t *testing.T, engine string) {
t.Fatal(err)
}
addr := clickhouseConnectionString(ip, port, engine)
p := &clickhouse.ClickHouse{}
addr := datastoreConnectionString(ip, port, engine)
p := &datastore.Datastore{}
d, err := p.Open(addr)
if err != nil {
t.Fatal(err)
@@ -169,8 +171,8 @@ func testVersion(t *testing.T, engine string) {
t.Fatal(err)
}
addr := clickhouseConnectionString(ip, port, engine)
p := &clickhouse.ClickHouse{}
addr := datastoreConnectionString(ip, port, engine)
p := &datastore.Datastore{}
d, err := p.Open(addr)
if err != nil {
t.Fatal(err)
@@ -204,8 +206,8 @@ func testDrop(t *testing.T, engine string) {
t.Fatal(err)
}
addr := clickhouseConnectionString(ip, port, engine)
p := &clickhouse.ClickHouse{}
addr := datastoreConnectionString(ip, port, engine)
p := &datastore.Datastore{}
d, err := p.Open(addr)
if err != nil {
t.Fatal(err)
@@ -1,3 +1,3 @@
CREATE TABLE test_1 (
Date Date
Date Date
) Engine=Memory;
@@ -1,3 +1,3 @@
CREATE TABLE test_2 (
Date Date
Date Date
) Engine=Memory;
+6 -11
View File
@@ -1,20 +1,19 @@
//go:build go1.9
// +build go1.9
package firebird
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
_ "github.com/nakagami/firebirdsql"
"go.uber.org/atomic"
)
func init() {
@@ -108,14 +107,14 @@ func (f *Firebird) Close() error {
}
func (f *Firebird) Lock() error {
if !f.isLocked.CAS(false, true) {
if !f.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (f *Firebird) Unlock() error {
if !f.isLocked.CAS(true, false) {
if !f.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -181,7 +180,7 @@ func (f *Firebird) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -224,11 +223,7 @@ func (f *Firebird) ensureVersionTable() (err error) {
defer func() {
if e := f.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+3 -2
View File
@@ -38,8 +38,9 @@ var (
},
}
specs = []dktesting.ContainerSpec{
{ImageName: "jacobalberty/firebird:2.5-ss", Options: opts},
{ImageName: "jacobalberty/firebird:3.0", Options: opts},
{ImageName: "jacobalberty/firebird:v3.0", Options: opts},
{ImageName: "jacobalberty/firebird:v4.0", Options: opts},
{ImageName: "jacobalberty/firebird:v5.0", Options: opts},
}
)
+3 -7
View File
@@ -2,21 +2,21 @@ package mongodb
import (
"context"
"errors"
"fmt"
"io"
"net/url"
"os"
"strconv"
"sync/atomic"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/x/mongo/driver/connstring"
"go.uber.org/atomic"
)
func init() {
@@ -325,11 +325,7 @@ func (m *Mongo) ensureVersionTable() (err error) {
defer func() {
if e := m.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+32 -7
View File
@@ -32,10 +32,10 @@ var (
opts = dktest.Options{PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.mongodb.com/support-policy
specs = []dktesting.ContainerSpec{
{ImageName: "mongo:3.4", Options: opts},
{ImageName: "mongo:3.6", Options: opts},
{ImageName: "mongo:4.0", Options: opts},
{ImageName: "mongo:4.2", Options: opts},
{ImageName: "mongo:5.0", Options: opts},
{ImageName: "mongo:6.0", Options: opts},
{ImageName: "mongo:7.0", Options: opts},
{ImageName: "mongo:8.0", Options: opts},
}
)
@@ -74,6 +74,22 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
}
func Test(t *testing.T) {
t.Run("test", test)
t.Run("testMigrate", testMigrate)
t.Run("testWithAuth", testWithAuth)
t.Run("testLockWorks", testLockWorks)
t.Cleanup(func() {
for _, spec := range specs {
t.Log("Cleaning up ", spec.ImageName)
if err := spec.Cleanup(); err != nil {
t.Error("Error removing ", spec.ImageName, "error:", err)
}
}
})
}
func test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -99,7 +115,7 @@ func Test(t *testing.T) {
})
}
func TestMigrate(t *testing.T) {
func testMigrate(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -125,7 +141,7 @@ func TestMigrate(t *testing.T) {
})
}
func TestWithAuth(t *testing.T) {
func testWithAuth(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -180,7 +196,7 @@ func TestWithAuth(t *testing.T) {
})
}
func TestLockWorks(t *testing.T) {
func testLockWorks(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -241,6 +257,15 @@ func TestTransaction(t *testing.T) {
{ImageName: "mongo:4", Options: dktest.Options{PortRequired: true, ReadyFunc: isReady,
Cmd: []string{"mongod", "--bind_ip_all", "--replSet", "rs0"}}},
}
t.Cleanup(func() {
for _, spec := range transactionSpecs {
t.Log("Cleaning up ", spec.ImageName)
if err := spec.Cleanup(); err != nil {
t.Error("Error removing ", spec.ImageName, "error:", err)
}
}
})
dktesting.ParallelTest(t, transactionSpecs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
+1 -1
View File
@@ -14,7 +14,7 @@
| `port` | | The port to bind to. |
| `tls` | | TLS / SSL encrypted connection parameter; see [go-sql-driver](https://github.com/go-sql-driver/mysql#tls). Use any name (e.g. `migrate`) if you want to use a custom TLS config (`x-tls-` queries). |
| `x-tls-ca` | | The location of the CA (certificate authority) file. |
| `x-tls-cert` | | The location of the client certicicate file. Must be used with `x-tls-key`. |
| `x-tls-cert` | | The location of the client certificate file. Must be used with `x-tls-key`. |
| `x-tls-key` | | The location of the private key file. Must be used with `x-tls-cert`. |
| `x-tls-insecure-skip-verify` | | Whether or not to use SSL (true\|false) |
+8 -14
View File
@@ -1,5 +1,4 @@
//go:build go1.9
// +build go1.9
package mysql
@@ -8,19 +7,18 @@ import (
"crypto/tls"
"crypto/x509"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"os"
"strconv"
"strings"
"sync/atomic"
"time"
"go.uber.org/atomic"
"github.com/go-sql-driver/mysql"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
)
var _ database.Driver = (*Mysql)(nil) // explicit compile time type check
@@ -36,7 +34,7 @@ var (
ErrNilConfig = fmt.Errorf("no config")
ErrNoDatabaseName = fmt.Errorf("no database name")
ErrAppendPEM = fmt.Errorf("failed to append PEM")
ErrTLSCertKeyConfig = fmt.Errorf("To use TLS client authentication, both x-tls-cert and x-tls-key must not be empty")
ErrTLSCertKeyConfig = fmt.Errorf("to use TLS client authentication, both x-tls-cert and x-tls-key must not be empty")
)
type Config struct {
@@ -361,10 +359,10 @@ func (m *Mysql) SetVersion(version int, dirty bool) error {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
query := "DELETE FROM `" + m.config.MigrationsTable + "`"
query := "DELETE FROM `" + m.config.MigrationsTable + "` LIMIT 1"
if _, err := tx.ExecContext(context.Background(), query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -376,7 +374,7 @@ func (m *Mysql) SetVersion(version int, dirty bool) error {
query := "INSERT INTO `" + m.config.MigrationsTable + "` (version, dirty) VALUES (?, ?)"
if _, err := tx.ExecContext(context.Background(), query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -418,7 +416,7 @@ func (m *Mysql) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -471,11 +469,7 @@ func (m *Mysql) ensureVersionTable() (err error) {
defer func() {
if e := m.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+7 -9
View File
@@ -40,16 +40,14 @@ var (
}
// Supported versions: https://www.mysql.com/support/supportedplatforms/database.html
specs = []dktesting.ContainerSpec{
{ImageName: "mysql:5.5", Options: opts},
{ImageName: "mysql:5.6", Options: opts},
{ImageName: "mysql:5.7", Options: opts},
{ImageName: "mysql:8", Options: opts},
{ImageName: "mysql:8.0", Options: opts},
{ImageName: "mysql:8.4", Options: opts},
{ImageName: "mysql:9.0", Options: opts},
}
specsAnsiQuotes = []dktesting.ContainerSpec{
{ImageName: "mysql:5.5", Options: optsAnsiQuotes},
{ImageName: "mysql:5.6", Options: optsAnsiQuotes},
{ImageName: "mysql:5.7", Options: optsAnsiQuotes},
{ImageName: "mysql:8", Options: optsAnsiQuotes},
{ImageName: "mysql:8.0", Options: optsAnsiQuotes},
{ImageName: "mysql:8.4", Options: optsAnsiQuotes},
{ImageName: "mysql:9.0", Options: optsAnsiQuotes},
}
)
@@ -391,7 +389,7 @@ func TestURLToMySQLConfig(t *testing.T) {
// Not supported yet: https://github.com/go-sql-driver/mysql/issues/591
// {name: "user/password - user with encoded :",
// urlStr: "mysql://username%3A:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
// expectedDSN: "username::pasword@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
// expectedDSN: "username::password@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "user/password - user with encoded @",
urlStr: "mysql://username%40:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username@:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
+6 -6
View File
@@ -2,6 +2,7 @@ package neo4j
import (
"bytes"
"errors"
"fmt"
"io"
neturl "net/url"
@@ -10,7 +11,6 @@ import (
"github.com/golang-migrate/migrate/v4/database"
"github.com/golang-migrate/migrate/v4/database/multistmt"
"github.com/hashicorp/go-multierror"
"github.com/neo4j/neo4j-go-driver/neo4j"
)
@@ -141,7 +141,7 @@ func (n *Neo4j) Run(migration io.Reader) (err error) {
}
defer func() {
if cerr := session.Close(); cerr != nil {
err = multierror.Append(err, cerr)
err = errors.Join(err, cerr)
}
}()
@@ -188,7 +188,7 @@ func (n *Neo4j) SetVersion(version int, dirty bool) (err error) {
}
defer func() {
if cerr := session.Close(); cerr != nil {
err = multierror.Append(err, cerr)
err = errors.Join(err, cerr)
}
}()
@@ -213,7 +213,7 @@ func (n *Neo4j) Version() (version int, dirty bool, err error) {
}
defer func() {
if cerr := session.Close(); cerr != nil {
err = multierror.Append(err, cerr)
err = errors.Join(err, cerr)
}
}()
@@ -261,7 +261,7 @@ func (n *Neo4j) Drop() (err error) {
}
defer func() {
if cerr := session.Close(); cerr != nil {
err = multierror.Append(err, cerr)
err = errors.Join(err, cerr)
}
}()
@@ -278,7 +278,7 @@ func (n *Neo4j) ensureVersionConstraint() (err error) {
}
defer func() {
if cerr := session.Close(); cerr != nil {
err = multierror.Append(err, cerr)
err = errors.Join(err, cerr)
}
}()
+2
View File
@@ -11,6 +11,8 @@ This package is for [pgx/v4](https://pkg.go.dev/github.com/jackc/pgx/v4). A back
| `x-statement-timeout` | `StatementTimeout` | Abort any statement that takes more than the specified number of milliseconds |
| `x-multi-statement` | `MultiStatementEnabled` | Enable multi-statement execution (default: false) |
| `x-multi-statement-max-size` | `MultiStatementMaxSize` | Maximum size of single statement in bytes (default: 10MB) |
| `x-lock-strategy` | `LockStrategy` | Strategy used for locking during migration (default: advisory) |
| `x-lock-table` | `LockTable` | Name of the table which maintains the migration lock (default: schema_lock) |
| `dbname` | `DatabaseName` | The name of the database to connect to |
| `search_path` | | This variable specifies the order in which schemas are searched when an object is referenced by a simple name with no schema specified. |
| `user` | | The user to sign in as |
+165 -36
View File
@@ -1,28 +1,32 @@
//go:build go1.9
// +build go1.9
package pgx
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"regexp"
"strconv"
"strings"
"sync/atomic"
"time"
"go.uber.org/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/golang-migrate/migrate/v4/database/multistmt"
"github.com/hashicorp/go-multierror"
"github.com/jackc/pgconn"
"github.com/jackc/pgerrcode"
_ "github.com/jackc/pgx/v4/stdlib"
"github.com/lib/pq"
)
const (
LockStrategyAdvisory = "advisory"
LockStrategyTable = "table"
)
func init() {
@@ -36,6 +40,8 @@ var (
DefaultMigrationsTable = "schema_migrations"
DefaultMultiStatementMaxSize = 10 * 1 << 20 // 10 MB
DefaultLockTable = "schema_lock"
DefaultLockStrategy = LockStrategyAdvisory
)
var (
@@ -49,6 +55,8 @@ type Config struct {
MigrationsTable string
DatabaseName string
SchemaName string
LockTable string
LockStrategy string
migrationsSchemaName string
migrationsTableName string
StatementTimeout time.Duration
@@ -108,6 +116,14 @@ func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
config.MigrationsTable = DefaultMigrationsTable
}
if len(config.LockTable) == 0 {
config.LockTable = DefaultLockTable
}
if len(config.LockStrategy) == 0 {
config.LockStrategy = DefaultLockStrategy
}
config.migrationsSchemaName = config.SchemaName
config.migrationsTableName = config.MigrationsTable
if config.MigrationsTableQuoted {
@@ -133,6 +149,10 @@ func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
config: config,
}
if err := px.ensureLockTable(); err != nil {
return nil, err
}
if err := px.ensureVersionTable(); err != nil {
return nil, err
}
@@ -161,7 +181,7 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
if s := purl.Query().Get("x-migrations-table-quoted"); len(s) > 0 {
migrationsTableQuoted, err = strconv.ParseBool(s)
if err != nil {
return nil, fmt.Errorf("Unable to parse option x-migrations-table-quoted: %w", err)
return nil, fmt.Errorf("unable to parse option x-migrations-table-quoted: %w", err)
}
}
if (len(migrationsTable) > 0) && (migrationsTableQuoted) && ((migrationsTable[0] != '"') || (migrationsTable[len(migrationsTable)-1] != '"')) {
@@ -192,10 +212,13 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
if s := purl.Query().Get("x-multi-statement"); len(s) > 0 {
multiStatementEnabled, err = strconv.ParseBool(s)
if err != nil {
return nil, fmt.Errorf("Unable to parse option x-multi-statement: %w", err)
return nil, fmt.Errorf("unable to parse option x-multi-statement: %w", err)
}
}
lockStrategy := purl.Query().Get("x-lock-strategy")
lockTable := purl.Query().Get("x-lock-table")
px, err := WithInstance(db, &Config{
DatabaseName: purl.Path,
MigrationsTable: migrationsTable,
@@ -203,6 +226,8 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
StatementTimeout: time.Duration(statementTimeout) * time.Millisecond,
MultiStatementEnabled: multiStatementEnabled,
MultiStatementMaxSize: multiStatementMaxSize,
LockStrategy: lockStrategy,
LockTable: lockTable,
})
if err != nil {
@@ -221,38 +246,118 @@ func (p *Postgres) Close() error {
return nil
}
// https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
func (p *Postgres) Lock() error {
return database.CasRestoreOnErr(&p.isLocked, false, true, database.ErrLocked, func() error {
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.migrationsSchemaName, p.config.migrationsTableName)
if err != nil {
return err
switch p.config.LockStrategy {
case LockStrategyAdvisory:
return p.applyAdvisoryLock()
case LockStrategyTable:
return p.applyTableLock()
default:
return fmt.Errorf("unknown lock strategy \"%s\"", p.config.LockStrategy)
}
// This will wait indefinitely until the lock can be acquired.
query := `SELECT pg_advisory_lock($1)`
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
}
return nil
})
}
func (p *Postgres) Unlock() error {
return database.CasRestoreOnErr(&p.isLocked, true, false, database.ErrNotLocked, func() error {
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.migrationsSchemaName, p.config.migrationsTableName)
if err != nil {
return err
switch p.config.LockStrategy {
case LockStrategyAdvisory:
return p.releaseAdvisoryLock()
case LockStrategyTable:
return p.releaseTableLock()
default:
return fmt.Errorf("unknown lock strategy \"%s\"", p.config.LockStrategy)
}
query := `SELECT pg_advisory_unlock($1)`
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
})
}
// https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
func (p *Postgres) applyAdvisoryLock() error {
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.migrationsSchemaName, p.config.migrationsTableName)
if err != nil {
return err
}
// This will wait indefinitely until the lock can be acquired.
query := `SELECT pg_advisory_lock($1)`
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
}
return nil
}
func (p *Postgres) applyTableLock() error {
tx, err := p.conn.BeginTx(context.Background(), &sql.TxOptions{})
if err != nil {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
defer func() {
errRollback := tx.Rollback()
if errRollback != nil {
err = errors.Join(err, errRollback)
}
}()
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName)
if err != nil {
return err
}
query := "SELECT * FROM " + pq.QuoteIdentifier(p.config.LockTable) + " WHERE lock_id = $1"
rows, err := tx.Query(query, aid)
if err != nil {
return database.Error{OrigErr: err, Err: "failed to fetch migration lock", Query: []byte(query)}
}
defer func() {
if errClose := rows.Close(); errClose != nil {
err = errors.Join(err, errClose)
}
}()
// If row exists at all, lock is present
locked := rows.Next()
if locked {
return database.ErrLocked
}
query = "INSERT INTO " + pq.QuoteIdentifier(p.config.LockTable) + " (lock_id) VALUES ($1)"
if _, err := tx.Exec(query, aid); err != nil {
return database.Error{OrigErr: err, Err: "failed to set migration lock", Query: []byte(query)}
}
return tx.Commit()
}
func (p *Postgres) releaseAdvisoryLock() error {
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.migrationsSchemaName, p.config.migrationsTableName)
if err != nil {
return err
}
query := `SELECT pg_advisory_unlock($1)`
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
func (p *Postgres) releaseTableLock() error {
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName)
if err != nil {
return err
}
query := "DELETE FROM " + pq.QuoteIdentifier(p.config.LockTable) + " WHERE lock_id = $1"
if _, err := p.db.Exec(query, aid); err != nil {
return database.Error{OrigErr: err, Err: "failed to release migration lock", Query: []byte(query)}
}
return nil
}
func (p *Postgres) Run(migration io.Reader) error {
if p.config.MultiStatementEnabled {
var err error
@@ -307,7 +412,7 @@ func (p *Postgres) runStatement(statement []byte) error {
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
// replace crlf with lf
s = strings.Replace(s, "\r\n", "\n", -1)
s = strings.ReplaceAll(s, "\r\n", "\n")
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
runes := []rune(s)
if pos > len(runes) {
@@ -349,7 +454,7 @@ func (p *Postgres) SetVersion(version int, dirty bool) error {
query := `TRUNCATE ` + quoteIdentifier(p.config.migrationsSchemaName) + `.` + quoteIdentifier(p.config.migrationsTableName)
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -361,7 +466,7 @@ func (p *Postgres) SetVersion(version int, dirty bool) error {
query = `INSERT INTO ` + quoteIdentifier(p.config.migrationsSchemaName) + `.` + quoteIdentifier(p.config.migrationsTableName) + ` (version, dirty) VALUES ($1, $2)`
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -403,7 +508,7 @@ func (p *Postgres) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -414,6 +519,12 @@ func (p *Postgres) Drop() (err error) {
if err := tables.Scan(&tableName); err != nil {
return err
}
// do not drop lock table
if tableName == p.config.LockTable && p.config.LockStrategy == LockStrategyTable {
continue
}
if len(tableName) > 0 {
tableNames = append(tableNames, tableName)
}
@@ -445,11 +556,7 @@ func (p *Postgres) ensureVersionTable() (err error) {
defer func() {
if e := p.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -478,11 +585,33 @@ func (p *Postgres) ensureVersionTable() (err error) {
return nil
}
func (p *Postgres) ensureLockTable() error {
if p.config.LockStrategy != LockStrategyTable {
return nil
}
var count int
query := `SELECT COUNT(1) FROM information_schema.tables WHERE table_name = $1 AND table_schema = (SELECT current_schema()) LIMIT 1`
if err := p.db.QueryRow(query, p.config.LockTable).Scan(&count); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if count == 1 {
return nil
}
query = `CREATE TABLE ` + pq.QuoteIdentifier(p.config.LockTable) + ` (lock_id BIGINT NOT NULL PRIMARY KEY)`
if _, err := p.db.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
// Copied from lib/pq implementation: https://github.com/lib/pq/blob/v1.9.0/conn.go#L1611
func quoteIdentifier(name string) string {
end := strings.IndexRune(name, 0)
if end > -1 {
name = name[:end]
}
return `"` + strings.Replace(name, `"`, `""`, -1) + `"`
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
}
+30 -7
View File
@@ -34,14 +34,11 @@ var (
PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.postgresql.org/support/versioning/
specs = []dktesting.ContainerSpec{
{ImageName: "postgres:9.5", Options: opts},
{ImageName: "postgres:9.6", Options: opts},
{ImageName: "postgres:10", Options: opts},
{ImageName: "postgres:11", Options: opts},
{ImageName: "postgres:12", Options: opts},
{ImageName: "postgres:13", Options: opts},
{ImageName: "postgres:14", Options: opts},
{ImageName: "postgres:15", Options: opts},
{ImageName: "postgres:16", Options: opts},
{ImageName: "postgres:17", Options: opts},
}
)
@@ -134,6 +131,32 @@ func TestMigrate(t *testing.T) {
})
}
func TestMigrateLockTable(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port, "x-lock-strategy=table", "x-lock-table=lock_table")
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatal(err)
}
defer func() {
if err := d.Close(); err != nil {
t.Error(err)
}
}()
m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "pgx", d)
if err != nil {
t.Fatal(err)
}
dt.TestMigrate(t, m)
})
}
func TestMultipleStatements(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
@@ -732,10 +755,10 @@ func Test_computeLineFromPos(t *testing.T) {
t.Run(name, func(t *testing.T) {
input := tc.input
if crlf {
input = strings.Replace(input, "\n", "\r\n", -1)
input = strings.ReplaceAll(input, "\n", "\r\n")
}
if nonASCII {
input = strings.Replace(input, "FROM", "FRÖM", -1)
input = strings.ReplaceAll(input, "FROM", "FRÖM")
}
gotLine, gotCol, gotOK := computeLineFromPos(input, tc.pos)
+10 -16
View File
@@ -1,25 +1,23 @@
//go:build go1.9
// +build go1.9
package pgx
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"regexp"
"strconv"
"strings"
"sync/atomic"
"time"
"go.uber.org/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/golang-migrate/migrate/v4/database/multistmt"
"github.com/hashicorp/go-multierror"
"github.com/jackc/pgerrcode"
"github.com/jackc/pgx/v5/pgconn"
_ "github.com/jackc/pgx/v5/stdlib"
@@ -159,7 +157,7 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
if s := purl.Query().Get("x-migrations-table-quoted"); len(s) > 0 {
migrationsTableQuoted, err = strconv.ParseBool(s)
if err != nil {
return nil, fmt.Errorf("Unable to parse option x-migrations-table-quoted: %w", err)
return nil, fmt.Errorf("unable to parse option x-migrations-table-quoted: %w", err)
}
}
if (len(migrationsTable) > 0) && (migrationsTableQuoted) && ((migrationsTable[0] != '"') || (migrationsTable[len(migrationsTable)-1] != '"')) {
@@ -190,7 +188,7 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
if s := purl.Query().Get("x-multi-statement"); len(s) > 0 {
multiStatementEnabled, err = strconv.ParseBool(s)
if err != nil {
return nil, fmt.Errorf("Unable to parse option x-multi-statement: %w", err)
return nil, fmt.Errorf("unable to parse option x-multi-statement: %w", err)
}
}
@@ -305,7 +303,7 @@ func (p *Postgres) runStatement(statement []byte) error {
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
// replace crlf with lf
s = strings.Replace(s, "\r\n", "\n", -1)
s = strings.ReplaceAll(s, "\r\n", "\n")
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
runes := []rune(s)
if pos > len(runes) {
@@ -347,7 +345,7 @@ func (p *Postgres) SetVersion(version int, dirty bool) error {
query := `TRUNCATE ` + quoteIdentifier(p.config.migrationsSchemaName) + `.` + quoteIdentifier(p.config.migrationsTableName)
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -359,7 +357,7 @@ func (p *Postgres) SetVersion(version int, dirty bool) error {
query = `INSERT INTO ` + quoteIdentifier(p.config.migrationsSchemaName) + `.` + quoteIdentifier(p.config.migrationsTableName) + ` (version, dirty) VALUES ($1, $2)`
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -401,7 +399,7 @@ func (p *Postgres) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -443,11 +441,7 @@ func (p *Postgres) ensureVersionTable() (err error) {
defer func() {
if e := p.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -482,5 +476,5 @@ func quoteIdentifier(name string) string {
if end > -1 {
name = name[:end]
}
return `"` + strings.Replace(name, `"`, `""`, -1) + `"`
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
}
+4 -7
View File
@@ -35,14 +35,11 @@ var (
PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.postgresql.org/support/versioning/
specs = []dktesting.ContainerSpec{
{ImageName: "postgres:9.5", Options: opts},
{ImageName: "postgres:9.6", Options: opts},
{ImageName: "postgres:10", Options: opts},
{ImageName: "postgres:11", Options: opts},
{ImageName: "postgres:12", Options: opts},
{ImageName: "postgres:13", Options: opts},
{ImageName: "postgres:14", Options: opts},
{ImageName: "postgres:15", Options: opts},
{ImageName: "postgres:16", Options: opts},
{ImageName: "postgres:17", Options: opts},
}
)
@@ -733,10 +730,10 @@ func Test_computeLineFromPos(t *testing.T) {
t.Run(name, func(t *testing.T) {
input := tc.input
if crlf {
input = strings.Replace(input, "\n", "\r\n", -1)
input = strings.ReplaceAll(input, "\n", "\r\n")
}
if nonASCII {
input = strings.Replace(input, "FROM", "FRÖM", -1)
input = strings.ReplaceAll(input, "FROM", "FRÖM")
}
gotLine, gotCol, gotOK := computeLineFromPos(input, tc.pos)
+9 -15
View File
@@ -1,25 +1,23 @@
//go:build go1.9
// +build go1.9
package postgres
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"regexp"
"strconv"
"strings"
"sync/atomic"
"time"
"go.uber.org/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/golang-migrate/migrate/v4/database/multistmt"
"github.com/hashicorp/go-multierror"
"github.com/lib/pq"
)
@@ -167,7 +165,7 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
if s := purl.Query().Get("x-migrations-table-quoted"); len(s) > 0 {
migrationsTableQuoted, err = strconv.ParseBool(s)
if err != nil {
return nil, fmt.Errorf("Unable to parse option x-migrations-table-quoted: %w", err)
return nil, fmt.Errorf("unable to parse option x-migrations-table-quoted: %w", err)
}
}
if (len(migrationsTable) > 0) && (migrationsTableQuoted) && ((migrationsTable[0] != '"') || (migrationsTable[len(migrationsTable)-1] != '"')) {
@@ -198,7 +196,7 @@ func (p *Postgres) Open(url string) (database.Driver, error) {
if s := purl.Query().Get("x-multi-statement"); len(s) > 0 {
multiStatementEnabled, err = strconv.ParseBool(s)
if err != nil {
return nil, fmt.Errorf("Unable to parse option x-multi-statement: %w", err)
return nil, fmt.Errorf("unable to parse option x-multi-statement: %w", err)
}
}
@@ -321,7 +319,7 @@ func (p *Postgres) runStatement(statement []byte) error {
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
// replace crlf with lf
s = strings.Replace(s, "\r\n", "\n", -1)
s = strings.ReplaceAll(s, "\r\n", "\n")
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
runes := []rune(s)
if pos > len(runes) {
@@ -363,7 +361,7 @@ func (p *Postgres) SetVersion(version int, dirty bool) error {
query := `TRUNCATE ` + pq.QuoteIdentifier(p.config.migrationsSchemaName) + `.` + pq.QuoteIdentifier(p.config.migrationsTableName)
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -375,7 +373,7 @@ func (p *Postgres) SetVersion(version int, dirty bool) error {
query = `INSERT INTO ` + pq.QuoteIdentifier(p.config.migrationsSchemaName) + `.` + pq.QuoteIdentifier(p.config.migrationsTableName) + ` (version, dirty) VALUES ($1, $2)`
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -417,7 +415,7 @@ func (p *Postgres) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -459,11 +457,7 @@ func (p *Postgres) ensureVersionTable() (err error) {
defer func() {
if e := p.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+46 -20
View File
@@ -35,11 +35,11 @@ var (
PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.postgresql.org/support/versioning/
specs = []dktesting.ContainerSpec{
{ImageName: "postgres:9.5", Options: opts},
{ImageName: "postgres:9.6", Options: opts},
{ImageName: "postgres:10", Options: opts},
{ImageName: "postgres:11", Options: opts},
{ImageName: "postgres:12", Options: opts},
{ImageName: "postgres:13", Options: opts},
{ImageName: "postgres:14", Options: opts},
{ImageName: "postgres:15", Options: opts},
{ImageName: "postgres:16", Options: opts},
{ImageName: "postgres:17", Options: opts},
}
)
@@ -85,6 +85,32 @@ func mustRun(t *testing.T, d database.Driver, statements []string) {
}
func Test(t *testing.T) {
t.Run("test", test)
t.Run("testMigrate", testMigrate)
t.Run("testMultipleStatements", testMultipleStatements)
t.Run("testMultipleStatementsInMultiStatementMode", testMultipleStatementsInMultiStatementMode)
t.Run("testErrorParsing", testErrorParsing)
t.Run("testFilterCustomQuery", testFilterCustomQuery)
t.Run("testWithSchema", testWithSchema)
t.Run("testMigrationTableOption", testMigrationTableOption)
t.Run("testFailToCreateTableWithoutPermissions", testFailToCreateTableWithoutPermissions)
t.Run("testCheckBeforeCreateTable", testCheckBeforeCreateTable)
t.Run("testParallelSchema", testParallelSchema)
t.Run("testPostgresLock", testPostgresLock)
t.Run("testWithInstanceConcurrent", testWithInstanceConcurrent)
t.Run("testWithConnection", testWithConnection)
t.Cleanup(func() {
for _, spec := range specs {
t.Log("Cleaning up ", spec.ImageName)
if err := spec.Cleanup(); err != nil {
t.Error("Error removing ", spec.ImageName, "error:", err)
}
}
})
}
func test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -106,7 +132,7 @@ func Test(t *testing.T) {
})
}
func TestMigrate(t *testing.T) {
func testMigrate(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -132,7 +158,7 @@ func TestMigrate(t *testing.T) {
})
}
func TestMultipleStatements(t *testing.T) {
func testMultipleStatements(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -165,7 +191,7 @@ func TestMultipleStatements(t *testing.T) {
})
}
func TestMultipleStatementsInMultiStatementMode(t *testing.T) {
func testMultipleStatementsInMultiStatementMode(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -198,7 +224,7 @@ func TestMultipleStatementsInMultiStatementMode(t *testing.T) {
})
}
func TestErrorParsing(t *testing.T) {
func testErrorParsing(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -227,7 +253,7 @@ func TestErrorParsing(t *testing.T) {
})
}
func TestFilterCustomQuery(t *testing.T) {
func testFilterCustomQuery(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -249,7 +275,7 @@ func TestFilterCustomQuery(t *testing.T) {
})
}
func TestWithSchema(t *testing.T) {
func testWithSchema(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -319,7 +345,7 @@ func TestWithSchema(t *testing.T) {
})
}
func TestMigrationTableOption(t *testing.T) {
func testMigrationTableOption(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -387,7 +413,7 @@ func TestMigrationTableOption(t *testing.T) {
})
}
func TestFailToCreateTableWithoutPermissions(t *testing.T) {
func testFailToCreateTableWithoutPermissions(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -457,7 +483,7 @@ func TestFailToCreateTableWithoutPermissions(t *testing.T) {
})
}
func TestCheckBeforeCreateTable(t *testing.T) {
func testCheckBeforeCreateTable(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -534,7 +560,7 @@ func TestCheckBeforeCreateTable(t *testing.T) {
})
}
func TestParallelSchema(t *testing.T) {
func testParallelSchema(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -602,7 +628,7 @@ func TestParallelSchema(t *testing.T) {
})
}
func TestPostgres_Lock(t *testing.T) {
func testPostgresLock(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -642,7 +668,7 @@ func TestPostgres_Lock(t *testing.T) {
})
}
func TestWithInstance_Concurrent(t *testing.T) {
func testWithInstanceConcurrent(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -685,7 +711,7 @@ func TestWithInstance_Concurrent(t *testing.T) {
})
}
func TestWithConnection(t *testing.T) {
func testWithConnection(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
@@ -772,10 +798,10 @@ func Test_computeLineFromPos(t *testing.T) {
t.Run(name, func(t *testing.T) {
input := tc.input
if crlf {
input = strings.Replace(input, "\n", "\r\n", -1)
input = strings.ReplaceAll(input, "\n", "\r\n")
}
if nonASCII {
input = strings.Replace(input, "FROM", "FRÖM", -1)
input = strings.ReplaceAll(input, "FROM", "FRÖM")
}
gotLine, gotCol, gotOK := computeLineFromPos(input, tc.pos)
+8 -13
View File
@@ -2,13 +2,12 @@ package ql
import (
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strings"
"github.com/hashicorp/go-multierror"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
@@ -72,11 +71,7 @@ func (m *Ql) ensureVersionTable() (err error) {
defer func() {
if e := m.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -133,7 +128,7 @@ func (m *Ql) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -166,13 +161,13 @@ func (m *Ql) Drop() (err error) {
return nil
}
func (m *Ql) Lock() error {
if !m.isLocked.CAS(false, true) {
if !m.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (m *Ql) Unlock() error {
if !m.isLocked.CAS(true, false) {
if !m.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -193,7 +188,7 @@ func (m *Ql) executeQuery(query string) error {
}
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -221,7 +216,7 @@ func (m *Ql) SetVersion(version int, dirty bool) error {
m.config.MigrationsTable)
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
+9 -15
View File
@@ -1,22 +1,20 @@
//go:build go1.9
// +build go1.9
package redshift
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
"github.com/lib/pq"
)
@@ -128,14 +126,14 @@ func (p *Redshift) Close() error {
// Redshift does not support advisory lock functions: https://docs.aws.amazon.com/redshift/latest/dg/c_unsupported-postgresql-functions.html
func (p *Redshift) Lock() error {
if !p.isLocked.CAS(false, true) {
if !p.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (p *Redshift) Unlock() error {
if !p.isLocked.CAS(true, false) {
if !p.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -176,7 +174,7 @@ func (p *Redshift) Run(migration io.Reader) error {
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
// replace crlf with lf
s = strings.Replace(s, "\r\n", "\n", -1)
s = strings.ReplaceAll(s, "\r\n", "\n")
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
runes := []rune(s)
if pos > len(runes) {
@@ -218,7 +216,7 @@ func (p *Redshift) SetVersion(version int, dirty bool) error {
query := `DELETE FROM "` + p.config.MigrationsTable + `"`
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -230,7 +228,7 @@ func (p *Redshift) SetVersion(version int, dirty bool) error {
query = `INSERT INTO "` + p.config.MigrationsTable + `" (version, dirty) VALUES ($1, $2)`
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -272,7 +270,7 @@ func (p *Redshift) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -314,11 +312,7 @@ func (p *Redshift) ensureVersionTable() (err error) {
defer func() {
if e := p.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+16 -7
View File
@@ -28,10 +28,19 @@ import (
_ "github.com/golang-migrate/migrate/v4/source/file"
)
const (
pgPassword = "redshift"
)
var (
opts = dktest.Options{PortRequired: true, ReadyFunc: isReady}
opts = dktest.Options{
Env: map[string]string{"POSTGRES_PASSWORD": pgPassword},
PortRequired: true,
ReadyFunc: isReady,
}
specs = []dktesting.ContainerSpec{
{ImageName: "postgres:8", Options: opts},
{ImageName: "migrate/postgres8:8", Options: opts},
}
)
@@ -44,7 +53,7 @@ func pgConnectionString(host, port string) string {
}
func connectionString(schema, host, port string) string {
return fmt.Sprintf("%s://postgres@%s:%s/postgres?sslmode=disable", schema, host, port)
return fmt.Sprintf("%s://postgres:%s@%s:%s/postgres?sslmode=disable", schema, pgPassword, host, port)
}
func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
@@ -192,7 +201,7 @@ func TestFilterCustomQuery(t *testing.T) {
t.Fatal(err)
}
addr := fmt.Sprintf("postgres://postgres@%v:%v/postgres?sslmode=disable&x-custom=foobar", ip, port)
addr := fmt.Sprintf("postgres://postgres:%s@%v:%v/postgres?sslmode=disable&x-custom=foobar", pgPassword, ip, port)
p := &Redshift{}
d, err := p.Open(addr)
if err != nil {
@@ -234,7 +243,7 @@ func TestWithSchema(t *testing.T) {
}
// re-connect using that schema
d2, err := p.Open(fmt.Sprintf("postgres://postgres@%v:%v/postgres?sslmode=disable&search_path=foobar", ip, port))
d2, err := p.Open(fmt.Sprintf("postgres://postgres:%s@%v:%v/postgres?sslmode=disable&search_path=foobar", pgPassword, ip, port))
if err != nil {
t.Fatal(err)
}
@@ -369,10 +378,10 @@ func Test_computeLineFromPos(t *testing.T) {
t.Run(name, func(t *testing.T) {
input := tc.input
if crlf {
input = strings.Replace(input, "\n", "\r\n", -1)
input = strings.ReplaceAll(input, "\n", "\r\n")
}
if nonASCII {
input = strings.Replace(input, "FROM", "FRÖM", -1)
input = strings.ReplaceAll(input, "FROM", "FRÖM")
}
gotLine, gotCol, gotOK := computeLineFromPos(input, tc.pos)
+8 -14
View File
@@ -1,18 +1,16 @@
package rqlite
import (
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"github.com/rqlite/gorqlite"
)
@@ -90,11 +88,7 @@ func (r *Rqlite) ensureVersionTable() (err error) {
defer func() {
if e := r.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -144,7 +138,7 @@ func (r *Rqlite) Close() error {
// If the implementation can't provide this functionality, return nil.
// Return database.ErrLocked if database is already locked.
func (r *Rqlite) Lock() error {
if !r.isLocked.CAS(false, true) {
if !r.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
@@ -153,7 +147,7 @@ func (r *Rqlite) Lock() error {
// Unlock should release the lock. Migrate will call this function after
// all migrations have been run.
func (r *Rqlite) Unlock() error {
if !r.isLocked.CAS(true, false) {
if !r.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -292,7 +286,7 @@ func parseUrl(url string) (*nurl.URL, *Config, error) {
}
if parsedUrl.Scheme != "rqlite" {
return nil, nil, errors.Wrap(ErrBadConfig, "bad scheme")
return nil, nil, fmt.Errorf("bad scheme: %w", ErrBadConfig)
}
// adapt from rqlite to http/https schemes
@@ -316,7 +310,7 @@ func parseConfigFromQuery(queryVals nurl.Values) (*Config, error) {
migrationsTable := queryVals.Get("x-migrations-table")
if migrationsTable != "" {
if strings.HasPrefix(migrationsTable, "sqlite_") {
return nil, errors.Wrap(ErrBadConfig, "invalid value for x-migrations-table")
return nil, fmt.Errorf("invalid value for x-migrations-table: %w", ErrBadConfig)
}
c.MigrationsTable = migrationsTable
}
@@ -325,7 +319,7 @@ func parseConfigFromQuery(queryVals nurl.Values) (*Config, error) {
if connectInsecureStr != "" {
connectInsecure, err := strconv.ParseBool(connectInsecureStr)
if err != nil {
return nil, errors.Wrap(ErrBadConfig, "invalid value for x-connect-insecure")
return nil, fmt.Errorf("invalid value for x-connect-insecure: %w", ErrBadConfig)
}
c.ConnectInsecure = connectInsecure
}
+9 -14
View File
@@ -3,16 +3,15 @@ package snowflake
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
"github.com/lib/pq"
sf "github.com/snowflakedb/gosnowflake"
)
@@ -159,14 +158,14 @@ func (p *Snowflake) Close() error {
}
func (p *Snowflake) Lock() error {
if !p.isLocked.CAS(false, true) {
if !p.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (p *Snowflake) Unlock() error {
if !p.isLocked.CAS(true, false) {
if !p.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -207,7 +206,7 @@ func (p *Snowflake) Run(migration io.Reader) error {
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
// replace crlf with lf
s = strings.Replace(s, "\r\n", "\n", -1)
s = strings.ReplaceAll(s, "\r\n", "\n")
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
runes := []rune(s)
if pos > len(runes) {
@@ -249,7 +248,7 @@ func (p *Snowflake) SetVersion(version int, dirty bool) error {
query := `DELETE FROM "` + p.config.MigrationsTable + `"`
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -263,7 +262,7 @@ func (p *Snowflake) SetVersion(version int, dirty bool) error {
` + strconv.FormatBool(dirty) + `)`
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -305,7 +304,7 @@ func (p *Snowflake) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -347,11 +346,7 @@ func (p *Snowflake) ensureVersionTable() (err error) {
defer func() {
if e := p.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+5 -16
View File
@@ -10,6 +10,7 @@ import (
"regexp"
"strconv"
"strings"
"sync/atomic"
"cloud.google.com/go/spanner"
sdb "cloud.google.com/go/spanner/admin/database/apiv1"
@@ -19,8 +20,6 @@ import (
"github.com/golang-migrate/migrate/v4/database"
adminpb "cloud.google.com/go/spanner/admin/database/apiv1/databasepb"
"github.com/hashicorp/go-multierror"
uatomic "go.uber.org/atomic"
"google.golang.org/api/iterator"
)
@@ -32,11 +31,6 @@ func init() {
// DefaultMigrationsTable is used if no custom table is specified
const DefaultMigrationsTable = "SchemaMigrations"
const (
unlockedVal = 0
lockedVal = 1
)
// Driver errors
var (
ErrNilConfig = errors.New("no config")
@@ -64,7 +58,7 @@ type Spanner struct {
config *Config
lock *uatomic.Uint32
lock atomic.Bool
}
type DB struct {
@@ -96,7 +90,6 @@ func WithInstance(instance *DB, config *Config) (database.Driver, error) {
sx := &Spanner{
db: instance,
config: config,
lock: uatomic.NewUint32(unlockedVal),
}
if err := sx.ensureVersionTable(); err != nil {
@@ -153,7 +146,7 @@ func (s *Spanner) Close() error {
// Lock implements database.Driver but doesn't do anything because Spanner only
// enqueues the UpdateDatabaseDdlRequest.
func (s *Spanner) Lock() error {
if swapped := s.lock.CAS(unlockedVal, lockedVal); swapped {
if swapped := s.lock.CompareAndSwap(false, true); swapped {
return nil
}
return ErrLockHeld
@@ -161,7 +154,7 @@ func (s *Spanner) Lock() error {
// Unlock implements database.Driver but no action required, see Lock.
func (s *Spanner) Unlock() error {
if swapped := s.lock.CAS(lockedVal, unlockedVal); swapped {
if swapped := s.lock.CompareAndSwap(true, false); swapped {
return nil
}
return ErrLockNotHeld
@@ -305,11 +298,7 @@ func (s *Spanner) ensureVersionTable() (err error) {
defer func() {
if e := s.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
+8 -13
View File
@@ -2,17 +2,16 @@ package sqlcipher
import (
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
_ "github.com/mutecomm/go-sqlcipher/v4"
)
@@ -73,11 +72,7 @@ func (m *Sqlite) ensureVersionTable() (err error) {
defer func() {
if e := m.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -141,7 +136,7 @@ func (m *Sqlite) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -178,14 +173,14 @@ func (m *Sqlite) Drop() (err error) {
}
func (m *Sqlite) Lock() error {
if !m.isLocked.CAS(false, true) {
if !m.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (m *Sqlite) Unlock() error {
if !m.isLocked.CAS(true, false) {
if !m.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -211,7 +206,7 @@ func (m *Sqlite) executeQuery(query string) error {
}
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -246,7 +241,7 @@ func (m *Sqlite) SetVersion(version int, dirty bool) error {
query := fmt.Sprintf(`INSERT INTO %s (version, dirty) VALUES (?, ?)`, m.config.MigrationsTable)
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
+8 -13
View File
@@ -2,17 +2,16 @@ package sqlite
import (
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
_ "modernc.org/sqlite"
)
@@ -73,11 +72,7 @@ func (m *Sqlite) ensureVersionTable() (err error) {
defer func() {
if e := m.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -141,7 +136,7 @@ func (m *Sqlite) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -178,14 +173,14 @@ func (m *Sqlite) Drop() (err error) {
}
func (m *Sqlite) Lock() error {
if !m.isLocked.CAS(false, true) {
if !m.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (m *Sqlite) Unlock() error {
if !m.isLocked.CAS(true, false) {
if !m.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -211,7 +206,7 @@ func (m *Sqlite) executeQuery(query string) error {
}
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -246,7 +241,7 @@ func (m *Sqlite) SetVersion(version int, dirty bool) error {
query := fmt.Sprintf(`INSERT INTO %s (version, dirty) VALUES (?, ?)`, m.config.MigrationsTable)
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
+8 -13
View File
@@ -2,17 +2,16 @@ package sqlite3
import (
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
_ "github.com/mattn/go-sqlite3"
)
@@ -73,11 +72,7 @@ func (m *Sqlite) ensureVersionTable() (err error) {
defer func() {
if e := m.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -141,7 +136,7 @@ func (m *Sqlite) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -178,14 +173,14 @@ func (m *Sqlite) Drop() (err error) {
}
func (m *Sqlite) Lock() error {
if !m.isLocked.CAS(false, true) {
if !m.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (m *Sqlite) Unlock() error {
if !m.isLocked.CAS(true, false) {
if !m.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
@@ -211,7 +206,7 @@ func (m *Sqlite) executeQuery(query string) error {
}
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -246,7 +241,7 @@ func (m *Sqlite) SetVersion(version int, dirty bool) error {
query := fmt.Sprintf(`INSERT INTO %s (version, dirty) VALUES (?, ?)`, m.config.MigrationsTable)
if _, err := tx.Exec(query, version, dirty); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
+28 -23
View File
@@ -3,18 +3,17 @@ package sqlserver
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
nurl "net/url"
"strconv"
"strings"
"go.uber.org/atomic"
"sync/atomic"
"github.com/Azure/go-autorest/autorest/adal"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
mssql "github.com/microsoft/go-mssqldb" // mssql support
)
@@ -30,10 +29,10 @@ var (
ErrNoDatabaseName = fmt.Errorf("no database name")
ErrNoSchema = fmt.Errorf("no schema")
ErrDatabaseDirty = fmt.Errorf("database is dirty")
ErrMultipleAuthOptionsPassed = fmt.Errorf("both password and useMsi=true were passed.")
ErrMultipleAuthOptionsPassed = fmt.Errorf("both password and useMsi=true were passed")
)
var lockErrorMap = map[mssql.ReturnStatus]string{
var lockErrorMap = map[int]string{
-1: "The lock request timed out.",
-2: "The lock request was canceled.",
-3: "The lock request was chosen as a deadlock victim.",
@@ -198,18 +197,24 @@ func (ss *SQLServer) Lock() error {
return err
}
// This will either obtain the lock immediately and return true,
// or return false if the lock cannot be acquired immediately.
// This will block until the lock is acquired.
// MS Docs: sp_getapplock: https://docs.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-getapplock-transact-sql?view=sql-server-2017
query := `EXEC sp_getapplock @Resource = @p1, @LockMode = 'Update', @LockOwner = 'Session', @LockTimeout = 0`
query := `
DECLARE @lockResult int;
EXEC @lockResult = sp_getapplock @Resource = @p1, @LockMode = 'Exclusive', @LockOwner = 'Session', @LockTimeout = -1;
SELECT @lockResult;`
var status mssql.ReturnStatus
if _, err = ss.conn.ExecContext(context.Background(), query, aid, &status); err == nil && status > -1 {
var status int
if err = ss.conn.QueryRowContext(context.Background(), query, aid).Scan(&status); err == nil && status > -1 {
return nil
} else if err != nil {
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
} else {
return &database.Error{Err: fmt.Sprintf("try lock failed with error %v: %v", status, lockErrorMap[status]), Query: []byte(query)}
errorDescription, ok := lockErrorMap[status]
if !ok {
errorDescription = "Unknown error"
}
return &database.Error{Err: fmt.Sprintf("try lock failed with error %v: %v", status, errorDescription), Query: []byte(query)}
}
})
}
@@ -263,10 +268,10 @@ func (ss *SQLServer) SetVersion(version int, dirty bool) error {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
query := `TRUNCATE TABLE "` + ss.config.MigrationsTable + `"`
query := `TRUNCATE TABLE ` + ss.getMigrationTable()
if _, err := tx.Exec(query); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -279,10 +284,10 @@ func (ss *SQLServer) SetVersion(version int, dirty bool) error {
if dirty {
dirtyBit = 1
}
query = `INSERT INTO "` + ss.config.MigrationsTable + `" (version, dirty) VALUES (@p1, @p2)`
query = `INSERT INTO ` + ss.getMigrationTable() + ` (version, dirty) VALUES (@p1, @p2)`
if _, err := tx.Exec(query, version, dirtyBit); err != nil {
if errRollback := tx.Rollback(); errRollback != nil {
err = multierror.Append(err, errRollback)
err = errors.Join(err, errRollback)
}
return &database.Error{OrigErr: err, Query: []byte(query)}
}
@@ -297,7 +302,7 @@ func (ss *SQLServer) SetVersion(version int, dirty bool) error {
// Version of the current database state
func (ss *SQLServer) Version() (version int, dirty bool, err error) {
query := `SELECT TOP 1 version, dirty FROM "` + ss.config.MigrationsTable + `"`
query := `SELECT TOP 1 version, dirty FROM ` + ss.getMigrationTable()
err = ss.conn.QueryRowContext(context.Background(), query).Scan(&version, &dirty)
switch {
case err == sql.ErrNoRows:
@@ -354,21 +359,17 @@ func (ss *SQLServer) ensureVersionTable() (err error) {
defer func() {
if e := ss.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
query := `IF NOT EXISTS
(SELECT *
FROM sysobjects
WHERE id = object_id(N'[dbo].[` + ss.config.MigrationsTable + `]')
WHERE id = object_id(N'` + ss.getMigrationTable() + `')
AND OBJECTPROPERTY(id, N'IsUserTable') = 1
)
CREATE TABLE ` + ss.config.MigrationsTable + ` ( version BIGINT PRIMARY KEY NOT NULL, dirty BIT NOT NULL );`
CREATE TABLE ` + ss.getMigrationTable() + ` ( version BIGINT PRIMARY KEY NOT NULL, dirty BIT NOT NULL );`
if _, err = ss.conn.ExecContext(context.Background(), query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
@@ -377,6 +378,10 @@ func (ss *SQLServer) ensureVersionTable() (err error) {
return nil
}
func (ss *SQLServer) getMigrationTable() string {
return fmt.Sprintf("[%s].[%s]", ss.config.SchemaName, ss.config.MigrationsTable)
}
func getMSITokenProvider(resource string) (func() (string, error), error) {
msi, err := adal.NewServicePrincipalTokenFromManagedIdentity(resource, nil)
if err != nil {
+30 -22
View File
@@ -12,7 +12,6 @@ import (
"time"
"github.com/dhui/dktest"
"github.com/docker/go-connections/nat"
"github.com/golang-migrate/migrate/v4"
dt "github.com/golang-migrate/migrate/v4/database/testing"
@@ -25,27 +24,16 @@ const defaultPort = 1433
const saPassword = "Root1234"
var (
sqlEdgeOpts = dktest.Options{
Env: map[string]string{"ACCEPT_EULA": "Y", "MSSQL_SA_PASSWORD": saPassword},
PortBindings: map[nat.Port][]nat.PortBinding{
nat.Port(fmt.Sprintf("%d/tcp", defaultPort)): {
nat.PortBinding{
HostIP: "0.0.0.0",
HostPort: "0/tcp",
},
},
},
PortRequired: true, ReadyFunc: isReady, PullTimeout: 2 * time.Minute,
}
sqlServerOpts = dktest.Options{
Env: map[string]string{"ACCEPT_EULA": "Y", "MSSQL_SA_PASSWORD": saPassword, "MSSQL_PID": "Express"},
PortRequired: true, ReadyFunc: isReady, PullTimeout: 2 * time.Minute,
}
// Container versions: https://mcr.microsoft.com/v2/mssql/server/tags/list
specs = []dktesting.ContainerSpec{
{ImageName: "mcr.microsoft.com/azure-sql-edge:latest", Options: sqlEdgeOpts},
{ImageName: "mcr.microsoft.com/mssql/server:2017-latest", Options: sqlServerOpts},
{ImageName: "mcr.microsoft.com/mssql/server:2022-latest", Options: sqlServerOpts},
{ImageName: "mcr.microsoft.com/mssql/server:2019-latest", Options: sqlServerOpts},
// Add back support for 2017 version once the image is fixed: https://github.com/microsoft/mssql-docker/issues/899
// {ImageName: "mcr.microsoft.com/mssql/server:2017-latest", Options: sqlServerOpts},
}
)
@@ -96,6 +84,26 @@ func SkipIfUnsupportedArch(t *testing.T, c dktest.ContainerInfo) {
}
func Test(t *testing.T) {
t.Run("test", test)
t.Run("testMigrate", testMigrate)
t.Run("testMultiStatement", testMultiStatement)
t.Run("testErrorParsing", testErrorParsing)
t.Run("testLockWorks", testLockWorks)
t.Run("testMsiTrue", testMsiTrue)
t.Run("testOpenWithPasswordAndMSI", testOpenWithPasswordAndMSI)
t.Run("testMsiFalse", testMsiFalse)
t.Cleanup(func() {
for _, spec := range specs {
t.Log("Cleaning up ", spec.ImageName)
if err := spec.Cleanup(); err != nil {
t.Error("Error removing ", spec.ImageName, "error:", err)
}
}
})
}
func test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -120,7 +128,7 @@ func Test(t *testing.T) {
})
}
func TestMigrate(t *testing.T) {
func testMigrate(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -149,7 +157,7 @@ func TestMigrate(t *testing.T) {
})
}
func TestMultiStatement(t *testing.T) {
func testMultiStatement(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -183,7 +191,7 @@ func TestMultiStatement(t *testing.T) {
})
}
func TestErrorParsing(t *testing.T) {
func testErrorParsing(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -215,7 +223,7 @@ func TestErrorParsing(t *testing.T) {
})
}
func TestLockWorks(t *testing.T) {
func testLockWorks(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -254,7 +262,7 @@ func TestLockWorks(t *testing.T) {
})
}
func TestMsiTrue(t *testing.T) {
func testMsiTrue(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -271,7 +279,7 @@ func TestMsiTrue(t *testing.T) {
})
}
func TestOpenWithPasswordAndMSI(t *testing.T) {
func testOpenWithPasswordAndMSI(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
@@ -303,7 +311,7 @@ func TestOpenWithPasswordAndMSI(t *testing.T) {
})
}
func TestMsiFalse(t *testing.T) {
func testMsiFalse(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
SkipIfUnsupportedArch(t, c)
ip, port, err := c.Port(defaultPort)
+3 -4
View File
@@ -3,8 +3,7 @@ package stub
import (
"io"
"reflect"
"go.uber.org/atomic"
"sync/atomic"
"github.com/golang-migrate/migrate/v4/database"
)
@@ -50,14 +49,14 @@ func (s *Stub) Close() error {
}
func (s *Stub) Lock() error {
if !s.isLocked.CAS(false, true) {
if !s.isLocked.CompareAndSwap(false, true) {
return database.ErrLocked
}
return nil
}
func (s *Stub) Unlock() error {
if !s.isLocked.CAS(true, false) {
if !s.isLocked.CompareAndSwap(true, false) {
return database.ErrNotLocked
}
return nil
+1 -2
View File
@@ -50,7 +50,7 @@ func TestLockAndUnlock(t *testing.T, d database.Driver) {
case <-done:
return
case <-timeout:
errs <- fmt.Errorf("Timeout after 15 seconds. Looks like a deadlock in Lock/UnLock.\n%#v", d)
errs <- fmt.Errorf("timeout after 15 seconds, looks like a deadlock in Lock/UnLock\n%#v", d)
return
}
}
@@ -116,7 +116,6 @@ func TestDrop(t *testing.T, d database.Driver) {
}
func TestSetVersion(t *testing.T, d database.Driver) {
// nolint:maligned
testCases := []struct {
name string
version int
+2 -2
View File
@@ -2,9 +2,9 @@ package database
import (
"fmt"
"go.uber.org/atomic"
"hash/crc32"
"strings"
"sync/atomic"
)
const advisoryLockIDSalt uint = 1486364155
@@ -21,7 +21,7 @@ func GenerateAdvisoryLockId(databaseName string, additionalNames ...string) (str
// CasRestoreOnErr CAS wrapper to automatically restore the lock state on error
func CasRestoreOnErr(lock *atomic.Bool, o, n bool, casErr error, f func() error) error {
if !lock.CAS(o, n) {
if !lock.CompareAndSwap(o, n) {
return casErr
}
if err := f(); err != nil {
+9 -7
View File
@@ -2,7 +2,7 @@ package database
import (
"errors"
"go.uber.org/atomic"
"sync/atomic"
"testing"
)
@@ -54,7 +54,7 @@ func TestCasRestoreOnErr(t *testing.T) {
testcases := []struct {
name string
lock *atomic.Bool
lock bool
from bool
to bool
expectLock bool
@@ -63,7 +63,7 @@ func TestCasRestoreOnErr(t *testing.T) {
}{
{
name: "Test positive CAS lock",
lock: atomic.NewBool(false),
lock: false,
from: false,
to: true,
expectLock: true,
@@ -72,7 +72,7 @@ func TestCasRestoreOnErr(t *testing.T) {
},
{
name: "Test negative CAS lock",
lock: atomic.NewBool(true),
lock: true,
from: false,
to: true,
expectLock: true,
@@ -81,7 +81,7 @@ func TestCasRestoreOnErr(t *testing.T) {
},
{
name: "Test negative with callback lock",
lock: atomic.NewBool(false),
lock: false,
from: false,
to: true,
expectLock: false,
@@ -92,13 +92,15 @@ func TestCasRestoreOnErr(t *testing.T) {
for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
if err := CasRestoreOnErr(tc.lock, tc.from, tc.to, casErr, func() error {
var lock atomic.Bool
lock.Store(tc.lock)
if err := CasRestoreOnErr(&lock, tc.from, tc.to, casErr, func() error {
return tc.fErr
}); err != tc.expectError {
t.Error("Incorrect error value returned")
}
if tc.lock.Load() != tc.expectLock {
if lock.Load() != tc.expectLock {
t.Error("Incorrect state of lock")
}
})
+7 -11
View File
@@ -8,16 +8,15 @@ import (
"net/url"
"regexp"
"strconv"
"sync/atomic"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/hashicorp/go-multierror"
"github.com/jackc/pgconn"
"github.com/jackc/pgerrcode"
"github.com/lib/pq"
"go.uber.org/atomic"
)
const (
@@ -206,7 +205,7 @@ func (c *YugabyteDB) Lock() error {
}
defer func() {
if errClose := rows.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -320,7 +319,7 @@ func (c *YugabyteDB) Drop() (err error) {
}
defer func() {
if errClose := tables.Close(); errClose != nil {
err = multierror.Append(err, errClose)
err = errors.Join(err, errClose)
}
}()
@@ -360,11 +359,7 @@ func (c *YugabyteDB) ensureVersionTable() (err error) {
defer func() {
if e := c.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
err = errors.Join(err, e)
}
}()
@@ -420,8 +415,9 @@ func (c *YugabyteDB) doTxWithRetry(
}
// If we've tried to commit the transaction Rollback just returns sql.ErrTxDone.
//nolint:errcheck
defer tx.Rollback()
defer func() {
_ = tx.Rollback()
}()
if err := fn(tx); err != nil {
if errIsRetryable(err) {
+22 -8
View File
@@ -31,12 +31,10 @@ var (
ReadyFunc: isReady,
Timeout: time.Duration(60) * time.Second,
}
// Released versions: https://docs.yugabyte.com/latest/releases/#current-supported-releases
// Released versions: https://docs.yugabyte.com/preview/releases/release-notes/
specs = []dktesting.ContainerSpec{
{ImageName: "yugabytedb/yugabyte:2.6.16.0-b14", Options: opts},
{ImageName: "yugabytedb/yugabyte:2.8.4.0-b30", Options: opts},
{ImageName: "yugabytedb/yugabyte:2.12.2.0-b58", Options: opts},
{ImageName: "yugabytedb/yugabyte:2.13.0.1-b2", Options: opts},
{ImageName: "yugabytedb/yugabyte:2.14.15.0-b57", Options: opts},
{ImageName: "yugabytedb/yugabyte:2.20.2.1-b3", Options: opts},
}
)
@@ -93,6 +91,22 @@ func getConnectionString(ip, port string, options ...string) string {
}
func Test(t *testing.T) {
t.Run("test", test)
t.Run("testMigrate", testMigrate)
t.Run("testMultiStatement", testMultiStatement)
t.Run("testFilterCustomQuery", testFilterCustomQuery)
t.Cleanup(func() {
for _, spec := range specs {
t.Log("Cleaning up ", spec.ImageName)
if err := spec.Cleanup(); err != nil {
t.Error("Error removing ", spec.ImageName, "error:", err)
}
}
})
}
func test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
@@ -111,7 +125,7 @@ func Test(t *testing.T) {
})
}
func TestMigrate(t *testing.T) {
func testMigrate(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
@@ -135,7 +149,7 @@ func TestMigrate(t *testing.T) {
})
}
func TestMultiStatement(t *testing.T) {
func testMultiStatement(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
@@ -165,7 +179,7 @@ func TestMultiStatement(t *testing.T) {
})
}
func TestFilterCustomQuery(t *testing.T) {
func testFilterCustomQuery(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
+28 -2
View File
@@ -1,11 +1,13 @@
package dktesting
import (
"context"
"fmt"
"testing"
)
import (
"github.com/dhui/dktest"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/client"
)
// ContainerSpec holds Docker testing setup specifications
@@ -14,6 +16,30 @@ type ContainerSpec struct {
Options dktest.Options
}
// Cleanup cleanups the ContainerSpec after a test run by removing the ContainerSpec's image
func (s *ContainerSpec) Cleanup() (retErr error) {
// copied from dktest.RunContext()
dc, err := client.NewClientWithOpts(client.FromEnv, client.WithVersion("1.41"))
if err != nil {
return err
}
defer func() {
if err := dc.Close(); err != nil && retErr == nil {
retErr = fmt.Errorf("error closing Docker client: %w", err)
}
}()
cleanupTimeout := s.Options.CleanupTimeout
if cleanupTimeout <= 0 {
cleanupTimeout = dktest.DefaultCleanupTimeout
}
ctx, timeoutCancelFunc := context.WithTimeout(context.Background(), cleanupTimeout)
defer timeoutCancelFunc()
if _, err := dc.ImageRemove(ctx, s.ImageName, image.RemoveOptions{Force: true, PruneChildren: true}); err != nil {
return err
}
return nil
}
// ParallelTest runs Docker tests in parallel
func ParallelTest(t *testing.T, specs []ContainerSpec,
testFunc func(*testing.T, dktest.ContainerInfo)) {
+101 -68
View File
@@ -1,54 +1,96 @@
module github.com/golang-migrate/migrate/v4
go 1.18
go 1.26.4
require (
cloud.google.com/go/spanner v1.51.0
cloud.google.com/go/storage v1.30.1
cloud.google.com/go/spanner v1.85.0
cloud.google.com/go/storage v1.56.0
github.com/Azure/go-autorest/autorest/adal v0.9.16
github.com/ClickHouse/clickhouse-go v1.4.3
github.com/aws/aws-sdk-go v1.49.6
github.com/cenkalti/backoff/v4 v4.1.2
github.com/cenkalti/backoff/v4 v4.3.0
github.com/cockroachdb/cockroach-go/v2 v2.1.1
github.com/dhui/dktest v0.4.0
github.com/docker/docker v24.0.7+incompatible
github.com/docker/go-connections v0.4.0
github.com/dhui/dktest v0.4.6
github.com/docker/docker v28.3.3+incompatible
github.com/fsouza/fake-gcs-server v1.17.0
github.com/go-sql-driver/mysql v1.5.0
github.com/gobuffalo/here v0.6.0
github.com/gocql/gocql v0.0.0-20210515062232-b7ef815b4556
github.com/google/go-github/v39 v39.2.0
github.com/hashicorp/go-multierror v1.1.1
github.com/jackc/pgconn v1.14.0
github.com/hanzo-ds/go v1.0.1
github.com/jackc/pgconn v1.14.3
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa
github.com/jackc/pgx/v4 v4.18.1
github.com/jackc/pgx/v5 v5.3.1
github.com/jackc/pgx/v4 v4.18.2
github.com/jackc/pgx/v5 v5.7.6
github.com/ktrysmt/go-bitbucket v0.6.4
github.com/lib/pq v1.10.9
github.com/markbates/pkger v0.15.1
github.com/mattn/go-sqlite3 v1.14.16
github.com/mattn/go-sqlite3 v1.14.22
github.com/microsoft/go-mssqldb v1.0.0
github.com/mutecomm/go-sqlcipher/v4 v4.4.0
github.com/nakagami/firebirdsql v0.0.0-20190310045651-3c02a58cfed8
github.com/neo4j/neo4j-go-driver v1.8.1-0.20200803113522-b626aa943eba
github.com/snowflakedb/gosnowflake v1.6.19
github.com/stretchr/testify v1.8.3
github.com/stretchr/testify v1.11.1
github.com/xanzy/go-gitlab v0.15.0
go.mongodb.org/mongo-driver v1.7.5
go.uber.org/atomic v1.7.0
golang.org/x/oauth2 v0.14.0
golang.org/x/tools v0.10.0
google.golang.org/api v0.150.0
go.mongodb.org/mongo-driver v1.11.4
golang.org/x/oauth2 v0.30.0
golang.org/x/tools/godoc v0.1.0-deprecated
google.golang.org/api v0.247.0
modernc.org/ql v1.0.0
modernc.org/sqlite v1.18.1
)
require (
cloud.google.com/go v0.110.10 // indirect
cloud.google.com/go/compute v1.23.3 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.5 // indirect
cloud.google.com/go/longrunning v0.5.4 // indirect
cel.dev/expr v0.24.0 // indirect
cloud.google.com/go/auth v0.16.4 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/monitoring v1.24.2 // indirect
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.5.3 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-faster/city v1.0.1 // indirect
github.com/go-faster/errors v0.7.1 // indirect
github.com/go-jose/go-jose/v4 v4.0.5 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/hanzo-ds/native v0.71.0 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/montanaflynn/stats v0.6.6 // indirect
github.com/paulmach/orb v0.12.0 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/segmentio/asm v1.2.1 // indirect
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
github.com/zeebo/errs v1.4.0 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.36.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.44.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.29.0 // indirect
go.opentelemetry.io/otel/metric v1.44.0 // indirect
go.opentelemetry.io/otel/sdk v1.40.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.40.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/telemetry v0.0.0-20251203150158-8fff8a5912fc // indirect
golang.org/x/tools v0.40.0 // indirect
)
require (
cloud.google.com/go v0.121.6 // indirect
cloud.google.com/go/compute/metadata v0.8.0 // indirect
cloud.google.com/go/iam v1.5.2 // indirect
cloud.google.com/go/longrunning v0.6.7 // indirect
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
github.com/99designs/keyring v1.2.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.4.0 // indirect
@@ -59,8 +101,8 @@ require (
github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
github.com/Azure/go-autorest/logger v0.2.1 // indirect
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/apache/arrow/go/v10 v10.0.1 // indirect
github.com/apache/thrift v0.16.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.16.16 // indirect
@@ -76,55 +118,47 @@ require (
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.17 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.27.11 // indirect
github.com/aws/smithy-go v1.13.3 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/cloudflare/golz4 v0.0.0-20150217214814-ef862a3cdc58 // indirect
github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe // indirect
github.com/cncf/xds/go v0.0.0-20231109132714-523115ebc101 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
github.com/cznic/mathutil v0.0.0-20180504122225-ca4c9f2c1369 // indirect
github.com/danieljoos/wincred v1.1.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dvsekhvalnov/jose2go v1.5.0 // indirect
github.com/dvsekhvalnov/jose2go v1.7.0 // indirect
github.com/edsrzf/mmap-go v0.0.0-20170320065105-0bce6a688712 // indirect
github.com/envoyproxy/go-control-plane v0.11.1 // indirect
github.com/envoyproxy/protoc-gen-validate v1.0.2 // indirect
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
github.com/form3tech-oss/jwt-go v3.2.5+incompatible // indirect
github.com/gabriel-vasile/mimetype v1.4.1 // indirect
github.com/go-stack/stack v1.8.0 // indirect
github.com/goccy/go-json v0.9.11 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe // indirect
github.com/golang-sql/sqlexp v0.1.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/flatbuffers v2.0.8+incompatible // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/google/uuid v1.4.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/gorilla/handlers v1.4.2 // indirect
github.com/gorilla/mux v1.7.4 // indirect
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/jackc/chunkreader/v2 v2.0.1 // indirect
github.com/jackc/pgio v1.0.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgproto3/v2 v2.3.2 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/pgproto3/v2 v2.3.3 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgtype v1.14.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/k0kubun/pp v2.3.0+incompatible // indirect
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/klauspost/compress v1.18.6 // indirect
github.com/klauspost/cpuid/v2 v2.0.9 // indirect
github.com/mattn/go-colorable v0.1.6 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
@@ -137,15 +171,15 @@ require (
github.com/onsi/ginkgo v1.16.4 // indirect
github.com/onsi/gomega v1.15.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/pierrec/lz4/v4 v4.1.16 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/pierrec/lz4/v4 v4.1.27 // indirect
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
github.com/pkg/errors v0.9.1
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect
github.com/rqlite/gorqlite v0.0.0-20230708021416-2acd02b70b79
github.com/shopspring/decimal v1.2.0 // indirect
github.com/sirupsen/logrus v1.9.2 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.1.1 // indirect
github.com/xdg-go/stringprep v1.0.3 // indirect
@@ -153,22 +187,21 @@ require (
github.com/zeebo/xxh3 v1.0.2 // indirect
gitlab.com/nyarla/go-crypt v0.0.0-20160106005555-d9a5dc2b789b // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/crypto v0.17.0 // indirect
golang.org/x/crypto v0.47.0 // indirect
golang.org/x/exp v0.0.0-20230315142452-642cacee5cc0 // indirect
golang.org/x/mod v0.11.0 // indirect
golang.org/x/net v0.18.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/term v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231016165738-49dd2c1f3d0b // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231030173426-d783a09b4405 // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect
golang.org/x/mod v0.31.0 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/term v0.39.0 // indirect
golang.org/x/text v0.33.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250818200422-3122310a409c // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c // indirect
google.golang.org/grpc v1.74.2 // indirect
google.golang.org/protobuf v1.36.7 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/uint128 v1.2.0 // indirect
+1401 -157
View File
File diff suppressed because it is too large Load Diff
-1
View File
@@ -1,5 +1,4 @@
//go:build aws_s3
// +build aws_s3
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build bitbucket
// +build bitbucket
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build cassandra
// +build cassandra
package cli
-9
View File
@@ -1,9 +0,0 @@
//go:build clickhouse
// +build clickhouse
package cli
import (
_ "github.com/ClickHouse/clickhouse-go"
_ "github.com/golang-migrate/migrate/v4/database/clickhouse"
)
-1
View File
@@ -1,5 +1,4 @@
//go:build cockroachdb
// +build cockroachdb
package cli
+7
View File
@@ -0,0 +1,7 @@
//go:build datastore
package cli
import (
_ "github.com/golang-migrate/migrate/v4/database/datastore"
)
-1
View File
@@ -1,5 +1,4 @@
//go:build firebird
// +build firebird
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build github
// +build github
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build github
// +build github
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build gitlab
// +build gitlab
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build go_bindata
// +build go_bindata
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build godoc_vfs
// +build godoc_vfs
package cli
@@ -1,5 +1,4 @@
//go:build google_cloud_storage
// +build google_cloud_storage
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build mongodb
// +build mongodb
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build mysql
// +build mysql
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build neo4j
// +build neo4j
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build pgx
// +build pgx
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build pgx5
// +build pgx5
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build postgres
// +build postgres
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build ql
// +build ql
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build redshift
// +build redshift
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build rqlite
// +build rqlite
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build snowflake
// +build snowflake
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build spanner
// +build spanner
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build sqlcipher
// +build sqlcipher
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build sqlite
// +build sqlite
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build sqlite3
// +build sqlite3
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build sqlserver
// +build sqlserver
package cli
-1
View File
@@ -1,5 +1,4 @@
//go:build yugabytedb
// +build yugabytedb
package cli
+5 -5
View File
@@ -15,9 +15,9 @@ import (
)
var (
errInvalidSequenceWidth = errors.New("Digits must be positive")
errIncompatibleSeqAndFormat = errors.New("The seq and format options are mutually exclusive")
errInvalidTimeFormat = errors.New("Time format may not be empty")
errInvalidSequenceWidth = errors.New("digits must be positive")
errIncompatibleSeqAndFormat = errors.New("the seq and format options are mutually exclusive")
errInvalidTimeFormat = errors.New("time format may not be empty")
)
func nextSeqVersion(matches []string, seqDigits int) (string, error) {
@@ -33,7 +33,7 @@ func nextSeqVersion(matches []string, seqDigits int) (string, error) {
idx := strings.Index(matchSeqStr, "_")
if idx < 1 { // Using 1 instead of 0 since there should be at least 1 digit
return "", fmt.Errorf("Malformed migration filename: %s", filename)
return "", fmt.Errorf("malformed migration filename: %s", filename)
}
var err error
@@ -50,7 +50,7 @@ func nextSeqVersion(matches []string, seqDigits int) (string, error) {
version := fmt.Sprintf("%0[2]*[1]d", nextSeq, seqDigits)
if len(version) > seqDigits {
return "", fmt.Errorf("Next sequence number %s too large. At most %d digits are allowed", version, seqDigits)
return "", fmt.Errorf("next sequence number %s too large, at most %d digits are allowed", version, seqDigits)
}
return version, nil
+6 -6
View File
@@ -84,17 +84,17 @@ func (s *CreateCmdSuite) TestNextSeqVersion() {
}{
{"Bad digits", []string{}, 0, "", errInvalidSequenceWidth},
{"Single digit initialize", []string{}, 1, "1", nil},
{"Single digit malformed", []string{"bad"}, 1, "", errors.New("Malformed migration filename: bad")},
{"Single digit malformed", []string{"bad"}, 1, "", errors.New("malformed migration filename: bad")},
{"Single digit no int", []string{"bad_bad"}, 1, "", errors.New(`strconv.ParseUint: parsing "bad": invalid syntax`)},
{"Single digit negative seq", []string{"-5_test"}, 1, "", errors.New(`strconv.ParseUint: parsing "-5": invalid syntax`)},
{"Single digit increment", []string{"3_test", "4_test"}, 1, "5", nil},
{"Single digit overflow", []string{"9_test"}, 1, "", errors.New("Next sequence number 10 too large. At most 1 digits are allowed")},
{"Single digit overflow", []string{"9_test"}, 1, "", errors.New("next sequence number 10 too large, at most 1 digits are allowed")},
{"Zero-pad initialize", []string{}, 6, "000001", nil},
{"Zero-pad malformed", []string{"bad"}, 6, "", errors.New("Malformed migration filename: bad")},
{"Zero-pad malformed", []string{"bad"}, 6, "", errors.New("malformed migration filename: bad")},
{"Zero-pad no int", []string{"bad_bad"}, 6, "", errors.New(`strconv.ParseUint: parsing "bad": invalid syntax`)},
{"Zero-pad negative seq", []string{"-000005_test"}, 6, "", errors.New(`strconv.ParseUint: parsing "-000005": invalid syntax`)},
{"Zero-pad increment", []string{"000003_test", "000004_test"}, 6, "000005", nil},
{"Zero-pad overflow", []string{"999999_test"}, 6, "", errors.New("Next sequence number 1000000 too large. At most 6 digits are allowed")},
{"Zero-pad overflow", []string{"999999_test"}, 6, "", errors.New("next sequence number 1000000 too large, at most 6 digits are allowed")},
{"dir absolute path", []string{"/migrationDir/000001_test"}, 6, "000002", nil},
{"dir relative path", []string{"migrationDir/000001_test"}, 6, "000002", nil},
{"dir dot prefix", []string{"./migrationDir/000001_test"}, 6, "000002", nil},
@@ -188,11 +188,11 @@ func (s *CreateCmdSuite) TestCreateCmd() {
{"seq init dir double dot relative trailing slash", []string{"subdir"}, "subdir", nil, []string{"subdir/0001_name.up.sql", "subdir/0001_name.down.sql"}, nil, "../subdir/", ts, defaultTimeFormat, true, 4, "sql", "name"},
{"seq init dir maze", []string{"subdir"}, "subdir", nil, []string{"0001_name.up.sql", "0001_name.down.sql"}, nil, "..//subdir/./.././/subdir/..", ts, defaultTimeFormat, true, 4, "sql", "name"},
{"seq width invalid", nil, "", nil, nil, errInvalidSequenceWidth, ".", ts, defaultTimeFormat, true, 0, "sql", "name"},
{"seq malformed", nil, "", []string{"bad.sql"}, []string{"bad.sql"}, errors.New("Malformed migration filename: bad.sql"), ".", ts, defaultTimeFormat, true, 4, "sql", "name"},
{"seq malformed", nil, "", []string{"bad.sql"}, []string{"bad.sql"}, errors.New("malformed migration filename: bad.sql"), ".", ts, defaultTimeFormat, true, 4, "sql", "name"},
{"seq not int", nil, "", []string{"bad_bad.sql"}, []string{"bad_bad.sql"}, errors.New(`strconv.ParseUint: parsing "bad": invalid syntax`), ".", ts, defaultTimeFormat, true, 4, "sql", "name"},
{"seq negative", nil, "", []string{"-5_negative.sql"}, []string{"-5_negative.sql"}, errors.New(`strconv.ParseUint: parsing "-5": invalid syntax`), ".", ts, defaultTimeFormat, true, 4, "sql", "name"},
{"seq increment", nil, "", []string{"3_three.sql", "4_four.sql"}, []string{"3_three.sql", "4_four.sql", "0005_five.up.sql", "0005_five.down.sql"}, nil, ".", ts, defaultTimeFormat, true, 4, "sql", "five"},
{"seq overflow", nil, "", []string{"9_nine.sql"}, []string{"9_nine.sql"}, errors.New(`Next sequence number 10 too large. At most 1 digits are allowed`), ".", ts, defaultTimeFormat, true, 1, "sql", "ten"},
{"seq overflow", nil, "", []string{"9_nine.sql"}, []string{"9_nine.sql"}, errors.New(`next sequence number 10 too large, at most 1 digits are allowed`), ".", ts, defaultTimeFormat, true, 1, "sql", "ten"},
{"time empty format", nil, "", nil, nil, errInvalidTimeFormat, ".", ts, "", false, 0, "sql", "name"},
{"time unix", nil, "", nil, []string{tsUnixStr + "_name.up.sql", tsUnixStr + "_name.down.sql"}, nil, ".", ts, "unix", false, 0, "sql", "name"},
{"time unixNano", nil, "", nil, []string{tsUnixNanoStr + "_name.up.sql", tsUnixNanoStr + "_name.down.sql"}, nil, ".", ts, "unixNano", false, 0, "sql", "name"},
+2 -2
View File
@@ -275,7 +275,7 @@ Database drivers: `+strings.Join(database.List(), ", ")+"\n", createUsage, gotoU
if needsConfirm {
log.Println("Are you sure you want to apply all down migrations? [y/N]")
var response string
fmt.Scanln(&response)
_, _ = fmt.Scanln(&response)
response = strings.ToLower(strings.TrimSpace(response))
if response == "y" {
@@ -306,7 +306,7 @@ Database drivers: `+strings.Join(database.List(), ", ")+"\n", createUsage, gotoU
if !*forceDrop {
log.Println("Are you sure you want to drop the entire database schema? [y/N]")
var response string
fmt.Scanln(&response)
_, _ = fmt.Scanln(&response)
response = strings.ToLower(strings.TrimSpace(response))
if response == "y" {
+26 -28
View File
@@ -11,8 +11,6 @@ import (
"sync"
"time"
"github.com/hashicorp/go-multierror"
"github.com/golang-migrate/migrate/v4/database"
iurl "github.com/golang-migrate/migrate/v4/internal/url"
"github.com/golang-migrate/migrate/v4/source"
@@ -56,10 +54,10 @@ func (e ErrDirty) Error() string {
}
type Migrate struct {
sourceName string
sourceDrv source.Driver
databaseName string
databaseDrv database.Driver
sourceName string
sourceDrv source.Driver
databaseDriverName string
databaseDrv database.Driver
// Log accepts a Logger interface
Log Logger
@@ -89,25 +87,25 @@ func New(sourceURL, databaseURL string) (*Migrate, error) {
sourceName, err := iurl.SchemeFromURL(sourceURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to parse scheme from source URL: %w", err)
}
m.sourceName = sourceName
databaseName, err := iurl.SchemeFromURL(databaseURL)
databaseDriverName, err := iurl.SchemeFromURL(databaseURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to parse scheme from database URL: %w", err)
}
m.databaseName = databaseName
m.databaseDriverName = databaseDriverName
sourceDrv, err := source.Open(sourceURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to open source, %q: %w", sourceURL, err)
}
m.sourceDrv = sourceDrv
databaseDrv, err := database.Open(databaseURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to open database: %w", err)
}
m.databaseDrv = databaseDrv
@@ -116,9 +114,9 @@ func New(sourceURL, databaseURL string) (*Migrate, error) {
// NewWithDatabaseInstance returns a new Migrate instance from a source URL
// and an existing database instance. The source URL scheme is defined by each driver.
// Use any string that can serve as an identifier during logging as databaseName.
// Use any string that can serve as an identifier during logging as databaseDriverName.
// You are responsible for closing the underlying database client if necessary.
func NewWithDatabaseInstance(sourceURL string, databaseName string, databaseInstance database.Driver) (*Migrate, error) {
func NewWithDatabaseInstance(sourceURL string, databaseDriverName string, databaseInstance database.Driver) (*Migrate, error) {
m := newCommon()
sourceName, err := iurl.SchemeFromURL(sourceURL)
@@ -127,11 +125,11 @@ func NewWithDatabaseInstance(sourceURL string, databaseName string, databaseInst
}
m.sourceName = sourceName
m.databaseName = databaseName
m.databaseDriverName = databaseDriverName
sourceDrv, err := source.Open(sourceURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to open source, %q: %w", sourceURL, err)
}
m.sourceDrv = sourceDrv
@@ -147,17 +145,17 @@ func NewWithDatabaseInstance(sourceURL string, databaseName string, databaseInst
func NewWithSourceInstance(sourceName string, sourceInstance source.Driver, databaseURL string) (*Migrate, error) {
m := newCommon()
databaseName, err := iurl.SchemeFromURL(databaseURL)
databaseDriverName, err := iurl.SchemeFromURL(databaseURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to parse scheme from database URL: %w", err)
}
m.databaseName = databaseName
m.databaseDriverName = databaseDriverName
m.sourceName = sourceName
databaseDrv, err := database.Open(databaseURL)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed to open database: %w", err)
}
m.databaseDrv = databaseDrv
@@ -168,13 +166,13 @@ func NewWithSourceInstance(sourceName string, sourceInstance source.Driver, data
// NewWithInstance returns a new Migrate instance from an existing source and
// database instance. Use any string that can serve as an identifier during logging
// as sourceName and databaseName. You are responsible for closing down
// as sourceName and databaseDriverName. You are responsible for closing down
// the underlying source and database client if necessary.
func NewWithInstance(sourceName string, sourceInstance source.Driver, databaseName string, databaseInstance database.Driver) (*Migrate, error) {
func NewWithInstance(sourceName string, sourceInstance source.Driver, databaseDriverName string, databaseInstance database.Driver) (*Migrate, error) {
m := newCommon()
m.sourceName = sourceName
m.databaseName = databaseName
m.databaseDriverName = databaseDriverName
m.sourceDrv = sourceInstance
m.databaseDrv = databaseInstance
@@ -526,7 +524,7 @@ func (m *Migrate) read(from int, to int, ret chan<- interface{}) {
}
}
// readUp reads up migrations from `from` limitted by `limit`.
// readUp reads up migrations from `from` limited by `limit`.
// limit can be -1, implying no limit and reading until there are no more migrations.
// Each migration is then written to the ret channel.
// If an error occurs during reading, that error is written to the ret channel, too.
@@ -626,7 +624,7 @@ func (m *Migrate) readUp(from int, limit int, ret chan<- interface{}) {
}
}
// readDown reads down migrations from `from` limitted by `limit`.
// readDown reads down migrations from `from` limited by `limit`.
// limit can be -1, implying no limit and reading until there are no more migrations.
// Each migration is then written to the ret channel.
// If an error occurs during reading, that error is written to the ret channel, too.
@@ -781,7 +779,7 @@ func (m *Migrate) versionExists(version uint) (result error) {
if err == nil {
defer func() {
if errClose := up.Close(); errClose != nil {
result = multierror.Append(result, errClose)
result = errors.Join(result, errClose)
}
}()
}
@@ -796,7 +794,7 @@ func (m *Migrate) versionExists(version uint) (result error) {
if err == nil {
defer func() {
if errClose := down.Close(); errClose != nil {
result = multierror.Append(result, errClose)
result = errors.Join(result, errClose)
}
}()
}
@@ -954,7 +952,7 @@ func (m *Migrate) unlock() error {
// if a prevErr is not nil.
func (m *Migrate) unlockErr(prevErr error) error {
if err := m.unlock(); err != nil {
return multierror.Append(prevErr, err)
prevErr = errors.Join(prevErr, err)
}
return prevErr
}
+8 -13
View File
@@ -54,8 +54,8 @@ func TestNew(t *testing.T) {
t.Error("expected sourceDrv not to be nil")
}
if m.databaseName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseName)
if m.databaseDriverName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseDriverName)
}
if m.databaseDrv == nil {
t.Error("expected databaseDrv not to be nil")
@@ -94,8 +94,8 @@ func TestNewWithDatabaseInstance(t *testing.T) {
t.Error("expected sourceDrv not to be nil")
}
if m.databaseName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseName)
if m.databaseDriverName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseDriverName)
}
if m.databaseDrv == nil {
t.Error("expected databaseDrv not to be nil")
@@ -153,8 +153,8 @@ func TestNewWithSourceInstance(t *testing.T) {
t.Error("expected sourceDrv not to be nil")
}
if m.databaseName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseName)
if m.databaseDriverName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseDriverName)
}
if m.databaseDrv == nil {
t.Error("expected databaseDrv not to be nil")
@@ -209,8 +209,8 @@ func TestNewWithInstance(t *testing.T) {
t.Error("expected sourceDrv not to be nil")
}
if m.databaseName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseName)
if m.databaseDriverName != dbDrvNameStub {
t.Errorf("expected stub, got %v", m.databaseDriverName)
}
if m.databaseDrv == nil {
t.Error("expected databaseDrv not to be nil")
@@ -1339,11 +1339,6 @@ func newMigSeq(migr ...*Migration) migrationSequence {
return migr
}
func (m *migrationSequence) add(migr ...*Migration) migrationSequence { // nolint:unused
*m = append(*m, migr...)
return *m
}
func (m *migrationSequence) bodySequence() []string {
r := make([]string, 0)
for _, v := range *m {
+17 -12
View File
@@ -2,6 +2,7 @@ package migrate
import (
"bufio"
"errors"
"fmt"
"io"
"time"
@@ -118,7 +119,7 @@ func (m *Migration) LogString() string {
// Buffer buffers Body up to BufferSize.
// Calling this function blocks. Call with goroutine.
func (m *Migration) Buffer() error {
func (m *Migration) Buffer() (berr error) {
if m.Body == nil {
return nil
}
@@ -127,6 +128,21 @@ func (m *Migration) Buffer() error {
b := bufio.NewReaderSize(m.Body, int(m.BufferSize))
// defer closing buffer writer and body.
defer func() {
// close bufferWriter so Buffer knows that there is no
// more data coming.
if err := m.bufferWriter.Close(); err != nil {
berr = errors.Join(berr, err)
}
// it's safe to close the Body too.
if err := m.Body.Close(); err != nil {
berr = errors.Join(berr, err)
}
}()
// start reading from body, peek won't move the read pointer though
// poor man's solution?
if _, err := b.Peek(int(m.BufferSize)); err != nil && err != io.EOF {
@@ -145,16 +161,5 @@ func (m *Migration) Buffer() error {
m.FinishedReading = time.Now()
m.BytesRead = n
// close bufferWriter so Buffer knows that there is no
// more data coming
if err := m.bufferWriter.Close(); err != nil {
return err
}
// it's safe to close the Body too
if err := m.Body.Close(); err != nil {
return err
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More