refactor(datastore): back the datastore migration driver with native hanzo-ds/go, drop clickhouse-go

The datastore:// migration driver is now a self-contained golang-migrate
database.Driver that opens through github.com/hanzo-ds/go (the native
ClickHouse-wire driver registered as the database/sql name "datastore").
The ClickHouse-specific migration logic (multi-statement exec,
schema_migrations DDL, cluster/engine config, Version/SetVersion/Drop/Lock)
is ported directly into database/datastore, so it no longer embeds or
imports the upstream database/clickhouse driver or
github.com/ClickHouse/clickhouse-go.

- Delete database/clickhouse (driver + test + examples) and
  internal/cli/build_clickhouse.go.
- internal/cli/build_datastore.go drops the clickhouse-go blank import.
- go.mod: drop github.com/ClickHouse/clickhouse-go, add
  github.com/hanzo-ds/go v1.0.1; tidy. Zero clickhouse-go in go.mod,
  go.sum, and the module/dependency graph.
- Port the driver test + example migrations under database/datastore
  (integration test requires a live datastore/clickhouse-wire backend).
- Update README, release workflow build tags, bug report template.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.circleci/config.yml

@@ -0,0 +1,56 @@
+# Golang CircleCI 2.0 configuration file
+#
+# Check https://circleci.com/docs/2.0/language-go/ for more details
+version: 2.1
+
+jobs:
+  "golang-1_15": &template
+    machine:
+      # https://circleci.com/docs/2.0/configuration-reference/#available-machine-images
+      image: ubuntu-2004:202010-01
+      # docker_layer_caching: true
+
+    # https://circleci.com/docs/2.0/configuration-reference/#resource_class
+    resource_class: medium
+
+    # Leave working directory unspecified and use defaults:
+    # https://circleci.com/blog/go-v1.11-modules-and-circleci/
+    # working_directory: /go/src/github.com/golang-migrate/migrate
+
+    environment:
+      GO111MODULE: "on"
+      GO_VERSION: "1.15.x"
+
+    steps:
+      # - setup_remote_docker:
+      #    version: 19.03.13
+          # docker_layer_caching: true
+      - run: curl -sL -o ~/bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
+      - run: curl -sfL -o ~/bin/golangci-lint.sh https://install.goreleaser.com/github.com/golangci/golangci-lint.sh
+      - run: chmod +x ~/bin/gimme ~/bin/golangci-lint.sh
+      - run: eval "$(gimme $GO_VERSION)"
+      - run: golangci-lint.sh -b ~/bin v1.37.0
+      - checkout
+      - restore_cache:
+          keys:
+            - go-mod-v1-{{ arch }}-{{ checksum "go.sum" }}
+      - run: golangci-lint run
+      - run: make test COVERAGE_DIR=/tmp/coverage
+      - save_cache:
+          key: go-mod-v1-{{ arch }}-{{ checksum "go.sum" }}
+          paths:
+            - "/go/pkg/mod"
+      - run: go get github.com/mattn/goveralls
+      - run: goveralls -service=circle-ci -coverprofile /tmp/coverage/combined.txt
+
+  "golang-1_16":
+    <<: *template
+    environment:
+      GO_VERSION: "1.16.x"
+
+workflows:
+  version: 2
+  build:
+    jobs:
+      - "golang-1_15"
+      - "golang-1_16"

.dockerignore

@@ -2,7 +2,6 @@
 FAQ.md
 README.md
 LICENSE
-Makefile
 .gitignore
 .travis.yml
 CONTRIBUTING.md

.github/ISSUE_TEMPLATE/bug_report.md

@@ -25,7 +25,7 @@ e.g. s3, github, go-bindata, gcs, file
 Obtained by running: `migrate -help`
 
 **Loaded Database Drivers**
-e.g. spanner, stub, clickhouse, cockroachdb, crdb-postgres, postgres, postgresql, redshift, cassandra, cockroach, mysql
+e.g. spanner, stub, datastore, cockroachdb, crdb-postgres, postgres, postgresql, pgx, redshift, cassandra, cockroach, mysql
 Obtained by running: `migrate -help`
 
 **Go Version**

.github/hero.svg

@@ -0,0 +1,9 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="640" viewBox="0 0 1280 640" role="img" aria-label="migrate">
+<rect width="1280" height="640" fill="#0A0A0A"/>
+<svg x="96" y="215" width="210" height="210" viewBox="0 0 67 67"><path d="M22.21 67V44.6369H0V67H22.21Z" fill="#fff"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z" fill="#fff"/><path d="M22.21 0H0V22.3184H22.21V0Z" fill="#fff"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z" fill="#fff"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z" fill="#fff"/></svg>
+<text x="378" y="276" font-family="Inter,system-ui,-apple-system,sans-serif" font-size="78" font-weight="800" letter-spacing="-2" fill="#ffffff">migrate</text>
+<text x="378" y="322" font-family="Inter,system-ui,sans-serif" font-size="30" fill="#ffffff" opacity=".66">Database migrations. CLI and Golang library.</text>
+<rect x="378" y="338" width="806" height="3" rx="1.5" fill="#ffffff" opacity=".9"/>
+<text x="378" y="390" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">github.com/hanzoai</text>
+<text x="1184" y="390" text-anchor="end" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">hanzo.ai</text>
+</svg>

.github/workflows/ci.yaml

@@ -0,0 +1,111 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  lint:
+    name: lint
+    runs-on: hanzo-build-linux-amd64
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-go@v5
+        with:
+          go-version: "1.26.4.x"
+      - name: golangci-lint
+        uses: golangci/golangci-lint-action@v9
+        with:
+          version: v2.6.2
+
+  test:
+    runs-on: hanzo-build-linux-amd64
+    strategy:
+      matrix:
+        go: ["1.24.x", "1.25.x"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-go@v5
+        with:
+          go-version: ${{ matrix.go }}
+
+      - name: Run test
+        run: make test COVERAGE_DIR=/tmp/coverage
+
+      - name: Send goveralls coverage
+        uses: shogo82148/actions-goveralls@v1
+        with:
+          path-to-profile: /tmp/coverage/combined.txt
+          flag-name: Go-${{ matrix.go }}
+          parallel: true
+
+  check-coverage:
+    name: Check coverage
+    needs: [test]
+    runs-on: hanzo-build-linux-amd64
+    steps:
+      - uses: shogo82148/actions-goveralls@v1
+        with:
+          parallel-finished: true
+
+  goreleaser:
+    name: Release a new version
+    needs: [lint, test]
+    runs-on: hanzo-build-linux-amd64
+    environment: GoReleaser
+    # This job only runs when
+    # 1. When the previous `lint` and `test` jobs has completed successfully
+    # 2. When the repository is not a fork, i.e. it will only run on the official golang-migrate/migrate
+    # 3. When the workflow is triggered by a tag with `v` prefix
+    if: ${{ success() && github.repository == 'golang-migrate/migrate' && startsWith(github.ref, 'refs/tags/v') }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: 2.7
+      - uses: actions/setup-go@v5
+        with:
+          go-version: "1.26.4.x"
+
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          username: golangmigrate
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - run: echo "SOURCE=$(make echo-source)" >> $GITHUB_ENV
+      - run: echo "DATABASE=$(make echo-database)" >> $GITHUB_ENV
+
+      - uses: goreleaser/goreleaser-action@v5
+        with:
+          version: latest
+          args: release --clean
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - run: gem install package_cloud
+      - run: package_cloud push golang-migrate/migrate/ubuntu/bionic dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+      - run: package_cloud push golang-migrate/migrate/ubuntu/focal dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+      - run: package_cloud push golang-migrate/migrate/ubuntu/jammy dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+      - run: package_cloud push golang-migrate/migrate/ubuntu/noble dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+      - run: package_cloud push golang-migrate/migrate/debian/buster dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+      - run: package_cloud push golang-migrate/migrate/debian/bullseye dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}
+      - run: package_cloud push golang-migrate/migrate/debian/bookworm dist/migrate.linux-amd64.deb
+        env:
+          PACKAGECLOUD_TOKEN: ${{ secrets.PACKAGECLOUD_TOKEN }}

.github/workflows/release.yml

@@ -0,0 +1,52 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - 'v*'
+
+permissions:
+  contents: write
+  packages: write
+
+jobs:
+  release:
+    runs-on: hanzo-build-linux-amd64
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-go@v5
+        with:
+          go-version: '1.26.4'
+
+      - name: Build CLI binaries
+        run: |
+          VERSION=${GITHUB_REF_NAME#v}
+          mkdir -p cli/build
+
+          for pair in "linux/amd64" "linux/arm64" "darwin/amd64" "darwin/arm64"; do
+            OS=${pair%/*}
+            ARCH=${pair#*/}
+            OUTPUT="cli/build/migrate.${OS}-${ARCH}"
+            echo "Building $OUTPUT..."
+            CGO_ENABLED=0 GOOS=$OS GOARCH=$ARCH \
+              go build -a -o "$OUTPUT" \
+              -ldflags="-s -w -X main.Version=$VERSION" \
+              -tags 'datastore file' \
+              ./cmd/migrate
+            ln -f "$OUTPUT" cli/build/migrate
+            tar czf "${OUTPUT}.tar.gz" -C cli/build migrate
+          done
+
+          cd cli/build && sha256sum *.tar.gz > sha256sum.txt
+
+      - name: Create GitHub Release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh release create ${{ github.ref_name }} \
+            cli/build/*.tar.gz \
+            cli/build/sha256sum.txt \
+            --title "${{ github.ref_name }}" \
+            --generate-notes \
+            --latest

.gitignore

@@ -4,4 +4,8 @@ cli/cli
 cli/migrate
 .coverage
 .godoc.pid
-vendor/
+vendor/
+.vscode/
+.idea
+dist/
+migrate

.golangci.yml

@@ -0,0 +1,37 @@
+version: "2"
+linters:
+  enable:
+    - goconst
+    - misspell
+    - nakedret
+    - prealloc
+    - revive
+    - unconvert
+    - unparam
+  settings:
+    misspell:
+      locale: US
+    revive:
+      rules:
+        - name: redundant-build-tag
+  exclusions:
+    generated: lax
+    rules:
+      - path: (.+)\.go$
+        text: G104
+    paths:
+      - third_party$
+      - builtin$
+      - examples$
+issues:
+  max-issues-per-linter: 0
+  max-same-issues: 0
+formatters:
+  enable:
+    - gofmt
+  exclusions:
+    generated: lax
+    paths:
+      - third_party$
+      - builtin$
+      - examples$

.goreleaser.yml

@@ -0,0 +1,102 @@
+project_name: migrate
+before:
+  hooks:
+    - go mod tidy
+builds:
+  - env:
+      - CGO_ENABLED=0
+    goos:
+      - linux
+      - windows
+      - darwin
+    goarch:
+      - amd64
+      - arm
+      - arm64
+      - 386
+    goarm:
+      - 7
+    main: ./cmd/migrate
+    ldflags:
+      - '-w -s -X main.Version={{ .Version }} -extldflags "static"'
+    flags:
+      - "-tags={{ .Env.DATABASE }} {{ .Env.SOURCE }}"
+      - "-trimpath"
+nfpms:
+  - homepage: "https://github.com/golang-migrate/migrate"
+    maintainer: "dhui@users.noreply.github.com"
+    license: MIT
+    description: "Database migrations"
+    formats:
+      - deb
+    file_name_template: "{{ .ProjectName }}.{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"
+dockers:
+  - goos: linux
+    goarch: amd64
+    dockerfile: Dockerfile.github-actions
+    use: buildx
+    ids:
+      - migrate
+    image_templates:
+      - 'migrate/migrate:{{ .Tag }}-amd64'
+    build_flag_templates:
+      - '--label=org.opencontainers.image.created={{ .Date }}'
+      - '--label=org.opencontainers.image.title={{ .ProjectName }}'
+      - '--label=org.opencontainers.image.revision={{ .FullCommit }}'
+      - '--label=org.opencontainers.image.version={{ .Version }}'
+      - "--label=org.opencontainers.image.source={{ .GitURL }}"
+      - "--platform=linux/amd64"
+  - goos: linux
+    goarch: arm64
+    dockerfile: Dockerfile.github-actions
+    use: buildx
+    ids:
+      - migrate
+    image_templates:
+      - 'migrate/migrate:{{ .Tag }}-arm64'
+    build_flag_templates:
+      - '--label=org.opencontainers.image.created={{ .Date }}'
+      - '--label=org.opencontainers.image.title={{ .ProjectName }}'
+      - '--label=org.opencontainers.image.revision={{ .FullCommit }}'
+      - '--label=org.opencontainers.image.version={{ .Version }}'
+      - "--label=org.opencontainers.image.source={{ .GitURL }}"
+      - "--platform=linux/arm64"
+
+docker_manifests:
+- name_template: 'migrate/migrate:{{ .Tag }}'
+  image_templates:
+  - 'migrate/migrate:{{ .Tag }}-amd64'
+  - 'migrate/migrate:{{ .Tag }}-arm64'
+- name_template: 'migrate/migrate:{{ .Major }}'
+  image_templates:
+  - 'migrate/migrate:{{ .Tag }}-amd64'
+  - 'migrate/migrate:{{ .Tag }}-arm64'
+- name_template: 'migrate/migrate:latest'
+  image_templates:
+  - 'migrate/migrate:{{ .Tag }}-amd64'
+  - 'migrate/migrate:{{ .Tag }}-arm64'
+archives:
+  - name_template: "{{ .ProjectName }}.{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"
+    format_overrides:
+      - goos: windows
+        format: zip
+checksum:
+  name_template: 'sha256sum.txt'
+release:
+  draft: true
+  prerelease: auto
+source:
+  enabled: true
+  format: zip
+changelog:
+  skip: false
+  sort: asc
+  filters:
+    exclude:
+      - '^docs:'
+      - '^test:'
+      - Merge pull request
+      - Merge branch
+      - go mod tidy
+snapshot:
+  name_template: "{{ .Tag }}-next"

.travis.yml

@@ -6,8 +6,8 @@ matrix:
     - go: master
   include:
     # Supported versions of Go: https://golang.org/dl/
-    - go: "1.10.x"
-    - go: "1.11.x"
+    - go: "1.14.x"
+    - go: "1.15.x"
     - go: master
 
 go_import_path: github.com/golang-migrate/migrate
@@ -26,18 +26,22 @@ cache:
   directories:
     - $GOPATH/pkg
 
+
 before_install:
+  # Update docker to latest version: https://docs.travis-ci.com/user/docker/#installing-a-newer-docker-version
+  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
+  - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
+  - sudo apt-get update
+  - sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
+  # Install golangci-lint
+  - curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.30.0
   - echo "TRAVIS_GO_VERSION=${TRAVIS_GO_VERSION}"
-  # Download the binary to bin folder in $GOPATH
-  - if [[ "${TRAVIS_GO_VERSION}" == 1.10* ]]; then curl -L -s https://github.com/golang/dep/releases/download/v0.5.0/dep-linux-amd64 -o $GOPATH/bin/dep; fi
-  # Make the binary executable
-  - if [[ "${TRAVIS_GO_VERSION}" == 1.10* ]]; then chmod +x $GOPATH/bin/dep; fi
 
 install:
-  - if [[ "${TRAVIS_GO_VERSION}" == 1.10* ]]; then dep ensure -vendor-only; fi
   - go get github.com/mattn/goveralls
 
 script:
+  - golangci-lint run
   - make test COVERAGE_DIR=/tmp/coverage
 
 after_success:
@@ -53,13 +57,16 @@ deploy:
       secure: hWH1HLPpzpfA8pXQ93T1qKQVFSpQp0as/JLQ7D91jHuJ8p+RxVeqblDrR6HQY/95R/nyiE9GJmvUolSuw5h449LSrGxPtVWhdh6EnkxlQHlen5XeMhVjRjFV0sE9qGe8v7uAkiTfRO61ktTWHrEAvw5qpyqnNISodmZS78XIasPODQbNlzwINhWhDTHIjXGb4FpizYaL3OGCanrxfR9fQyCaqKGGBjRq3Mfq8U6Yd4mApmsE+uJxgaZV8K5zBqpkSzQRWhcVGNL5DuLsU3gfSJOo7kZeA2G71SHffH577dBoqtCZ4VFv169CoUZehLWCb+7XKJZmHXVujCURATSySLGUOPc6EoLFAn3YtsCA04mS4bZVo5FZPWVwfhjmkhtDR4f6wscKp7r1HsFHSOgm59QfETQdrn4MnZ44H2Jd39axqndn5DvK9EcZVjPHynOPnueXP2u6mTuUgh2VyyWBCDO3CNo0fGlo7VJI69IkIWNSD87K9cHZWYMClyKZkUzS+PmRAhHRYbVd+9ZjKOmnU36kUHNDG/ft1D4ogsY+rhVtXB4lgWDM5adri+EIScYdYnB1/pQexLBigcJY9uE7nQTR0U6QgVNYvun7uRNs40E0c4voSfmPdFO0FlOD2y1oQhnaXfWLbu9nMcTcs4RFGrcC7NzkUN4/WjG8s285V6w=
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true
     file:
       - cli/build/migrate.linux-amd64.tar.gz
+      - cli/build/migrate.linux-armv7.tar.gz
+      - cli/build/migrate.linux-arm64.tar.gz
       - cli/build/migrate.darwin-amd64.tar.gz
       - cli/build/migrate.windows-amd64.exe.tar.gz
+      - cli/build/migrate.windows-386.exe.tar.gz
       - cli/build/sha256sum.txt
       - dependency_tree.txt
   - provider: packagecloud
@@ -71,7 +78,7 @@ deploy:
     package_glob: '*.deb'
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true
   - provider: packagecloud
@@ -83,7 +90,7 @@ deploy:
     package_glob: '*.deb'
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true
   - provider: packagecloud
@@ -91,11 +98,11 @@ deploy:
     username: golang-migrate
     token:
       secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o=
-    dist: ubuntu/cosmic
+    dist: ubuntu/focal
     package_glob: '*.deb'
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true
   - provider: packagecloud
@@ -107,7 +114,7 @@ deploy:
     package_glob: '*.deb'
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true
   - provider: packagecloud
@@ -119,13 +126,13 @@ deploy:
     package_glob: '*.deb'
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true
   - provider: script
     script: ./docker-deploy.sh
     skip_cleanup: true
     on:
-      go: "1.11.x"
+      go: "1.15.x"
       repo: golang-migrate/migrate
       tags: true

AGENTS.md

@@ -0,0 +1 @@
+LLM.md

CLAUDE.md

@@ -0,0 +1 @@
+LLM.md

CONTRIBUTING.md

@@ -5,6 +5,8 @@
   1. Use a version of Go that supports [modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) (e.g. Go 1.11+)
   1. Fork this repo and `git clone` somewhere to `$GOPATH/src/github.com/golang-migrate/migrate`
       * Ensure that [Go modules are enabled](https://golang.org/cmd/go/#hdr-Preliminary_module_support) (e.g. your repo path or the `GO111MODULE` environment variable are set correctly)
+  1. Install [golangci-lint](https://github.com/golangci/golangci-lint#install)
+  1. Run the linter: `golangci-lint run`
   1. Confirm tests are working: `make test-short`
   1. Write awesome code ...
   1. `make test` to run all tests against all database versions

Dockerfile

@@ -1,23 +1,26 @@
-FROM golang:1.11-alpine3.8 AS downloader
+FROM golang:1.26.4-alpine3.21 AS builder
 ARG VERSION
 
-RUN apk add --no-cache git gcc musl-dev
+RUN apk add --no-cache git gcc musl-dev make
 
 WORKDIR /go/src/github.com/golang-migrate/migrate
 
+ENV GO111MODULE=on
+
+COPY go.mod go.sum ./
+
+RUN go mod download
+
 COPY . ./
 
-ENV GO111MODULE=on
-ENV DATABASES="postgres mysql redshift cassandra spanner cockroachdb clickhouse"
-ENV SOURCES="file go_bindata github aws_s3 google_cloud_storage"
+RUN make build-docker
 
-RUN go build -a -o build/migrate.linux-386 -ldflags="-X main.Version=${VERSION}" -tags "$DATABASES $SOURCES" ./cmd/migrate
-
-FROM alpine:3.8
+FROM alpine:3.21
 
 RUN apk add --no-cache ca-certificates
 
-COPY --from=downloader /go/src/github.com/golang-migrate/migrate/build/migrate.linux-386 /migrate
+COPY --from=builder /go/src/github.com/golang-migrate/migrate/build/migrate.linux-386 /usr/local/bin/migrate
+RUN ln -s /usr/local/bin/migrate /migrate
 
-ENTRYPOINT ["/migrate"]
+ENTRYPOINT ["migrate"]
 CMD ["--help"]

Dockerfile.circleci

@@ -0,0 +1,17 @@
+ARG DOCKER_IMAGE
+FROM $DOCKER_IMAGE
+
+RUN apk add --no-cache git gcc musl-dev make
+
+WORKDIR /go/src/github.com/golang-migrate/migrate
+
+ENV GO111MODULE=on
+ENV COVERAGE_DIR=/tmp/coverage
+
+COPY go.mod go.sum ./
+
+RUN go mod download
+
+COPY . ./
+
+CMD ["make", "test"]

Dockerfile.github-actions

@@ -0,0 +1,11 @@
+FROM alpine:3.19
+
+RUN apk add --no-cache ca-certificates
+
+COPY migrate /usr/local/bin/migrate
+
+RUN ln -s /usr/local/bin/migrate /usr/bin/migrate
+RUN ln -s /usr/local/bin/migrate /migrate
+
+ENTRYPOINT ["migrate"]
+CMD ["--help"]

FAQ.md

@@ -16,7 +16,7 @@
   NilMigration defines a migration without a body. NilVersion is defined as const -1.
 
 #### What is the difference between uint(version) and int(targetVersion)?
-  version refers to an existing migration version coming from a source and therefor can never be negative.
+  version refers to an existing migration version coming from a source and therefore can never be negative.
   targetVersion can either be a version OR represent a NilVersion, which equals -1.
 
 #### What's the difference between Next/Previous and Up/Down?
@@ -50,10 +50,10 @@
   and whenever we want, not just once at the beginning of all tests.
 
 #### Can I maintain my driver in my own repository?
-  Yes, technically thats possible. We want to encourage you to contribute your driver to this respository though.
+  Yes, technically thats possible. We want to encourage you to contribute your driver to this repository though.
   The driver's functionality is dictated by migrate's interfaces. That means there should really
   just be one driver for a database/ source. We want to prevent a future where several drivers doing the exact same thing,
-  just implemented a bit differently, co-exist somewhere on Github. If users have to do research first to find the
+  just implemented a bit differently, co-exist somewhere on GitHub. If users have to do research first to find the
   "best" available driver for a database in order to get started, we would have failed as an open source community.
 
 #### Can I mix multiple sources during a batch of migrations?
@@ -65,6 +65,15 @@
   and then "force" the expected version.
 
 #### What happens if two programs try and update the database at the same time?
-Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations at the same time
+  Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations on
   the same database at the same time. For example, the MySQL driver uses the `GET_LOCK` function, while the Postgres driver uses
   the `pg_advisory_lock` function.
+
+#### Do I need to create a table for tracking migration version used?
+No, it is done automatically.
+
+#### Can I use migrate with a non-Go project?
+Yes, you can use the migrate CLI in a non-Go project, but there are probably other libraries/frameworks available that offer better test and deploy integrations in that language/framework.
+
+#### I have got an error `Dirty database version 1. Fix and force version`. What should I do?
+Keep calm and refer to [the getting started docs](GETTING_STARTED.md#forcing-your-database-version).

GETTING_STARTED.md

@@ -0,0 +1,53 @@
+# Getting started
+Before you start, you should understand the concept of forward/up and reverse/down database migrations.
+
+Configure a database for your application. Make sure that your database driver is supported [here](README.md#databases).
+
+## Create migrations
+Create some migrations using migrate CLI. Here is an example:
+```
+migrate create -ext sql -dir db/migrations -seq create_users_table
+```
+Once you create your files, you should fill them.
+
+**IMPORTANT:** In a project developed by more than one person there is a chance of migrations inconsistency - e.g. two developers can create conflicting migrations, and the developer that created their migration later gets it merged to the repository first.
+Developers and Teams should keep an eye on such cases (especially during code review).
+[Here](https://github.com/golang-migrate/migrate/issues/179#issuecomment-475821264) is the issue summary if you would like to read more.
+
+Consider making your migrations idempotent - we can run the same sql code twice in a row with the same result. This makes our migrations more robust. On the other hand, it causes slightly less control over database schema - e.g. let's say you forgot to drop the table in down migration. You run down migration - the table is still there. When you run up migration again - `CREATE TABLE` would return an error, helping you find an issue in down migration, while `CREATE TABLE IF NOT EXISTS` would not. Use those conditions wisely.
+
+In case you would like to run several commands/queries in one migration, you should wrap them in a transaction (if your database supports it).
+This way if one of commands fails, our database will remain unchanged.
+
+## Run migrations
+Run your migrations through the CLI or your app and check if they applied expected changes.
+Just to give you an idea:
+```
+migrate -database YOUR_DATABASE_URL -path PATH_TO_YOUR_MIGRATIONS up
+```
+
+Just add the code to your app and you're ready to go!
+
+Before committing your migrations you should run your migrations up, down, and then up again to see if migrations are working properly both ways.
+(e.g. if you created a table in a migration but reverse migration did not delete it, you will encounter an error when running the forward migration again)
+It's also worth checking your migrations in a separate, containerized environment. You can find some tools at the [end of this document](#further-reading).
+
+**IMPORTANT:** If you would like to run multiple instances of your app on different machines be sure to use a database that supports locking when running migrations. Otherwise you may encounter issues.
+
+## Forcing your database version
+In case you run a migration that contained an error, migrate will not let you run other migrations on the same database. You will see an error like `Dirty database version 1. Fix and force version`, even when you fix the erred migration. This means your database was marked as 'dirty'.
+You need to investigate the migration error - was your migration applied partially, or was it not applied at all? Once you know, you should force your database to a version reflecting it's real state. You can do so with `force` command:
+```
+migrate -path PATH_TO_YOUR_MIGRATIONS -database YOUR_DATABASE_URL force VERSION
+```
+Once you force the version and your migration was fixed, your database is 'clean' again and you can proceed with your migrations.
+
+For details and example of usage see [this comment](https://github.com/golang-migrate/migrate/issues/282#issuecomment-530743258).
+
+## Further reading:
+- [PostgreSQL tutorial](database/postgres/TUTORIAL.md)
+- [Best practices](MIGRATIONS.md)
+- [FAQ](FAQ.md)
+- Tools for testing your migrations in a container:
+	- https://github.com/dhui/dktest
+	- https://github.com/ory/dockertest

Gopkg.lock

@@ -1,828 +0,0 @@
-# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
-
-
-[[projects]]
-  digest = "1:4a799555f4a58b849ca3a3bed67c10ca484a2d3cedf4199773211d04b0109793"
-  name = "cloud.google.com/go"
-  packages = [
-    "civil",
-    "compute/metadata",
-    "iam",
-    "internal",
-    "internal/atomiccache",
-    "internal/fields",
-    "internal/optional",
-    "internal/protostruct",
-    "internal/trace",
-    "internal/version",
-    "longrunning",
-    "longrunning/autogen",
-    "spanner",
-    "spanner/admin/database/apiv1",
-    "storage",
-  ]
-  pruneopts = "UT"
-  revision = "c728a003b238b26cef9ab6753a5dc424b331c3ad"
-  version = "v0.27.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:6da51e5ec493ad2b44cb04129e2d0a068c8fb9bd6cb5739d199573558696bb94"
-  name = "github.com/Azure/go-ansiterm"
-  packages = [
-    ".",
-    "winterm",
-  ]
-  pruneopts = "UT"
-  revision = "d6e3b3328b783f23731bc4d058875b0371ff8109"
-
-[[projects]]
-  digest = "1:f9ae348e1f793dcf9ed930ed47136a67343dbd6809c5c91391322267f4476892"
-  name = "github.com/Microsoft/go-winio"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "97e4973ce50b2ff5f09635a57e2b88a037aae829"
-  version = "v0.4.11"
-
-[[projects]]
-  branch = "master"
-  digest = "1:3721a10686511b80c052323423f0de17a8c06d417dbdd3b392b1578432a33aae"
-  name = "github.com/Nvveen/Gotty"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "cd527374f1e5bff4938207604a14f2e38a9cf512"
-
-[[projects]]
-  digest = "1:1d58a675bc7df5691f81b4a28a39d2d218f020b23644a6c7771f7ebaab272ac8"
-  name = "github.com/aws/aws-sdk-go"
-  packages = [
-    "aws",
-    "aws/awserr",
-    "aws/awsutil",
-    "aws/client",
-    "aws/client/metadata",
-    "aws/corehandlers",
-    "aws/credentials",
-    "aws/credentials/ec2rolecreds",
-    "aws/credentials/endpointcreds",
-    "aws/credentials/stscreds",
-    "aws/csm",
-    "aws/defaults",
-    "aws/ec2metadata",
-    "aws/endpoints",
-    "aws/request",
-    "aws/session",
-    "aws/signer/v4",
-    "internal/s3err",
-    "internal/sdkio",
-    "internal/sdkrand",
-    "internal/sdkuri",
-    "internal/shareddefaults",
-    "private/protocol",
-    "private/protocol/eventstream",
-    "private/protocol/eventstream/eventstreamapi",
-    "private/protocol/query",
-    "private/protocol/query/queryutil",
-    "private/protocol/rest",
-    "private/protocol/restxml",
-    "private/protocol/xml/xmlutil",
-    "service/s3",
-    "service/s3/s3iface",
-    "service/sts",
-  ]
-  pruneopts = "UT"
-  revision = "bf8067ceb6e7f51e150c218972dccfeeed892b85"
-  version = "v1.15.54"
-
-[[projects]]
-  branch = "master"
-  digest = "1:5a75ff130630f57609ffca00d6a76dc31bb50ad6a03809848e0f9e13add31987"
-  name = "github.com/cockroachdb/cockroach-go"
-  packages = ["crdb"]
-  pruneopts = "UT"
-  revision = "e0a95dfd547cc9c3ebaaba1a12c2afe4bf621ac5"
-
-[[projects]]
-  branch = "master"
-  digest = "1:3ba3d94d45f2d5c4b3411452474d96996b7e0dce730b689920f7c307511e5655"
-  name = "github.com/cznic/b"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "35e9bbe41f07452a183c517a5fc5f3c9f45eaa0f"
-
-[[projects]]
-  branch = "master"
-  digest = "1:4688e1d4b22367c5bd24fe65f44b3e53a443a93fd19cf34169f1c8337d93dcf7"
-  name = "github.com/cznic/fileutil"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "6a051e75936f623600b67c2b1116b6b6c0ffb936"
-
-[[projects]]
-  branch = "master"
-  digest = "1:2a4afef2ef0e4e4ab66b4f4f6a70bf6551ae8b727184771b5547e2b90245fe58"
-  name = "github.com/cznic/golex"
-  packages = ["lex"]
-  pruneopts = "UT"
-  revision = "4ab7c5e190e49208c823ce8ec803aa39e6a4b31a"
-
-[[projects]]
-  digest = "1:a9e2943ed681d4758a43d1bd9e80d7e993b66ce5adc7aa5dcc083ba48a47a4eb"
-  name = "github.com/cznic/internal"
-  packages = [
-    "buffer",
-    "file",
-    "slice",
-  ]
-  pruneopts = "UT"
-  revision = "cef02a853c3a93623c42eacd574e7ea05f55531b"
-  version = "1.0.0"
-
-[[projects]]
-  digest = "1:ec7b58207ac8eee1c554b7947b3f5124f711238a80edb7224a53bae7263c95ae"
-  name = "github.com/cznic/lldb"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "bea8611dd5c407f3c5eab9f9c68e887a27dc6f0e"
-  version = "v1.1.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:42951d361293c2f7068ab0411afeb57856afba843fe681cea1e2731b47021222"
-  name = "github.com/cznic/mathutil"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "ca4c9f2c136954238c3158b92de72078c7672ecc"
-
-[[projects]]
-  digest = "1:336a34e066968a8e30ee73a0d4419f37d3e8286b94f996cddf46a5e82d13b2fa"
-  name = "github.com/cznic/ql"
-  packages = [
-    ".",
-    "driver",
-    "vendored/github.com/camlistore/go4/lock",
-  ]
-  pruneopts = "UT"
-  revision = "47bf73cf8ed137969f12b223b7016784cdd9d42e"
-  version = "v1.2.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:06877d57543857a87186f43b6195bbdf5e8f4a0f51412fdf4baef504cead7973"
-  name = "github.com/cznic/sortutil"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "4c7342852e65c2088c981288f2c5610d10b9f7f4"
-
-[[projects]]
-  branch = "master"
-  digest = "1:8a26e11e94f1c9f03ab9fc1e816b003629d41b26ab41eefcb7444fad5b24fc4b"
-  name = "github.com/cznic/strutil"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "529a34b1c186b483642a7a230c67521d9aa4b0fb"
-
-[[projects]]
-  branch = "master"
-  digest = "1:bb8236408f88f411f565dffcf1e648fe34c7456820b707f96ccbfb89e1cd8eeb"
-  name = "github.com/cznic/zappy"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "2533cb5b45cc6c07421468ce262899ddc9d53fb7"
-
-[[projects]]
-  digest = "1:8779ca6bbfd7b38a775f89f564834d2f1031fbf337f2eb5f18f115e40ca9a290"
-  name = "github.com/dhui/dktest"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "d72c967df8c2fb5403bfc0c37425986977343f6a"
-  version = "v0.3.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:4ddc17aeaa82cb18c5f0a25d7c253a10682f518f4b2558a82869506eec223d76"
-  name = "github.com/docker/distribution"
-  packages = [
-    "digestset",
-    "reference",
-  ]
-  pruneopts = "UT"
-  revision = "1cb4180b1a5b9c029b2f2beaeb38f0b5cf64e12e"
-
-[[projects]]
-  digest = "1:8a2c57c577e78f40e2b3cd34ba1d050d9afd1060f91552ea03704324d1f50024"
-  name = "github.com/docker/docker"
-  packages = [
-    "api",
-    "api/types",
-    "api/types/blkiodev",
-    "api/types/container",
-    "api/types/events",
-    "api/types/filters",
-    "api/types/image",
-    "api/types/mount",
-    "api/types/network",
-    "api/types/registry",
-    "api/types/strslice",
-    "api/types/swarm",
-    "api/types/swarm/runtime",
-    "api/types/time",
-    "api/types/versions",
-    "api/types/volume",
-    "client",
-    "pkg/jsonmessage",
-    "pkg/term",
-    "pkg/term/windows",
-  ]
-  pruneopts = "UT"
-  revision = "453f2b8b40b923862ce1c08c11531ff5042770f1"
-
-[[projects]]
-  digest = "1:811c86996b1ca46729bad2724d4499014c4b9effd05ef8c71b852aad90deb0ce"
-  name = "github.com/docker/go-connections"
-  packages = [
-    "nat",
-    "sockets",
-    "tlsconfig",
-  ]
-  pruneopts = "UT"
-  revision = "7395e3f8aa162843a74ed6d48e79627d9792ac55"
-  version = "v0.4.0"
-
-[[projects]]
-  digest = "1:6f82cacd0af5921e99bf3f46748705239b36489464f4529a1589bc895764fb18"
-  name = "github.com/docker/go-units"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "47565b4f722fb6ceae66b95f853feed578a4a51c"
-  version = "v0.3.3"
-
-[[projects]]
-  branch = "master"
-  digest = "1:67d0b50be0549e610017cb91e0b0b745ec0cad7c613bc8e18ff2d1c1fc8825a7"
-  name = "github.com/edsrzf/mmap-go"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "0bce6a6887123b67a60366d2c9fe2dfb74289d2e"
-
-[[projects]]
-  digest = "1:812c304ad1cbb214459f7a198cc246a9c8a114a6a92063cef7b31c4ec42a5d6c"
-  name = "github.com/fsouza/fake-gcs-server"
-  packages = [
-    "fakestorage",
-    "internal/backend",
-  ]
-  pruneopts = "UT"
-  revision = "9f9efdc50d6ad4314a4ff85646341f2a9b75ae5f"
-  version = "v1.3.0"
-
-[[projects]]
-  digest = "1:15e27372d379b45b18ac917b9dafc45c45485239490ece18cca97a12f9591146"
-  name = "github.com/go-ini/ini"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "9c8236e659b76e87bf02044d06fde8683008ff3e"
-  version = "v1.39.0"
-
-[[projects]]
-  digest = "1:ec6f9bf5e274c833c911923c9193867f3f18788c461f76f05f62bb1510e0ae65"
-  name = "github.com/go-sql-driver/mysql"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "72cd26f257d44c1114970e19afddcd812016007e"
-  version = "v1.4.1"
-
-[[projects]]
-  digest = "1:586ea76dbd0374d6fb649a91d70d652b7fe0ccffb8910a77468e7702e7901f3d"
-  name = "github.com/go-stack/stack"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "2fee6af1a9795aafbe0253a0cfbdf668e1fb8a9a"
-  version = "v1.8.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:1a5f28249f1a5b118994359a1c7b8c53ee1ff47aa7fe65884fbd20cd237001f4"
-  name = "github.com/gocql/gocql"
-  packages = [
-    ".",
-    "internal/lru",
-    "internal/murmur",
-    "internal/streams",
-  ]
-  pruneopts = "UT"
-  revision = "70385f88b28b43805bd83d212169ab2d38810b15"
-
-[[projects]]
-  digest = "1:bbadccf3d3317ea03c0dac0b45b673b4b397c8f91a1d2eff550a3c51c4ad770e"
-  name = "github.com/gogo/protobuf"
-  packages = ["proto"]
-  pruneopts = "UT"
-  revision = "636bf0302bc95575d69441b25a2603156ffdddf1"
-  version = "v1.1.1"
-
-[[projects]]
-  digest = "1:ec0bde6dc8cf3cb9254f79603bd8489c366a0a8584949cac5a73fa10b9f47c21"
-  name = "github.com/golang/protobuf"
-  packages = [
-    "proto",
-    "protoc-gen-go/descriptor",
-    "ptypes",
-    "ptypes/any",
-    "ptypes/duration",
-    "ptypes/empty",
-    "ptypes/struct",
-    "ptypes/timestamp",
-  ]
-  pruneopts = "UT"
-  revision = "aa810b61a9c79d51363740d207bb46cf8e620ed5"
-  version = "v1.2.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:4a0c6bb4805508a6287675fac876be2ac1182539ca8a32468d8128882e9d5009"
-  name = "github.com/golang/snappy"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "2e65f85255dbc3072edf28d6b5b8efc472979f5a"
-
-[[projects]]
-  digest = "1:f3fb942d9a52aff79191a8318317147ab76ca609bced931b8be9ab1825569993"
-  name = "github.com/google/go-github"
-  packages = ["github"]
-  pruneopts = "UT"
-  revision = "747f93dd9b489bd56f7442b3c42bfea12800a75b"
-  version = "v17.0.0"
-
-[[projects]]
-  digest = "1:a63cff6b5d8b95638bfe300385d93b2a6d9d687734b863da8e09dc834510a690"
-  name = "github.com/google/go-querystring"
-  packages = ["query"]
-  pruneopts = "UT"
-  revision = "44c6ddd0a2342c386950e880b658017258da92fc"
-  version = "v1.0.0"
-
-[[projects]]
-  digest = "1:e145e9710a10bc114a6d3e2738aadf8de146adaa031854ffdf7bbfe15da85e63"
-  name = "github.com/googleapis/gax-go"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "317e0006254c44a0ac427cc52a0e083ff0b9622f"
-  version = "v2.0.0"
-
-[[projects]]
-  digest = "1:c79fb010be38a59d657c48c6ba1d003a8aa651fa56b579d959d74573b7dff8e1"
-  name = "github.com/gorilla/context"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "08b5f424b9271eedf6f9f0ce86cb9396ed337a42"
-  version = "v1.1.1"
-
-[[projects]]
-  digest = "1:e73f5b0152105f18bc131fba127d9949305c8693f8a762588a82a48f61756f5f"
-  name = "github.com/gorilla/mux"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "e3702bed27f0d39777b0b37b664b6280e8ef8fbf"
-  version = "v1.6.2"
-
-[[projects]]
-  branch = "master"
-  digest = "1:364b908b9b27b97ab838f2f6f1b1f46281fa29b978a037d72a9b1d4f6d940190"
-  name = "github.com/hailocab/go-hostpool"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "e80d13ce29ede4452c43dea11e79b9bc8a15b478"
-
-[[projects]]
-  digest = "1:0ade334594e69404d80d9d323445d2297ff8161637f9b2d347cc6973d2d6f05b"
-  name = "github.com/hashicorp/errwrap"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "8a6fb523712970c966eefc6b39ed2c5e74880354"
-  version = "v1.0.0"
-
-[[projects]]
-  digest = "1:f668349b83f7d779567c880550534addeca7ebadfdcf44b0b9c39be61864b4b7"
-  name = "github.com/hashicorp/go-multierror"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "886a7fbe3eb1c874d46f623bfa70af45f425b3d1"
-  version = "v1.0.0"
-
-[[projects]]
-  digest = "1:8dfbec76b4834d96c37ba4c4969fc47c9a15e67c5a7eecfa031fbe67dab50429"
-  name = "github.com/jackc/pgx"
-  packages = [
-    ".",
-    "chunkreader",
-    "internal/sanitize",
-    "pgio",
-    "pgproto3",
-    "pgtype",
-  ]
-  pruneopts = "UT"
-  revision = "89f1e6ac7276b61d885db5e5aed6fcbedd1c7e31"
-  version = "v3.2.0"
-
-[[projects]]
-  digest = "1:e22af8c7518e1eab6f2eab2b7d7558927f816262586cd6ed9f349c97a6c285c4"
-  name = "github.com/jmespath/go-jmespath"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "0b12d6b5"
-
-[[projects]]
-  digest = "1:0a69a1c0db3591fcefb47f115b224592c8dfa4368b7ba9fae509d5e16cdc95c8"
-  name = "github.com/konsorten/go-windows-terminal-sequences"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "5c8c8bd35d3832f5d134ae1e1e375b69a4d25242"
-  version = "v1.0.1"
-
-[[projects]]
-  digest = "1:f02dbf2124e5dfb5305a6c3c8957a41d54a89d2a3ef60f030769f86e7851d5f5"
-  name = "github.com/kshvakov/clickhouse"
-  packages = [
-    ".",
-    "lib/binary",
-    "lib/column",
-    "lib/data",
-    "lib/protocol",
-    "lib/types",
-    "lib/writebuffer",
-  ]
-  pruneopts = "UT"
-  revision = "aeedb7b9d0584f393905447021238b5c6e9c5154"
-  version = "v1.3.4"
-
-[[projects]]
-  digest = "1:8ef506fc2bb9ced9b151dafa592d4046063d744c646c1bbe801982ce87e4bc24"
-  name = "github.com/lib/pq"
-  packages = [
-    ".",
-    "oid",
-  ]
-  pruneopts = "UT"
-  revision = "4ded0e9383f75c197b3a2aaa6d590ac52df6fd79"
-  version = "v1.0.0"
-
-[[projects]]
-  digest = "1:3cafc6a5a1b8269605d9df4c6956d43d8011fc57f266ca6b9d04da6c09dee548"
-  name = "github.com/mattn/go-sqlite3"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "25ecb14adfc7543176f7d85291ec7dba82c6f7e4"
-  version = "v1.9.0"
-
-[[projects]]
-  digest = "1:820eeb72f78947f9b2e87bd99a57b5cc791c8fb47676a5c3824d408441ab1546"
-  name = "github.com/mongodb/mongo-go-driver"
-  packages = [
-    "bson",
-    "bson/bsoncodec",
-    "bson/bsonrw",
-    "bson/bsontype",
-    "bson/primitive",
-    "event",
-    "internal",
-    "mongo",
-    "mongo/options",
-    "mongo/readconcern",
-    "mongo/readpref",
-    "mongo/writeconcern",
-    "tag",
-    "version",
-    "x/bsonx",
-    "x/bsonx/bsoncore",
-    "x/mongo/driver",
-    "x/mongo/driver/auth",
-    "x/mongo/driver/auth/internal/gssapi",
-    "x/mongo/driver/session",
-    "x/mongo/driver/topology",
-    "x/mongo/driver/uuid",
-    "x/network/address",
-    "x/network/command",
-    "x/network/compressor",
-    "x/network/connection",
-    "x/network/connstring",
-    "x/network/description",
-    "x/network/result",
-    "x/network/wiremessage",
-  ]
-  pruneopts = "UT"
-  revision = "29905d4bda472574c8b499e2f93b5b2747d8fbd7"
-  version = "v0.1.0"
-
-[[projects]]
-  digest = "1:ee4d4af67d93cc7644157882329023ce9a7bcfce956a079069a9405521c7cc8d"
-  name = "github.com/opencontainers/go-digest"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "279bed98673dd5bef374d3b6e4b09e2af76183bf"
-  version = "v1.0.0-rc1"
-
-[[projects]]
-  digest = "1:11db38d694c130c800d0aefb502fb02519e514dc53d9804ce51d1ad25ec27db6"
-  name = "github.com/opencontainers/image-spec"
-  packages = [
-    "specs-go",
-    "specs-go/v1",
-  ]
-  pruneopts = "UT"
-  revision = "d60099175f88c47cd379c4738d158884749ed235"
-  version = "v1.0.1"
-
-[[projects]]
-  digest = "1:40e195917a951a8bf867cd05de2a46aaf1806c50cf92eebf4c16f78cd196f747"
-  name = "github.com/pkg/errors"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
-  version = "v0.8.0"
-
-[[projects]]
-  digest = "1:87c2e02fb01c27060ccc5ba7c5a407cc91147726f8f40b70cceeedbc52b1f3a8"
-  name = "github.com/sirupsen/logrus"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "e1e72e9de974bd926e5c56f83753fba2df402ce5"
-  version = "v1.3.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:40fdfd6ab85ca32b6935853bbba35935dcb1d796c8135efd85947566c76e662e"
-  name = "github.com/xdg/scram"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "7eeb5667e42c09cb51bf7b7c28aea8c56767da90"
-
-[[projects]]
-  branch = "master"
-  digest = "1:f5c1d04bc09c644c592b45b9f0bad4030521b1a7d11c7dadbb272d9439fa6e8e"
-  name = "github.com/xdg/stringprep"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "73f8eece6fdcd902c185bf651de50f3828bed5ed"
-
-[[projects]]
-  digest = "1:8e7fcf54cc7c0c3aba7801812ae70343585b3cb663f6cb11ad50c0c28552f359"
-  name = "go.opencensus.io"
-  packages = [
-    ".",
-    "internal",
-    "internal/tagencoding",
-    "plugin/ocgrpc",
-    "plugin/ochttp",
-    "plugin/ochttp/propagation/b3",
-    "stats",
-    "stats/internal",
-    "stats/view",
-    "tag",
-    "trace",
-    "trace/internal",
-    "trace/propagation",
-    "trace/tracestate",
-  ]
-  pruneopts = "UT"
-  revision = "79993219becaa7e29e3b60cb67f5b8e82dee11d6"
-  version = "v0.17.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:9ff2b9f8d4e47013a213b9ea0d850ed787c51f5eab2038842d82397ad85a19c1"
-  name = "golang.org/x/crypto"
-  packages = [
-    "pbkdf2",
-    "ssh/terminal",
-  ]
-  pruneopts = "UT"
-  revision = "8d7daa0c54b357f3071e11eaef7efc4e19a417e2"
-
-[[projects]]
-  branch = "master"
-  digest = "1:56975096c005d57a2c9c2faefbec772c44c29c084af35facbfca85961469ff6d"
-  name = "golang.org/x/net"
-  packages = [
-    "context",
-    "context/ctxhttp",
-    "http/httpguts",
-    "http2",
-    "http2/hpack",
-    "idna",
-    "internal/socks",
-    "internal/timeseries",
-    "proxy",
-    "trace",
-  ]
-  pruneopts = "UT"
-  revision = "49bb7cea24b1df9410e1712aa6433dae904ff66a"
-
-[[projects]]
-  branch = "master"
-  digest = "1:39273b3e6c96fed52bd8f7793fc463c40b1a27a7caa51302b81d45306b9646b4"
-  name = "golang.org/x/oauth2"
-  packages = [
-    ".",
-    "google",
-    "internal",
-    "jws",
-    "jwt",
-  ]
-  pruneopts = "UT"
-  revision = "c57b0facaced709681d9f90397429b9430a74754"
-
-[[projects]]
-  branch = "master"
-  digest = "1:75515eedc0dc2cb0b40372008b616fa2841d831c63eedd403285ff286c593295"
-  name = "golang.org/x/sync"
-  packages = ["semaphore"]
-  pruneopts = "UT"
-  revision = "37e7f081c4d4c64e13b10787722085407fe5d15f"
-
-[[projects]]
-  branch = "master"
-  digest = "1:f5aa274a0377f85735edc7fedfb0811d3cbc20af91633797cb359e29c3272271"
-  name = "golang.org/x/sys"
-  packages = [
-    "unix",
-    "windows",
-  ]
-  pruneopts = "UT"
-  revision = "fa43e7bc11baaae89f3f902b2b4d832b68234844"
-
-[[projects]]
-  digest = "1:a2ab62866c75542dd18d2b069fec854577a20211d7c0ea6ae746072a1dccdd18"
-  name = "golang.org/x/text"
-  packages = [
-    "collate",
-    "collate/build",
-    "internal/colltab",
-    "internal/gen",
-    "internal/tag",
-    "internal/triegen",
-    "internal/ucd",
-    "language",
-    "secure/bidirule",
-    "transform",
-    "unicode/bidi",
-    "unicode/cldr",
-    "unicode/norm",
-    "unicode/rangetable",
-  ]
-  pruneopts = "UT"
-  revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
-  version = "v0.3.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:fc393bc931c39e70d322b5158a90ef38e1a1617afdffb4efaaa4567f5c4f0410"
-  name = "golang.org/x/tools"
-  packages = [
-    "godoc/vfs",
-    "godoc/vfs/mapfs",
-  ]
-  pruneopts = "UT"
-  revision = "8919434dde1e35f7b9ef589ecb80ec0833f021bc"
-
-[[projects]]
-  branch = "master"
-  digest = "1:c8a31b4403741a053bf08594b91189ec63c96b041bdfddef06a272f42549f2cf"
-  name = "google.golang.org/api"
-  packages = [
-    "gensupport",
-    "googleapi",
-    "googleapi/internal/uritemplates",
-    "googleapi/transport",
-    "internal",
-    "iterator",
-    "option",
-    "storage/v1",
-    "transport",
-    "transport/grpc",
-    "transport/http",
-    "transport/http/internal/propagation",
-  ]
-  pruneopts = "UT"
-  revision = "625cd1887957946515db468ce519bb71fa31fc7f"
-
-[[projects]]
-  digest = "1:d2cfb607095cee410f25a3c04eed77f76c4721d1bb0cbc894f5d8624b33394c8"
-  name = "google.golang.org/appengine"
-  packages = [
-    ".",
-    "cloudsql",
-    "internal",
-    "internal/app_identity",
-    "internal/base",
-    "internal/datastore",
-    "internal/log",
-    "internal/modules",
-    "internal/remote_api",
-    "internal/socket",
-    "internal/urlfetch",
-    "socket",
-    "urlfetch",
-  ]
-  pruneopts = "UT"
-  revision = "ae0ab99deb4dc413a2b4bd6c8bdd0eb67f1e4d06"
-  version = "v1.2.0"
-
-[[projects]]
-  branch = "master"
-  digest = "1:315f8fe35ad73727d15b20fc3a533e7ab9ea8f0c3ff6441753d04086e5ca5003"
-  name = "google.golang.org/genproto"
-  packages = [
-    "googleapis/api/annotations",
-    "googleapis/iam/v1",
-    "googleapis/longrunning",
-    "googleapis/rpc/code",
-    "googleapis/rpc/errdetails",
-    "googleapis/rpc/status",
-    "googleapis/spanner/admin/database/v1",
-    "googleapis/spanner/v1",
-  ]
-  pruneopts = "UT"
-  revision = "af9cb2a35e7f169ec875002c1829c9b315cddc04"
-
-[[projects]]
-  digest = "1:4ad047d772a7d4841687df9534a767a9e243885ed6d7d2ced6af5995a5dc44b3"
-  name = "google.golang.org/grpc"
-  packages = [
-    ".",
-    "balancer",
-    "balancer/base",
-    "balancer/roundrobin",
-    "codes",
-    "connectivity",
-    "credentials",
-    "credentials/oauth",
-    "encoding",
-    "encoding/proto",
-    "grpclog",
-    "internal",
-    "internal/backoff",
-    "internal/channelz",
-    "internal/envconfig",
-    "internal/grpcrand",
-    "internal/transport",
-    "keepalive",
-    "metadata",
-    "naming",
-    "peer",
-    "resolver",
-    "resolver/dns",
-    "resolver/passthrough",
-    "stats",
-    "status",
-    "tap",
-  ]
-  pruneopts = "UT"
-  revision = "8dea3dc473e90c8179e519d91302d0597c0ca1d1"
-  version = "v1.15.0"
-
-[[projects]]
-  digest = "1:2d1fbdc6777e5408cabeb02bf336305e724b925ff4546ded0fa8715a7267922a"
-  name = "gopkg.in/inf.v0"
-  packages = ["."]
-  pruneopts = "UT"
-  revision = "d2d2541c53f18d2a059457998ce2876cc8e67cbf"
-  version = "v0.9.1"
-
-[solve-meta]
-  analyzer-name = "dep"
-  analyzer-version = 1
-  input-imports = [
-    "cloud.google.com/go/spanner",
-    "cloud.google.com/go/spanner/admin/database/apiv1",
-    "cloud.google.com/go/storage",
-    "github.com/aws/aws-sdk-go/aws",
-    "github.com/aws/aws-sdk-go/aws/session",
-    "github.com/aws/aws-sdk-go/service/s3",
-    "github.com/aws/aws-sdk-go/service/s3/s3iface",
-    "github.com/cockroachdb/cockroach-go/crdb",
-    "github.com/cznic/ql/driver",
-    "github.com/dhui/dktest",
-    "github.com/docker/docker/api/types",
-    "github.com/docker/docker/api/types/container",
-    "github.com/docker/docker/api/types/network",
-    "github.com/docker/docker/client",
-    "github.com/fsouza/fake-gcs-server/fakestorage",
-    "github.com/go-sql-driver/mysql",
-    "github.com/gocql/gocql",
-    "github.com/google/go-github/github",
-    "github.com/hashicorp/go-multierror",
-    "github.com/kshvakov/clickhouse",
-    "github.com/lib/pq",
-    "github.com/mattn/go-sqlite3",
-    "github.com/mongodb/mongo-go-driver/bson",
-    "github.com/mongodb/mongo-go-driver/mongo",
-    "github.com/mongodb/mongo-go-driver/x/network/connstring",
-    "golang.org/x/net/context",
-    "golang.org/x/tools/godoc/vfs",
-    "golang.org/x/tools/godoc/vfs/mapfs",
-    "google.golang.org/api/iterator",
-    "google.golang.org/genproto/googleapis/spanner/admin/database/v1",
-  ]
-  solver-name = "gps-cdcl"
-  solver-version = 1

Gopkg.toml

@@ -1,110 +0,0 @@
-# Gopkg.toml example
-#
-# Refer to https://golang.github.io/dep/docs/Gopkg.toml.html
-# for detailed Gopkg.toml documentation.
-#
-# required = ["github.com/user/thing/cmd/thing"]
-# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
-#
-# [[constraint]]
-#   name = "github.com/user/project"
-#   version = "1.0.0"
-#
-# [[constraint]]
-#   name = "github.com/user/project2"
-#   branch = "dev"
-#   source = "github.com/myfork/project2"
-#
-# [[override]]
-#   name = "github.com/x/y"
-#   version = "2.4.0"
-#
-# [prune]
-#   non-go = false
-#   go-tests = true
-#   unused-packages = true
-
-
-[[constraint]]
-  name = "github.com/aws/aws-sdk-go"
-  version = "1.15.34"
-
-[[constraint]]
-  branch = "master"
-  name = "github.com/cockroachdb/cockroach-go"
-
-[[constraint]]
-  name = "github.com/cznic/ql"
-  version = "1.2.0"
-
-[[constraint]]
-  name = "github.com/docker/docker"
-  revision = "453f2b8b40b923862ce1c08c11531ff5042770f1"
-
-[[constraint]]
-  name = "github.com/fsouza/fake-gcs-server"
-  version = "1.2.0"
-
-[[constraint]]
-  name = "github.com/go-sql-driver/mysql"
-  version = "v1.4.0"
-
-[[constraint]]
-  branch = "master"
-  name = "github.com/gocql/gocql"
-
-[[constraint]]
-  name = "github.com/google/go-github"
-  version = "17.0.0"
-
-[[constraint]]
-  name = "github.com/kshvakov/clickhouse"
-  version = "1.3.4"
-
-[[constraint]]
-  name = "github.com/lib/pq"
-  version = "1.0.0"
-
-[[constraint]]
-  name = "github.com/mattn/go-sqlite3"
-  version = "1.9.0"
-
-[[constraint]]
-  branch = "master"
-  name = "golang.org/x/net"
-
-[[constraint]]
-  branch = "master"
-  name = "google.golang.org/api"
-
-[[constraint]]
-  branch = "master"
-  name = "google.golang.org/genproto"
-
-[[constraint]]
-  name = "github.com/mongodb/mongo-go-driver"
-  version = "0.1.0"
-
-[[override]]
-  name = "cloud.google.com/go"
-  version = "0.27.0"
-
-[[override]]
-  branch = "master"
-  name = "golang.org/x/tools"
-
-[[override]]
-  branch = "master"
-  name = "github.com/docker/distribution"
-
-[prune]
-  go-tests = true
-  unused-packages = true
-
-[[constraint]]
-  name = "github.com/dhui/dktest"
-  version = "0.3.0"
-
-[[constraint]]
-  name = "github.com/hashicorp/go-multierror"
-  version = "1.0.0"

LLM.md

@@ -0,0 +1,39 @@
+# Hanzo Golang Migrate
+
+## Overview
+Go module: github.com/golang-migrate/migrate/v4
+
+## Tech Stack
+- **Language**: Go
+
+## Build & Run
+```bash
+go build ./...
+go test ./...
+```
+
+## Structure
+```
+golang-migrate/
+  CONTRIBUTING.md
+  Dockerfile
+  Dockerfile.circleci
+  Dockerfile.github-actions
+  FAQ.md
+  GETTING_STARTED.md
+  LICENSE
+  MIGRATIONS.md
+  Makefile
+  README.md
+  SECURITY.md
+  cli/
+  cmd/
+  database/
+  dktesting/
+```
+
+## Key Files
+- `README.md` -- Project documentation
+- `go.mod` -- Go module definition
+- `Makefile` -- Build automation
+- `Dockerfile` -- Container build

MIGRATIONS.md

@@ -44,9 +44,14 @@ It is suggested that the version number of corresponding `up` and `down` migrati
 files be equivalent for clarity, but they are allowed to differ so long as the
 relative ordering of the migrations is preserved.
 
-The migration files are permitted to be empty, so in the event that a migration
-is a no-op or is irreversible, it is recommended to still include both migration
-files, and either leaving them empty or adding a comment as appropriate.
+The migration files are permitted to be "empty", in the event that a migration
+is a no-op or is irreversible. It is recommended to still include both migration
+files by making the whole migration file consist of a comment.
+If your database does not support comments, then deleting the migration file will also work.
+Note, an actual empty file (e.g. a 0 byte file) may cause issues with your database since migrate
+will attempt to run an empty query. In this case, deleting the migration file will also work.
+For the rational of this behavior see:
+[#244 (comment)](https://github.com/golang-migrate/migrate/issues/244#issuecomment-510758270)
 
 ## Migration Content Format
 

Makefile

@@ -1,15 +1,24 @@
-SOURCE ?= file go_bindata github aws_s3 google_cloud_storage godoc_vfs
-DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb clickhouse mongodb
+SOURCE ?= file go_bindata github github_ee bitbucket aws_s3 google_cloud_storage godoc_vfs gitlab
+DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb yugabytedb datastore mongodb sqlserver firebird neo4j pgx pgx5 rqlite
+DATABASE_TEST ?= $(DATABASE) sqlite sqlite3 sqlcipher
 VERSION ?= $(shell git describe --tags 2>/dev/null | cut -c 2-)
 TEST_FLAGS ?=
 REPO_OWNER ?= $(shell cd .. && basename "$$(pwd)")
 COVERAGE_DIR ?= .coverage
 
+build:
+	CGO_ENABLED=0 go build -ldflags='-X main.Version=$(VERSION)' -tags '$(DATABASE) $(SOURCE)' ./cmd/migrate
+
+build-docker:
+	CGO_ENABLED=0 go build -a -o build/migrate.linux-386 -ldflags="-s -w -X main.Version=${VERSION}" -tags "$(DATABASE) $(SOURCE)" ./cmd/migrate
 
 build-cli: clean
 	-mkdir ./cli/build
 	cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o ../../cli/build/migrate.linux-amd64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
+	cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -a -o ../../cli/build/migrate.linux-armv7 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
+	cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -o ../../cli/build/migrate.linux-arm64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
 	cd ./cmd/migrate && CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -a -o ../../cli/build/migrate.darwin-amd64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
+	cd ./cmd/migrate && CGO_ENABLED=0 GOOS=windows GOARCH=386 go build -a -o ../../cli/build/migrate.windows-386.exe -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
 	cd ./cmd/migrate && CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -a -o ../../cli/build/migrate.windows-amd64.exe -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
 	cd ./cli/build && find . -name 'migrate*' | xargs -I{} tar czf {}.tar.gz {}
 	cd ./cli/build && shasum -a 256 * > sha256sum.txt
@@ -27,27 +36,14 @@ test-short:
 test:
 	@-rm -r $(COVERAGE_DIR)
 	@mkdir $(COVERAGE_DIR)
-	make test-with-flags TEST_FLAGS='-v -race -covermode atomic -coverprofile $$(COVERAGE_DIR)/_$$(RAND).txt -bench=. -benchmem -timeout 20m'
-	@echo 'mode: atomic' > $(COVERAGE_DIR)/combined.txt
-	@cat $(COVERAGE_DIR)/_*.txt | grep -v 'mode: atomic' >> $(COVERAGE_DIR)/combined.txt
+	make test-with-flags TEST_FLAGS='-v -race -covermode atomic -coverprofile $$(COVERAGE_DIR)/combined.txt -bench=. -benchmem -timeout 20m'
 
 
 test-with-flags:
-	@echo SOURCE: $(SOURCE) 
-	@echo DATABASE: $(DATABASE)
+	@echo SOURCE: $(SOURCE)
+	@echo DATABASE_TEST: $(DATABASE_TEST)
 
-	@go test $(TEST_FLAGS) .
-	@go test $(TEST_FLAGS) ./cli/...
-	@go test $(TEST_FLAGS) ./database
-	@go test $(TEST_FLAGS) ./testing/...
-
-	@echo -n '$(SOURCE)' | tr -s ' ' '\n' | xargs -I{} go test $(TEST_FLAGS) ./source/{}
-	@go test $(TEST_FLAGS) ./source/testing/...
-	@go test $(TEST_FLAGS) ./source/stub/...
-
-	@echo -n '$(DATABASE)' | tr -s ' ' '\n' | xargs -I{} go test $(TEST_FLAGS) ./database/{}
-	@go test $(TEST_FLAGS) ./database/testing/...
-	@go test $(TEST_FLAGS) ./database/stub/...
+	@go test $(TEST_FLAGS) ./...
 
 
 kill-orphaned-docker-containers:
@@ -84,7 +80,7 @@ rewrite-import-paths:
 docs:
 	-make kill-docs
 	nohup godoc -play -http=127.0.0.1:6064 </dev/null >/dev/null 2>&1 & echo $$! > .godoc.pid
-	cat .godoc.pid  
+	cat .godoc.pid
 
 
 kill-docs:
@@ -102,6 +98,12 @@ release:
 	git tag v$(V)
 	@read -p "Press enter to confirm and push to origin ..." && git push origin v$(V)
 
+echo-source:
+	@echo "$(SOURCE)"
+
+echo-database:
+	@echo "$(DATABASE)"
+
 
 define external_deps
 	@echo '-- $(1)';  go list -f '{{join .Deps "\n"}}' $(1) | grep -v github.com/$(REPO_OWNER)/migrate | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}'
@@ -109,10 +111,10 @@ define external_deps
 endef
 
 
-.PHONY: build-cli clean test-short test test-with-flags html-coverage \
+.PHONY: build build-docker build-cli clean test-short test test-with-flags html-coverage \
         restore-import-paths rewrite-import-paths list-external-deps release \
-        docs kill-docs open-docs kill-orphaned-docker-containers
+		docs kill-docs open-docs kill-orphaned-docker-containers echo-source echo-database
 
-SHELL = /bin/bash
+SHELL = /bin/sh
 RAND = $(shell echo $$RANDOM)
 

README.md

@@ -1,21 +1,23 @@
-[![Build Status](https://img.shields.io/travis/com/golang-migrate/migrate/master.svg)](https://travis-ci.com/golang-migrate/migrate)
-[![GoDoc](https://godoc.org/github.com/golang-migrate/migrate?status.svg)](https://godoc.org/github.com/golang-migrate/migrate)
+<p align="center"><img src=".github/hero.svg" alt="migrate" width="880"></p>
+
+[![GitHub Workflow Status (branch)](https://img.shields.io/github/actions/workflow/status/golang-migrate/migrate/ci.yaml?branch=master)](https://github.com/golang-migrate/migrate/actions/workflows/ci.yaml?query=branch%3Amaster)
+[![GoDoc](https://pkg.go.dev/badge/github.com/golang-migrate/migrate)](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)
 [![Coverage Status](https://img.shields.io/coveralls/github/golang-migrate/migrate/master.svg)](https://coveralls.io/github/golang-migrate/migrate?branch=master)
 [![packagecloud.io](https://img.shields.io/badge/deb-packagecloud.io-844fec.svg)](https://packagecloud.io/golang-migrate/migrate?filter=debs)
 [![Docker Pulls](https://img.shields.io/docker/pulls/migrate/migrate.svg)](https://hub.docker.com/r/migrate/migrate/)
-![Supported Go Versions](https://img.shields.io/badge/Go-1.10%2C%201.11-lightgrey.svg)
+![Supported Go Versions](https://img.shields.io/badge/Go-1.24%2C%201.25-lightgrey.svg)
 [![GitHub Release](https://img.shields.io/github/release/golang-migrate/migrate.svg)](https://github.com/golang-migrate/migrate/releases)
-
+[![Go Report Card](https://goreportcard.com/badge/github.com/golang-migrate/migrate/v4)](https://goreportcard.com/report/github.com/golang-migrate/migrate/v4)
 
 # migrate
 
 __Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__
 
- * Migrate reads migrations from [sources](#migration-sources)
+* Migrate reads migrations from [sources](#migration-sources)
    and applies them in correct order to a [database](#databases).
- * Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof.
+* Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof.
    (Keeps the drivers lightweight, too.)
- * Database drivers don't assume things or try to correct user input. When in doubt, fail.
+* Database drivers don't assume things or try to correct user input. When in doubt, fail.
 
 Forked from [mattes/migrate](https://github.com/mattes/migrate)
 
@@ -23,23 +25,31 @@ Forked from [mattes/migrate](https://github.com/mattes/migrate)
 
 Database drivers run migrations. [Add a new database?](database/driver.go)
 
-  * [PostgreSQL](database/postgres)
-  * [Redshift](database/redshift)
-  * [Ql](database/ql)
-  * [Cassandra](database/cassandra)
-  * [SQLite](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165))
-  * [MySQL/ MariaDB](database/mysql)
-  * [Neo4j](database/neo4j) ([todo #167](https://github.com/mattes/migrate/issues/167))
-  * [MongoDB](database/mongodb)
-  * [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170))
-  * [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171))
-  * [Google Cloud Spanner](database/spanner)
-  * [CockroachDB](database/cockroachdb)
-  * [ClickHouse](database/clickhouse)
+* [PostgreSQL](database/postgres)
+* [PGX v4](database/pgx)
+* [PGX v5](database/pgx/v5)
+* [Redshift](database/redshift)
+* [Ql](database/ql)
+* [Cassandra / ScyllaDB](database/cassandra)
+* [SQLite](database/sqlite)
+* [SQLite3](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165))
+* [SQLCipher](database/sqlcipher)
+* [MySQL / MariaDB](database/mysql)
+* [Neo4j](database/neo4j)
+* [MongoDB](database/mongodb)
+* [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170))
+* [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171))
+* [Google Cloud Spanner](database/spanner)
+* [CockroachDB](database/cockroachdb)
+* [YugabyteDB](database/yugabytedb)
+* [Datastore](database/datastore)
+* [Firebird](database/firebird)
+* [MS SQL Server](database/sqlserver)
+* [rqlite](database/rqlite)
 
 ### Database URLs
 
-Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: `dbdriver://username:password@host:port/dbname?option1=true&option2=false`
+Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: `dbdriver://username:password@host:port/dbname?param1=true&param2=false`
 
 Any [reserved URL characters](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters) need to be escaped. Note, the `%` character also [needs to be escaped](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_the_percent_character)
 
@@ -47,6 +57,7 @@ Explicitly, the following characters need to be escaped:
 `!`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `/`, `:`, `;`, `=`, `?`, `@`, `[`, `]`
 
 It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:
+
 ```bash
 $ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
 String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
@@ -61,45 +72,48 @@ $
 
 Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go)
 
-  * [Filesystem](source/file) - read from fileystem
-  * [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata))
-  * [Github](source/github) - read from remote Github repositories
-  * [AWS S3](source/aws_s3) - read from Amazon Web Services S3
-  * [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage
-
-
+* [Filesystem](source/file) - read from filesystem
+* [io/fs](source/iofs) - read from a Go [io/fs](https://pkg.go.dev/io/fs#FS)
+* [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata))
+* [pkger](source/pkger) - read from embedded binary data ([markbates/pkger](https://github.com/markbates/pkger))
+* [GitHub](source/github) - read from remote GitHub repositories
+* [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories
+* [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories
+* [Gitlab](source/gitlab) - read from remote Gitlab repositories
+* [AWS S3](source/aws_s3) - read from Amazon Web Services S3
+* [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage
 
 ## CLI usage
 
-  * Simple wrapper around this library.
-  * Handles ctrl+c (SIGINT) gracefully.
-  * No config search paths, no config files, no magic ENV var injections.
+* Simple wrapper around this library.
+* Handles ctrl+c (SIGINT) gracefully.
+* No config search paths, no config files, no magic ENV var injections.
 
-__[CLI Documentation](cli)__
+[CLI Documentation](cmd/migrate) (includes CLI install instructions)
 
-### Basic usage:
+### Basic usage
 
-```
+```bash
 $ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2
 ```
 
 ### Docker usage
 
-```
-$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate 
+```bash
+$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
     -path=/migrations/ -database postgres://localhost:5432/database up 2
 ```
 
 ## Use in your Go project
 
- * API is stable and frozen for this release (v3 & v4).
- * Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies.
- * To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`.
- * Bring your own logger.
- * Uses `io.Reader` streams internally for low memory overhead.
- * Thread-safe and no goroutine leaks.
+* API is stable and frozen for this release (v3 & v4).
+* Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies.
+* To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`.
+* Bring your own logger.
+* Uses `io.Reader` streams internally for low memory overhead.
+* Thread-safe and no goroutine leaks.
 
-__[Go Documentation](https://godoc.org/github.com/golang-migrate/migrate)__
+__[Go Documentation](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)__
 
 ```go
 import (
@@ -133,28 +147,44 @@ func main() {
     m, err := migrate.NewWithDatabaseInstance(
         "file:///migrations",
         "postgres", driver)
-    m.Steps(2)
+    m.Up() // or m.Steps(2) if you want to explicitly set the number of migrations to run
 }
 ```
 
+## Getting started
+
+Go to [getting started](GETTING_STARTED.md)
+
+## Tutorials
+
+* [CockroachDB](database/cockroachdb/TUTORIAL.md)
+* [PostgreSQL](database/postgres/TUTORIAL.md)
+
+(more tutorials to come)
+
 ## Migration files
 
 Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration)
 
-```
+```bash
 1481574547_create_users_table.up.sql
 1481574547_create_users_table.down.sql
 ```
 
 [Best practices: How to write migrations.](MIGRATIONS.md)
 
-## Supported Major Versions
+## Coming from another db migration tool?
 
-Version | Go Modules | Min Go Version | Import | Notes
---------|------------|----------------|--------|------
-**master** | :white_check_mark: | 1.10.3 | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first |
-**v4** | :white_check_mark: | 1.10.3 | `import "github.com/golang-migrate/migrate/v4"` | |
-**v3** | :x: | 1.10 | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | Only security fixes are backported. Only use if your project is not using Go modules. |
+Check out [migradaptor](https://github.com/musinit/migradaptor/).
+*Note: migradaptor is not affiliated or supported by this project*
+
+## Versions
+
+Version | Supported? | Import | Notes
+--------|------------|--------|------
+**master** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first |
+**v4** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | Used for stable releases |
+**v3** | :x: | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | **DO NOT USE** - No longer supported |
 
 ## Development and Contributing
 
@@ -163,8 +193,6 @@ read the [development guide](CONTRIBUTING.md).
 
 Also have a look at the [FAQ](FAQ.md).
 
-
-
 ---
 
 Looking for alternatives? [https://awesome-go.com/#database](https://awesome-go.com/#database).

SECURITY.md

@@ -0,0 +1,16 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| master  | :white_check_mark: |
+| 4.x     | :white_check_mark: |
+| 3.x     | :x:                |
+| < 3.0   | :x:                |
+
+## Reporting a Vulnerability
+
+We prefer [coordinated disclosures](https://en.wikipedia.org/wiki/Coordinated_vulnerability_disclosure). To start one, create a GitHub security advisory following [these instructions](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability)
+
+Please suggest potential impact and urgency in your reports.

cli/README.md

@@ -1,125 +1,3 @@
-# migrate CLI
+# Deprecated
 
-## Installation
-
-#### Download pre-build binary (Windows, MacOS, or Linux)
-
-[Release Downloads](https://github.com/golang-migrate/migrate/releases)
-
-```
-$ curl -L https://github.com/golang-migrate/migrate/releases/download/$version/migrate.$platform-amd64.tar.gz | tar xvz
-```
-
-#### MacOS
-
-```
-$ brew install golang-migrate
-```
-
-#### Linux (*.deb package)
-
-```
-$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add -
-$ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
-$ apt-get update
-$ apt-get install -y migrate
-```
-
-#### With Go toolchain
-
-##### Unversioned
-```
-$ go get -tags 'postgres' -u github.com/golang-migrate/migrate/cmd/migrate
-```
-
-##### Versioned
-
-```
-$ go get -u -d github.com/golang-migrate/migrate/cmd/migrate
-$ cd $GOPATH/src/github.com/golang-migrate/migrate/cmd/migrate
-$ git checkout $TAG  # e.g. v4.1.0
-$ go build -tags 'postgres' -ldflags="-X main.Version=$(git describe --tags)" -o $GOPATH/bin/migrate github.com/golang-migrate/migrate/cmd/migrate
-```
-
-##### Notes:
-1. Requires a version of Go that [supports modules](https://golang.org/cmd/go/#hdr-Preliminary_module_support). e.g. Go 1.11+
-1. This example builds the cli which will only work with postgres.  In order
-to build the cli for use with other databases, replace the `postgres` build tag
-with the appropriate database tag(s) for the databases desired.  The tags
-correspond to the names of the sub-packages underneath the
-[`database`](../database) package.
-1. Similarly to the database build tags, if you need to support other sources, use the appropriate build tag(s).
-1. Support for build constraints will be removed in the future: https://github.com/golang-migrate/migrate/issues/60
-
-
-## Usage
-
-```
-$ migrate -help
-Usage: migrate OPTIONS COMMAND [arg...]
-       migrate [ -version | -help ]
-
-Options:
-  -source          Location of the migrations (driver://url)
-  -path            Shorthand for -source=file://path
-  -database        Run migrations against this database (driver://url)
-  -prefetch N      Number of migrations to load in advance before executing (default 10)
-  -lock-timeout N  Allow N seconds to acquire database lock (default 15)
-  -verbose         Print verbose logging
-  -version         Print version
-  -help            Print usage
-
-Commands:
-  create [-ext E] [-dir D] [-seq] [-digits N] [-format] NAME
-               Create a set of timestamped up/down migrations titled NAME, in directory D with extension E.
-               Use -seq option to generate sequential up/down migrations with N digits.
-               Use -format option to specify a Go time format string.
-  goto V       Migrate to version V
-  up [N]       Apply all or N up migrations
-  down [N]     Apply all or N down migrations
-  drop         Drop everyting inside database
-  force V      Set version V but don't run migration (ignores dirty state)
-  version      Print current migration version
-```
-
-
-So let's say you want to run the first two migrations
-
-```
-$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2
-```
-
-If your migrations are hosted on github
-
-```
-$ migrate -source github://mattes:personal-access-token@mattes/migrate_test \
-    -database postgres://localhost:5432/database down 2
-```
-
-The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received.
-Send SIGKILL for immediate halt.
-
-
-
-## Reading CLI arguments from somewhere else
-
-##### ENV variables
-
-```
-$ migrate -database "$MY_MIGRATE_DATABASE"
-```
-
-##### JSON files
-
-Check out https://stedolan.github.io/jq/
-
-```
-$ migrate -database "$(cat config.json | jq '.database')"
-```
-
-##### YAML files
-
-````
-$ migrate -database "$(cat config/database.yml | ruby -ryaml -e "print YAML.load(STDIN.read)['database']")"
-$ migrate -database "$(cat config/database.yml | python -c 'import yaml,sys;print yaml.safe_load(sys.stdin)["database"]')"
-```
+Use [cmd/migrate](../cmd/migrate) instead

cli/examples/Dockerfile

@@ -1,12 +0,0 @@
-FROM ubuntu:xenial
-
-RUN apt-get update && \
-    apt-get install -y curl apt-transport-https
-
-RUN curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add - && \
-    echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ xenial main" > /etc/apt/sources.list.d/migrate.list && \
-    apt-get update && \
-    apt-get install -y migrate
-
-RUN migrate -version
-

cmd/migrate/README.md

@@ -0,0 +1,142 @@
+# migrate CLI
+
+## Installation
+
+### Download pre-built binary (Windows, MacOS, or Linux)
+
+[Release Downloads](https://github.com/golang-migrate/migrate/releases)
+
+```bash
+$ curl -L https://github.com/golang-migrate/migrate/releases/download/$version/migrate.$os-$arch.tar.gz | tar xvz
+```
+
+### MacOS
+
+```bash
+$ brew install golang-migrate
+```
+
+### Windows
+
+Using [scoop](https://scoop.sh/)
+
+```bash
+$ scoop install migrate
+```
+
+### Linux (*.deb package)
+
+```bash
+$ curl -fsSL https://packagecloud.io/golang-migrate/migrate/gpgkey | sudo gpg --dearmor -o /etc/apt/keyrings/migrate.gpg
+$ echo "deb [signed-by=/etc/apt/keyrings/migrate.gpg] https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
+$ apt-get update
+$ apt-get install -y migrate
+```
+
+### With Go toolchain
+
+#### Versioned
+
+```bash
+$ go get -u -d github.com/golang-migrate/migrate/cmd/migrate
+$ cd $GOPATH/src/github.com/golang-migrate/migrate/cmd/migrate
+$ git checkout $TAG  # e.g. v4.1.0
+$ # Go 1.15 and below
+$ go build -tags 'postgres' -ldflags="-X main.Version=$(git describe --tags)" -o $GOPATH/bin/migrate $GOPATH/src/github.com/golang-migrate/migrate/cmd/migrate
+$ # Go 1.16+
+$ go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@$TAG
+```
+
+#### Unversioned
+
+```bash
+$ # Go 1.15 and below
+$ go get -tags 'postgres' -u github.com/golang-migrate/migrate/cmd/migrate
+$ # Go 1.16+
+$ go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest
+```
+
+#### Notes
+
+1. Requires a version of Go that [supports modules](https://golang.org/cmd/go/#hdr-Preliminary_module_support). e.g. Go 1.11+
+1. These examples build the cli which will only work with postgres.  In order
+to build the cli for use with other databases, replace the `postgres` build tag
+with the appropriate database tag(s) for the databases desired.  The tags
+correspond to the names of the sub-packages underneath the
+[`database`](../../database) package.
+1. Similarly to the database build tags, if you need to support other sources, use the appropriate build tag(s).
+1. Support for build constraints will be removed in the future: https://github.com/golang-migrate/migrate/issues/60
+1. For versions of Go 1.15 and lower, [make sure](https://github.com/golang-migrate/migrate/pull/257#issuecomment-705249902) you're not installing the `migrate` CLI from a module. e.g. there should not be any `go.mod` files in your current directory or any directory from your current directory to the root
+
+## Usage
+
+```bash
+$ migrate -help
+Usage: migrate OPTIONS COMMAND [arg...]
+       migrate [ -version | -help ]
+
+Options:
+  -source          Location of the migrations (driver://url)
+  -path            Shorthand for -source=file://path
+  -database        Run migrations against this database (driver://url)
+  -prefetch N      Number of migrations to load in advance before executing (default 10)
+  -lock-timeout N  Allow N seconds to acquire database lock (default 15)
+  -verbose         Print verbose logging
+  -version         Print version
+  -help            Print usage
+
+Commands:
+  create [-ext E] [-dir D] [-seq] [-digits N] [-format] [-tz] NAME
+           Create a set of timestamped up/down migrations titled NAME, in directory D with extension E.
+           Use -seq option to generate sequential up/down migrations with N digits.
+           Use -format option to specify a Go time format string. Note: migrations with the same time cause "duplicate migration version" error.
+           Use -tz option to specify the timezone that will be used when generating non-sequential migrations (defaults: UTC).
+
+  goto V       Migrate to version V
+  up [N]       Apply all or N up migrations
+  down [N] [-all]    Apply all or N down migrations
+        Use -all to apply all down migrations
+  drop [-f]    Drop everything inside database
+        Use -f to bypass confirmation
+  force V      Set version V but don't run migration (ignores dirty state)
+  version      Print current migration version
+```
+
+So let's say you want to run the first two migrations
+
+```bash
+$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2
+```
+
+If your migrations are hosted on github
+
+```bash
+$ migrate -source github://mattes:personal-access-token@mattes/migrate_test \
+    -database postgres://localhost:5432/database down 2
+```
+
+The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received.
+Send SIGKILL for immediate halt.
+
+## Reading CLI arguments from somewhere else
+
+### ENV variables
+
+```bash
+$ migrate -database "$MY_MIGRATE_DATABASE"
+```
+
+### JSON files
+
+Check out https://stedolan.github.io/jq/
+
+```bash
+$ migrate -database "$(cat config.json | jq -r '.database')"
+```
+
+### YAML files
+
+```bash
+$ migrate -database "$(cat config/database.yml | ruby -ryaml -e "print YAML.load(STDIN.read)['database']")"
+$ migrate -database "$(cat config/database.yml | python -c 'import yaml,sys;print yaml.safe_load(sys.stdin)["database"]')"
+```

cmd/migrate/examples/Dockerfile

@@ -0,0 +1,15 @@
+FROM ubuntu:bionic
+
+RUN apt-get update && \
+    apt-get install -y \
+    apt-transport-https \
+    ca-certificates \
+    curl \
+    gnupg-agent
+
+RUN curl -sSL https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add -
+RUN echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ bionic main" > /etc/apt/sources.list.d/migrate.list
+RUN apt-get update && \
+    apt-get install -y migrate
+
+RUN migrate -version

database/cassandra/README.md

@@ -1,12 +1,17 @@
-# Cassandra
+# Cassandra / ScyllaDB
 
-* Drop command will not work on Cassandra 2.X because it rely on
+* `Drop()` method will not work on Cassandra 2.X because it rely on
 system_schema table which comes with 3.X
-* Other commands should work properly but are **not tested**
-* The Cassandra driver (gocql) does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
+* Other methods should work properly but are **not tested**
+* The Cassandra driver (gocql) does not natively support executing multiple statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
   * This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
   * The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
 
+**ScyllaDB**
+
+* No additional configuration is required since it is a drop-in replacement for Cassandra.
+* The `Drop()` method` works for ScyllaDB 5.1
+
 
 ## Usage
 `cassandra://host:port/keyspace?param1=value&param2=value2`
@@ -20,12 +25,14 @@ system_schema table which comes with 3.X
 | `consistency` | ALL | Migration consistency
 | `protocol` |  | Cassandra protocol version (3 or 4)
 | `timeout` | 1 minute | Migration timeout
+| `connect-timeout` | 600ms | Initial connection timeout to the cluster |
 | `username` | nil | Username to use when authenticating. |
 | `password` | nil | Password to use when authenticating. |
 | `sslcert` | | Cert file location. The file must contain PEM encoded data. |
 | `sslkey` | | Key file location. The file must contain PEM encoded data. |
 | `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
 | `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
+| `disable-host-lookup`| false | Disable initial host lookup. |
 
 `timeout` is parsed using [time.ParseDuration(s string)](https://golang.org/pkg/time/#ParseDuration)
 

database/cassandra/cassandra.go

@@ -4,14 +4,15 @@ import (
 	"errors"
 	"fmt"
 	"io"
-	"io/ioutil"
 	nurl "net/url"
 	"strconv"
 	"strings"
+	"sync/atomic"
 	"time"
 
 	"github.com/gocql/gocql"
 	"github.com/golang-migrate/migrate/v4/database"
+	"github.com/golang-migrate/migrate/v4/database/multistmt"
 )
 
 func init() {
@@ -19,6 +20,12 @@ func init() {
 	database.Register("cassandra", db)
 }
 
+var (
+	multiStmtDelimiter = []byte(";")
+
+	DefaultMultiStatementMaxSize = 10 * 1 << 20 // 10 MB
+)
+
 var DefaultMigrationsTable = "schema_migrations"
 
 var (
@@ -32,11 +39,12 @@ type Config struct {
 	MigrationsTable       string
 	KeyspaceName          string
 	MultiStatementEnabled bool
+	MultiStatementMaxSize int
 }
 
 type Cassandra struct {
 	session  *gocql.Session
-	isLocked bool
+	isLocked atomic.Bool
 
 	// Open and WithInstance need to guarantee that config is never nil
 	config *Config
@@ -57,6 +65,10 @@ func WithInstance(session *gocql.Session, config *Config) (database.Driver, erro
 		config.MigrationsTable = DefaultMigrationsTable
 	}
 
+	if config.MultiStatementMaxSize <= 0 {
+		config.MultiStatementMaxSize = DefaultMultiStatementMaxSize
+	}
+
 	c := &Cassandra{
 		session: session,
 		config:  config,
@@ -119,17 +131,42 @@ func (c *Cassandra) Open(url string) (database.Driver, error) {
 		}
 		cluster.Timeout = timeout
 	}
+	if len(u.Query().Get("connect-timeout")) > 0 {
+		var connectTimeout time.Duration
+		connectTimeout, err = time.ParseDuration(u.Query().Get("connect-timeout"))
+		if err != nil {
+			return nil, err
+		}
+		cluster.ConnectTimeout = connectTimeout
+	}
 
-	if len(u.Query().Get("sslmode")) > 0 && len(u.Query().Get("sslrootcert")) > 0 && len(u.Query().Get("sslcert")) > 0 && len(u.Query().Get("sslkey")) > 0 {
+	if len(u.Query().Get("sslmode")) > 0 {
 		if u.Query().Get("sslmode") != "disable" {
-			cluster.SslOpts = &gocql.SslOptions{
-				CaPath:   u.Query().Get("sslrootcert"),
-				CertPath: u.Query().Get("sslcert"),
-				KeyPath:  u.Query().Get("sslkey"),
+			sslOpts := &gocql.SslOptions{}
+
+			if len(u.Query().Get("sslrootcert")) > 0 {
+				sslOpts.CaPath = u.Query().Get("sslrootcert")
 			}
+			if len(u.Query().Get("sslcert")) > 0 {
+				sslOpts.CertPath = u.Query().Get("sslcert")
+			}
+			if len(u.Query().Get("sslkey")) > 0 {
+				sslOpts.KeyPath = u.Query().Get("sslkey")
+			}
+
 			if u.Query().Get("sslmode") == "verify-full" {
-				cluster.SslOpts.EnableHostVerification = true
+				sslOpts.EnableHostVerification = true
 			}
+
+			cluster.SslOpts = sslOpts
+		}
+	}
+
+	if len(u.Query().Get("disable-host-lookup")) > 0 {
+		if flag, err := strconv.ParseBool(u.Query().Get("disable-host-lookup")); err != nil && flag {
+			cluster.DisableInitialHostLookup = true
+		} else if err != nil {
+			return nil, err
 		}
 	}
 
@@ -138,10 +175,19 @@ func (c *Cassandra) Open(url string) (database.Driver, error) {
 		return nil, err
 	}
 
+	multiStatementMaxSize := DefaultMultiStatementMaxSize
+	if s := u.Query().Get("x-multi-statement-max-size"); len(s) > 0 {
+		multiStatementMaxSize, err = strconv.Atoi(s)
+		if err != nil {
+			return nil, err
+		}
+	}
+
 	return WithInstance(session, &Config{
 		KeyspaceName:          strings.TrimPrefix(u.Path, "/"),
 		MigrationsTable:       u.Query().Get("x-migrations-table"),
 		MultiStatementEnabled: u.Query().Get("x-multi-statement") == "true",
+		MultiStatementMaxSize: multiStatementMaxSize,
 	})
 }
 
@@ -151,44 +197,44 @@ func (c *Cassandra) Close() error {
 }
 
 func (c *Cassandra) Lock() error {
-	if c.isLocked {
+	if !c.isLocked.CompareAndSwap(false, true) {
 		return database.ErrLocked
 	}
-	c.isLocked = true
 	return nil
 }
 
 func (c *Cassandra) Unlock() error {
-	c.isLocked = false
+	if !c.isLocked.CompareAndSwap(true, false) {
+		return database.ErrNotLocked
+	}
 	return nil
 }
 
 func (c *Cassandra) Run(migration io.Reader) error {
-	migr, err := ioutil.ReadAll(migration)
+	if c.config.MultiStatementEnabled {
+		var err error
+		if e := multistmt.Parse(migration, multiStmtDelimiter, c.config.MultiStatementMaxSize, func(m []byte) bool {
+			tq := strings.TrimSpace(string(m))
+			if tq == "" {
+				return true
+			}
+			if e := c.session.Query(tq).Exec(); e != nil {
+				err = database.Error{OrigErr: e, Err: "migration failed", Query: m}
+				return false
+			}
+			return true
+		}); e != nil {
+			return e
+		}
+		return err
+	}
+
+	migr, err := io.ReadAll(migration)
 	if err != nil {
 		return err
 	}
 	// run migration
-	query := string(migr[:])
-
-	if c.config.MultiStatementEnabled {
-		// split query by semi-colon
-		queries := strings.Split(query, ";")
-
-		for _, q := range queries {
-			tq := strings.TrimSpace(q)
-			if tq == "" {
-				continue
-			}
-			if err := c.session.Query(tq).Exec(); err != nil {
-				// TODO: cast to Cassandra error and get line number
-				return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
-			}
-		}
-		return nil
-	}
-
-	if err := c.session.Query(query).Exec(); err != nil {
+	if err := c.session.Query(string(migr)).Exec(); err != nil {
 		// TODO: cast to Cassandra error and get line number
 		return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
 	}
@@ -196,12 +242,26 @@ func (c *Cassandra) Run(migration io.Reader) error {
 }
 
 func (c *Cassandra) SetVersion(version int, dirty bool) error {
-	query := `TRUNCATE "` + c.config.MigrationsTable + `"`
-	if err := c.session.Query(query).Exec(); err != nil {
-		return &database.Error{OrigErr: err, Query: []byte(query)}
+	// DELETE instead of TRUNCATE because AWS Keyspaces does not support it
+	// see: https://docs.aws.amazon.com/keyspaces/latest/devguide/cassandra-apis.html
+	squery := `SELECT version FROM "` + c.config.MigrationsTable + `"`
+	dquery := `DELETE FROM "` + c.config.MigrationsTable + `" WHERE version = ?`
+	iter := c.session.Query(squery).Iter()
+	var previous int
+	for iter.Scan(&previous) {
+		if err := c.session.Query(dquery, previous).Exec(); err != nil {
+			return &database.Error{OrigErr: err, Query: []byte(dquery)}
+		}
 	}
-	if version >= 0 {
-		query = `INSERT INTO "` + c.config.MigrationsTable + `" (version, dirty) VALUES (?, ?)`
+	if err := iter.Close(); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(squery)}
+	}
+
+	// Also re-write the schema version for nil dirty versions to prevent
+	// empty schema version for failed down migration on the first migration
+	// See: https://github.com/golang-migrate/migrate/issues/330
+	if version >= 0 || (version == database.NilVersion && dirty) {
+		query := `INSERT INTO "` + c.config.MigrationsTable + `" (version, dirty) VALUES (?, ?)`
 		if err := c.session.Query(query, version, dirty).Exec(); err != nil {
 			return &database.Error{OrigErr: err, Query: []byte(query)}
 		}
@@ -240,13 +300,25 @@ func (c *Cassandra) Drop() error {
 			return err
 		}
 	}
-	// Re-create the version table
-	return c.ensureVersionTable()
+
+	return nil
 }
 
-// Ensure version table exists
-func (c *Cassandra) ensureVersionTable() error {
-	err := c.session.Query(fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (version bigint, dirty boolean, PRIMARY KEY(version))", c.config.MigrationsTable)).Exec()
+// ensureVersionTable checks if versions table exists and, if not, creates it.
+// Note that this function locks the database, which deviates from the usual
+// convention of "caller locks" in the Cassandra type.
+func (c *Cassandra) ensureVersionTable() (err error) {
+	if err = c.Lock(); err != nil {
+		return err
+	}
+
+	defer func() {
+		if e := c.Unlock(); e != nil {
+			err = errors.Join(err, e)
+		}
+	}()
+
+	err = c.session.Query(fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (version bigint, dirty boolean, PRIMARY KEY(version))", c.config.MigrationsTable)).Exec()
 	if err != nil {
 		return err
 	}
@@ -264,7 +336,7 @@ func parseConsistency(consistencyStr string) (consistency gocql.Consistency, err
 			var ok bool
 			err, ok = r.(error)
 			if !ok {
-				err = fmt.Errorf("Failed to parse consistency \"%s\": %v", consistencyStr, r)
+				err = fmt.Errorf("failed to parse consistency \"%s\": %v", consistencyStr, r)
 			}
 		}
 	}()

database/cassandra/cassandra_test.go

@@ -3,6 +3,7 @@ package cassandra
 import (
 	"context"
 	"fmt"
+	"github.com/golang-migrate/migrate/v4"
 	"strconv"
 	"testing"
 )
@@ -15,6 +16,7 @@ import (
 import (
 	dt "github.com/golang-migrate/migrate/v4/database/testing"
 	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
 )
 
 var (
@@ -23,9 +25,11 @@ var (
 	// Although Cassandra 2.x is supported by the Apache Foundation,
 	// the migrate db driver only supports Cassandra 3.x since it uses
 	// the system_schema keyspace.
+	// last ScyllaDB version tested is 5.1.11
 	specs = []dktesting.ContainerSpec{
 		{ImageName: "cassandra:3.0", Options: opts},
 		{ImageName: "cassandra:3.11", Options: opts},
+		{ImageName: "scylladb/scylla:5.1.11", Options: opts},
 	}
 )
 
@@ -57,6 +61,20 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
 }
 
 func Test(t *testing.T) {
+	t.Run("test", test)
+	t.Run("testMigrate", testMigrate)
+
+	t.Cleanup(func() {
+		for _, spec := range specs {
+			t.Log("Cleaning up ", spec.ImageName)
+			if err := spec.Cleanup(); err != nil {
+				t.Error("Error removing ", spec.ImageName, "error:", err)
+			}
+		}
+	})
+}
+
+func test(t *testing.T) {
 	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
 		ip, port, err := c.Port(9042)
 		if err != nil {
@@ -66,9 +84,39 @@ func Test(t *testing.T) {
 		p := &Cassandra{}
 		d, err := p.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
-		defer d.Close()
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
 		dt.Test(t, d, []byte("SELECT table_name from system_schema.tables"))
 	})
 }
+
+func testMigrate(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(9042)
+		if err != nil {
+			t.Fatal("Unable to get mapped port:", err)
+		}
+		addr := fmt.Sprintf("cassandra://%v:%v/testks", ip, port)
+		p := &Cassandra{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "testks", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+	})
+}

database/cassandra/examples/migrations/1_simple_select.down.sql

@@ -0,0 +1 @@
+SELECT table_name from system_schema.tables

database/cassandra/examples/migrations/1_simple_select.up.sql

@@ -0,0 +1 @@
+SELECT table_name from system_schema.tables

database/clickhouse/README.md

@@ -1,19 +0,0 @@
-# ClickHouse
-
-`clickhouse://host:port?username=user&password=qwerty&database=clicks&x-multi-statement=true`
-
-| URL Query  | Description |
-|------------|-------------|
-| `x-migrations-table`| Name of the migrations table |
-| `database` | The name of the database to connect to |
-| `username` | The user to sign in as |
-| `password` | The user's password | 
-| `host` | The host to connect to. |
-| `port` | The port to bind to. |
-| `x-multi-statement` | false | Enable multiple statements to be ran in a single migration (See note below) |
-
-## Notes
-
-* The Clickhouse driver does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
-  * This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
-  * The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.

database/clickhouse/clickhouse.go

@@ -1,215 +0,0 @@
-package clickhouse
-
-import (
-	"database/sql"
-	"fmt"
-	"io"
-	"io/ioutil"
-	"net/url"
-	"strings"
-	"time"
-
-	"github.com/golang-migrate/migrate/v4"
-	"github.com/golang-migrate/migrate/v4/database"
-)
-
-var DefaultMigrationsTable = "schema_migrations"
-
-var ErrNilConfig = fmt.Errorf("no config")
-
-type Config struct {
-	DatabaseName          string
-	MigrationsTable       string
-	MultiStatementEnabled bool
-}
-
-func init() {
-	database.Register("clickhouse", &ClickHouse{})
-}
-
-func WithInstance(conn *sql.DB, config *Config) (database.Driver, error) {
-	if config == nil {
-		return nil, ErrNilConfig
-	}
-
-	if err := conn.Ping(); err != nil {
-		return nil, err
-	}
-
-	ch := &ClickHouse{
-		conn:   conn,
-		config: config,
-	}
-
-	if err := ch.init(); err != nil {
-		return nil, err
-	}
-
-	return ch, nil
-}
-
-type ClickHouse struct {
-	conn   *sql.DB
-	config *Config
-}
-
-func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
-	purl, err := url.Parse(dsn)
-	if err != nil {
-		return nil, err
-	}
-	q := migrate.FilterCustomQuery(purl)
-	q.Scheme = "tcp"
-	conn, err := sql.Open("clickhouse", q.String())
-	if err != nil {
-		return nil, err
-	}
-
-	ch = &ClickHouse{
-		conn: conn,
-		config: &Config{
-			MigrationsTable:       purl.Query().Get("x-migrations-table"),
-			DatabaseName:          purl.Query().Get("database"),
-			MultiStatementEnabled: purl.Query().Get("x-multi-statement") == "true",
-		},
-	}
-
-	if err := ch.init(); err != nil {
-		return nil, err
-	}
-
-	return ch, nil
-}
-
-func (ch *ClickHouse) init() error {
-	if len(ch.config.DatabaseName) == 0 {
-		if err := ch.conn.QueryRow("SELECT currentDatabase()").Scan(&ch.config.DatabaseName); err != nil {
-			return err
-		}
-	}
-
-	if len(ch.config.MigrationsTable) == 0 {
-		ch.config.MigrationsTable = DefaultMigrationsTable
-	}
-
-	return ch.ensureVersionTable()
-}
-
-func (ch *ClickHouse) Run(r io.Reader) error {
-	migration, err := ioutil.ReadAll(r)
-	if err != nil {
-		return err
-	}
-
-	if ch.config.MultiStatementEnabled {
-		// split query by semi-colon
-		queries := strings.Split(string(migration), ";")
-		for _, q := range queries {
-			tq := strings.TrimSpace(q)
-			if tq == "" {
-				continue
-			}
-			if _, err := ch.conn.Exec(string(q)); err != nil {
-				return database.Error{OrigErr: err, Err: "migration failed", Query: []byte(q)}
-			}
-		}
-		return nil
-	}
-
-	if _, err := ch.conn.Exec(string(migration)); err != nil {
-		return database.Error{OrigErr: err, Err: "migration failed", Query: migration}
-	}
-
-	return nil
-}
-func (ch *ClickHouse) Version() (int, bool, error) {
-	var (
-		version int
-		dirty   uint8
-		query   = "SELECT version, dirty FROM `" + ch.config.MigrationsTable + "` ORDER BY sequence DESC LIMIT 1"
-	)
-	if err := ch.conn.QueryRow(query).Scan(&version, &dirty); err != nil {
-		if err == sql.ErrNoRows {
-			return database.NilVersion, false, nil
-		}
-		return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
-	}
-	return version, dirty == 1, nil
-}
-
-func (ch *ClickHouse) SetVersion(version int, dirty bool) error {
-	var (
-		bool = func(v bool) uint8 {
-			if v {
-				return 1
-			}
-			return 0
-		}
-		tx, err = ch.conn.Begin()
-	)
-	if err != nil {
-		return err
-	}
-
-	query := "INSERT INTO " + ch.config.MigrationsTable + " (version, dirty, sequence) VALUES (?, ?, ?)"
-	if _, err := tx.Exec(query, version, bool(dirty), time.Now().UnixNano()); err != nil {
-		return &database.Error{OrigErr: err, Query: []byte(query)}
-	}
-
-	return tx.Commit()
-}
-
-func (ch *ClickHouse) ensureVersionTable() error {
-	var (
-		table string
-		query = "SHOW TABLES FROM " + ch.config.DatabaseName + " LIKE '" + ch.config.MigrationsTable + "'"
-	)
-	// check if migration table exists
-	if err := ch.conn.QueryRow(query).Scan(&table); err != nil {
-		if err != sql.ErrNoRows {
-			return &database.Error{OrigErr: err, Query: []byte(query)}
-		}
-	} else {
-		return nil
-	}
-	// if not, create the empty migration table
-	query = `
-		CREATE TABLE ` + ch.config.MigrationsTable + ` (
-			version    Int64, 
-			dirty      UInt8,
-			sequence   UInt64
-		) Engine=TinyLog
-	`
-	if _, err := ch.conn.Exec(query); err != nil {
-		return &database.Error{OrigErr: err, Query: []byte(query)}
-	}
-	return nil
-}
-
-func (ch *ClickHouse) Drop() error {
-	var (
-		query       = "SHOW TABLES FROM " + ch.config.DatabaseName
-		tables, err = ch.conn.Query(query)
-	)
-	if err != nil {
-		return &database.Error{OrigErr: err, Query: []byte(query)}
-	}
-	defer tables.Close()
-	for tables.Next() {
-		var table string
-		if err := tables.Scan(&table); err != nil {
-			return err
-		}
-
-		query = "DROP TABLE IF EXISTS " + ch.config.DatabaseName + "." + table
-
-		if _, err := ch.conn.Exec(query); err != nil {
-			return &database.Error{OrigErr: err, Query: []byte(query)}
-		}
-	}
-	return ch.ensureVersionTable()
-}
-
-func (ch *ClickHouse) Lock() error   { return nil }
-func (ch *ClickHouse) Unlock() error { return nil }
-func (ch *ClickHouse) Close() error  { return ch.conn.Close() }

database/cockroachdb/TUTORIAL.md

@@ -0,0 +1,142 @@
+# CockroachDB tutorial for beginners (insecure cluster)
+
+## Create/configure database
+
+First, let's start a local cluster - follow step 1. and 2. from [the docs](https://www.cockroachlabs.com/docs/stable/start-a-local-cluster.html#step-1-start-the-first-node).
+
+Once you have it, create a database. Here I am going to create a database called `example`.
+Our user here is `cockroach`. We are not going to use a password, since it's not supported for insecure cluster.
+```
+cockroach sql --insecure --host=localhost:26257
+```
+```
+CREATE DATABASE example;
+CREATE USER IF NOT EXISTS cockroach;
+GRANT ALL ON DATABASE example TO cockroach;
+```
+
+When using Migrate CLI we need to pass to database URL. Let's export it to a variable for convenience:
+```
+export COCKROACHDB_URL='cockroachdb://cockroach:@localhost:26257/example?sslmode=disable'
+```
+`sslmode=disable` means that the connection with our database will not be encrypted. This is needed to connect to an insecure node.
+
+**NOTE:** Do not use COCKROACH_URL as a variable name here, it's already in use for discrete parameters and you may run into connection problems. For more info check out [docs](https://www.cockroachlabs.com/docs/stable/connection-parameters.html#connect-using-discrete-parameters).
+
+You can find further description of database URLs [here](README.md#database-urls).
+
+## Create migrations
+Let's create a table called `users`:
+```
+migrate create -ext sql -dir db/migrations -seq create_users_table
+```
+If there were no errors, we should have two files available under `db/migrations` folder:
+- 000001_create_users_table.down.sql
+- 000001_create_users_table.up.sql
+
+Note the `sql` extension that we provided.
+
+In the `.up.sql` file let's create the table:
+```
+CREATE TABLE IF NOT EXISTS example.users
+(
+   user_id INT PRIMARY KEY,
+   username VARCHAR (50) UNIQUE NOT NULL,
+   password VARCHAR (50) NOT NULL,
+   email VARCHAR (300) UNIQUE NOT NULL
+);
+```
+And in the `.down.sql` let's delete it:
+```
+DROP TABLE IF EXISTS example.users;
+```
+By adding `IF EXISTS/IF NOT EXISTS` we are making migrations idempotent - you can read more about idempotency in [getting started](/GETTING_STARTED.md#create-migrations)
+
+## Run migrations
+```
+migrate -database ${COCKROACHDB_URL} -path db/migrations up
+```
+Let's check if the table was created properly by running `cockroach sql --insecure --host=localhost:26257 -e "show columns from example.users;"`.
+The output you are supposed to see:
+```
+  column_name |  data_type   | is_nullable | column_default | generation_expression |                   indices                    | is_hidden
++-------------+--------------+-------------+----------------+-----------------------+----------------------------------------------+-----------+
+  user_id     | INT8         |    false    | NULL           |                       | {primary,users_username_key,users_email_key} |   false
+  username    | VARCHAR(50)  |    false    | NULL           |                       | {users_username_key}                         |   false
+  password    | VARCHAR(50)  |    false    | NULL           |                       | {}                                           |   false
+  email       | VARCHAR(300) |    false    | NULL           |                       | {users_email_key}                            |   false
+(4 rows)
+```
+Now let's check if running reverse migration also works:
+```
+migrate -database ${COCKROACHDB_URL} -path db/migrations down
+```
+Make sure to check if your database changed as expected in this case as well.
+
+## Database transactions
+
+To show database transactions usage, let's create another set of migrations by running:
+```
+migrate create -ext sql -dir db/migrations -seq add_mood_to_users
+```
+Again, it should create for us two migrations files:
+- 000002_add_mood_to_users.down.sql
+- 000002_add_mood_to_users.up.sql
+
+In Cockroach, when we want our queries to be done in a transaction, we need to wrap it with `BEGIN` and `COMMIT` commands, similar to PostgreSQL.
+In our example, we are going to add a column to our database that can only accept enumerable values or NULL.
+Migration up:
+```
+BEGIN;
+
+ALTER TABLE example.users ADD COLUMN mood STRING;
+ALTER TABLE example.users ADD CONSTRAINT check_mood CHECK (mood IN ('happy', 'sad', 'neutral'));
+
+COMMIT;
+```
+Migration down:
+```
+ALTER TABLE example.users DROP COLUMN mood;
+```
+
+Now we can run our new migration and check the database:
+```
+migrate -database ${COCKROACHDB_URL} -path db/migrations up
+cockroach sql --insecure --host=localhost:26257 -e "show columns from example.users;"
+```
+Expected output:
+```
+  column_name |  data_type   | is_nullable | column_default | generation_expression |                   indices                    | is_hidden  
++-------------+--------------+-------------+----------------+-----------------------+----------------------------------------------+-----------+
+  user_id     | INT8         |    false    | NULL           |                       | {primary,users_username_key,users_email_key} |   false    
+  username    | VARCHAR(50)  |    false    | NULL           |                       | {users_username_key}                         |   false    
+  password    | VARCHAR(50)  |    false    | NULL           |                       | {}                                           |   false    
+  email       | VARCHAR(300) |    false    | NULL           |                       | {users_email_key}                            |   false    
+  mood        | STRING       |    true     | NULL           |                       | {}                                           |   false    
+(5 rows)
+```
+
+## Optional: Run migrations within your Go app
+Here is a very simple app running migrations for the above configuration:
+```
+import (
+	"log"
+
+	"github.com/golang-migrate/migrate/v4"
+	_ "github.com/golang-migrate/migrate/v4/database/cockroachdb"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+)
+
+func main() {
+	m, err := migrate.New(
+		"file://db/migrations",
+		"cockroachdb://cockroach:@localhost:26257/example?sslmode=disable")
+	if err != nil {
+		log.Fatal(err)
+	}
+	if err := m.Up(); err != nil {
+		log.Fatal(err)
+	}
+}
+```
+You can find details [here](README.md#use-in-your-go-project)

database/cockroachdb/cockroachdb.go

@@ -3,22 +3,18 @@ package cockroachdb
 import (
 	"context"
 	"database/sql"
+	"errors"
 	"fmt"
 	"io"
-	"io/ioutil"
 	nurl "net/url"
 	"regexp"
 	"strconv"
-)
+	"sync/atomic"
 
-import (
-	"github.com/cockroachdb/cockroach-go/crdb"
-	"github.com/lib/pq"
-)
-
-import (
+	"github.com/cockroachdb/cockroach-go/v2/crdb"
 	"github.com/golang-migrate/migrate/v4"
 	"github.com/golang-migrate/migrate/v4/database"
+	"github.com/lib/pq"
 )
 
 func init() {
@@ -45,7 +41,7 @@ type Config struct {
 
 type CockroachDb struct {
 	db       *sql.DB
-	isLocked bool
+	isLocked atomic.Bool
 
 	// Open and WithInstance need to guarantee that config is never nil
 	config *Config
@@ -60,17 +56,19 @@ func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
 		return nil, err
 	}
 
-	query := `SELECT current_database()`
-	var databaseName string
-	if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
-		return nil, &database.Error{OrigErr: err, Query: []byte(query)}
-	}
+	if config.DatabaseName == "" {
+		query := `SELECT current_database()`
+		var databaseName string
+		if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
+			return nil, &database.Error{OrigErr: err, Query: []byte(query)}
+		}
 
-	if len(databaseName) == 0 {
-		return nil, ErrNoDatabaseName
-	}
+		if len(databaseName) == 0 {
+			return nil, ErrNoDatabaseName
+		}
 
-	config.DatabaseName = databaseName
+		config.DatabaseName = databaseName
+	}
 
 	if len(config.MigrationsTable) == 0 {
 		config.MigrationsTable = DefaultMigrationsTable
@@ -85,11 +83,12 @@ func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
 		config: config,
 	}
 
-	if err := px.ensureVersionTable(); err != nil {
+	// ensureVersionTable is a locking operation, so we need to ensureLockTable before we ensureVersionTable.
+	if err := px.ensureLockTable(); err != nil {
 		return nil, err
 	}
 
-	if err := px.ensureLockTable(); err != nil {
+	if err := px.ensureVersionTable(); err != nil {
 		return nil, err
 	}
 
@@ -148,71 +147,71 @@ func (c *CockroachDb) Close() error {
 // Locking is done manually with a separate lock table.  Implementing advisory locks in CRDB is being discussed
 // See: https://github.com/cockroachdb/cockroach/issues/13546
 func (c *CockroachDb) Lock() error {
-	err := crdb.ExecuteTx(context.Background(), c.db, nil, func(tx *sql.Tx) error {
-		aid, err := database.GenerateAdvisoryLockId(c.config.DatabaseName)
-		if err != nil {
-			return err
-		}
+	return database.CasRestoreOnErr(&c.isLocked, false, true, database.ErrLocked, func() (err error) {
+		return crdb.ExecuteTx(context.Background(), c.db, nil, func(tx *sql.Tx) (err error) {
+			aid, err := database.GenerateAdvisoryLockId(c.config.DatabaseName)
+			if err != nil {
+				return err
+			}
 
-		query := "SELECT * FROM " + c.config.LockTable + " WHERE lock_id = $1"
-		rows, err := tx.Query(query, aid)
-		if err != nil {
-			return database.Error{OrigErr: err, Err: "failed to fetch migration lock", Query: []byte(query)}
-		}
-		defer rows.Close()
+			query := "SELECT * FROM " + c.config.LockTable + " WHERE lock_id = $1"
+			rows, err := tx.Query(query, aid)
+			if err != nil {
+				return database.Error{OrigErr: err, Err: "failed to fetch migration lock", Query: []byte(query)}
+			}
+			defer func() {
+				if errClose := rows.Close(); errClose != nil {
+					err = errors.Join(err, errClose)
+				}
+			}()
 
-		// If row exists at all, lock is present
-		locked := rows.Next()
-		if locked && !c.config.ForceLock {
-			return database.ErrLocked
-		}
+			// If row exists at all, lock is present
+			locked := rows.Next()
+			if locked && !c.config.ForceLock {
+				return database.ErrLocked
+			}
 
-		query = "INSERT INTO " + c.config.LockTable + " (lock_id) VALUES ($1)"
-		if _, err := tx.Exec(query, aid); err != nil {
-			return database.Error{OrigErr: err, Err: "failed to set migration lock", Query: []byte(query)}
-		}
+			query = "INSERT INTO " + c.config.LockTable + " (lock_id) VALUES ($1)"
+			if _, err := tx.Exec(query, aid); err != nil {
+				return database.Error{OrigErr: err, Err: "failed to set migration lock", Query: []byte(query)}
+			}
 
-		return nil
+			return nil
+		})
 	})
-
-	if err != nil {
-		return err
-	} else {
-		c.isLocked = true
-		return nil
-	}
 }
 
 // Locking is done manually with a separate lock table.  Implementing advisory locks in CRDB is being discussed
 // See: https://github.com/cockroachdb/cockroach/issues/13546
 func (c *CockroachDb) Unlock() error {
-	aid, err := database.GenerateAdvisoryLockId(c.config.DatabaseName)
-	if err != nil {
-		return err
-	}
-
-	// In the event of an implementation (non-migration) error, it is possible for the lock to not be released.  Until
-	// a better locking mechanism is added, a manual purging of the lock table may be required in such circumstances
-	query := "DELETE FROM " + c.config.LockTable + " WHERE lock_id = $1"
-	if _, err := c.db.Exec(query, aid); err != nil {
-		if e, ok := err.(*pq.Error); ok {
-			// 42P01 is "UndefinedTableError" in CockroachDB
-			// https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/pgwire/pgerror/codes.go
-			if e.Code == "42P01" {
-				// On drops, the lock table is fully removed;  This is fine, and is a valid "unlocked" state for the schema
-				c.isLocked = false
-				return nil
-			}
+	return database.CasRestoreOnErr(&c.isLocked, true, false, database.ErrNotLocked, func() (err error) {
+		aid, err := database.GenerateAdvisoryLockId(c.config.DatabaseName)
+		if err != nil {
+			return err
 		}
-		return database.Error{OrigErr: err, Err: "failed to release migration lock", Query: []byte(query)}
-	}
 
-	c.isLocked = false
-	return nil
+		// In the event of an implementation (non-migration) error, it is possible for the lock to not be released.  Until
+		// a better locking mechanism is added, a manual purging of the lock table may be required in such circumstances
+		query := "DELETE FROM " + c.config.LockTable + " WHERE lock_id = $1"
+		if _, err := c.db.Exec(query, aid); err != nil {
+			if e, ok := err.(*pq.Error); ok {
+				// 42P01 is "UndefinedTableError" in CockroachDB
+				// https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/pgwire/pgerror/codes.go
+				if e.Code == "42P01" {
+					// On drops, the lock table is fully removed;  This is fine, and is a valid "unlocked" state for the schema
+					return nil
+				}
+			}
+
+			return database.Error{OrigErr: err, Err: "failed to release migration lock", Query: []byte(query)}
+		}
+
+		return nil
+	})
 }
 
 func (c *CockroachDb) Run(migration io.Reader) error {
-	migr, err := ioutil.ReadAll(migration)
+	migr, err := io.ReadAll(migration)
 	if err != nil {
 		return err
 	}
@@ -232,7 +231,10 @@ func (c *CockroachDb) SetVersion(version int, dirty bool) error {
 			return err
 		}
 
-		if version >= 0 {
+		// Also re-write the schema version for nil dirty versions to prevent
+		// empty schema version for failed down migration on the first migration
+		// See: https://github.com/golang-migrate/migrate/issues/330
+		if version >= 0 || (version == database.NilVersion && dirty) {
 			if _, err := tx.Exec(`INSERT INTO "`+c.config.MigrationsTable+`" (version, dirty) VALUES ($1, $2)`, version, dirty); err != nil {
 				return err
 			}
@@ -265,14 +267,18 @@ func (c *CockroachDb) Version() (version int, dirty bool, err error) {
 	}
 }
 
-func (c *CockroachDb) Drop() error {
+func (c *CockroachDb) Drop() (err error) {
 	// select all tables in current schema
 	query := `SELECT table_name FROM information_schema.tables WHERE table_schema=(SELECT current_schema())`
 	tables, err := c.db.Query(query)
 	if err != nil {
 		return &database.Error{OrigErr: err, Query: []byte(query)}
 	}
-	defer tables.Close()
+	defer func() {
+		if errClose := tables.Close(); errClose != nil {
+			err = errors.Join(err, errClose)
+		}
+	}()
 
 	// delete one table after another
 	tableNames := make([]string, 0)
@@ -285,6 +291,9 @@ func (c *CockroachDb) Drop() error {
 			tableNames = append(tableNames, tableName)
 		}
 	}
+	if err := tables.Err(); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
 
 	if len(tableNames) > 0 {
 		// delete one by one ...
@@ -294,15 +303,25 @@ func (c *CockroachDb) Drop() error {
 				return &database.Error{OrigErr: err, Query: []byte(query)}
 			}
 		}
-		if err := c.ensureVersionTable(); err != nil {
-			return err
-		}
 	}
 
 	return nil
 }
 
-func (c *CockroachDb) ensureVersionTable() error {
+// ensureVersionTable checks if versions table exists and, if not, creates it.
+// Note that this function locks the database, which deviates from the usual
+// convention of "caller locks" in the CockroachDb type.
+func (c *CockroachDb) ensureVersionTable() (err error) {
+	if err = c.Lock(); err != nil {
+		return err
+	}
+
+	defer func() {
+		if e := c.Unlock(); e != nil {
+			err = errors.Join(err, e)
+		}
+	}()
+
 	// check if migration table exists
 	var count int
 	query := `SELECT COUNT(1) FROM information_schema.tables WHERE table_name = $1 AND table_schema = (SELECT current_schema()) LIMIT 1`

database/cockroachdb/cockroachdb_test.go

@@ -6,6 +6,8 @@ import (
 	"context"
 	"database/sql"
 	"fmt"
+	"github.com/golang-migrate/migrate/v4"
+	"log"
 	"strings"
 	"testing"
 )
@@ -18,38 +20,41 @@ import (
 import (
 	dt "github.com/golang-migrate/migrate/v4/database/testing"
 	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
 )
 
 const defaultPort = 26257
 
 var (
-	opts = dktest.Options{Cmd: []string{"start", "--insecure"}, PortRequired: true, ReadyFunc: isReady}
-	// Released versions: https://www.cockroachlabs.com/docs/releases/
+	opts = dktest.Options{Cmd: []string{"start-single-node", "--insecure"}, PortRequired: true, ReadyFunc: isReady}
+	// Supported versions: https://www.cockroachlabs.com/docs/releases/release-support-policy#supported-versions
 	specs = []dktesting.ContainerSpec{
-		{ImageName: "cockroachdb/cockroach:v1.0.7", Options: opts},
-		{ImageName: "cockroachdb/cockroach:v1.1.9", Options: opts},
-		{ImageName: "cockroachdb/cockroach:v2.0.7", Options: opts},
-		{ImageName: "cockroachdb/cockroach:v2.1.3", Options: opts},
+		{ImageName: "cockroachdb/cockroach:latest-v24.3", Options: opts},
+		{ImageName: "cockroachdb/cockroach:latest-v24.1", Options: opts},
+		{ImageName: "cockroachdb/cockroach:latest-v23.2", Options: opts},
+		{ImageName: "cockroachdb/cockroach:latest-v23.1", Options: opts},
 	}
 )
 
 func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
 	ip, port, err := c.Port(defaultPort)
 	if err != nil {
-		fmt.Println("port error:", err)
+		log.Println("port error:", err)
 		return false
 	}
 
 	db, err := sql.Open("postgres", fmt.Sprintf("postgres://root@%v:%v?sslmode=disable", ip, port))
 	if err != nil {
-		fmt.Println("open error:", err)
+		log.Println("open error:", err)
 		return false
 	}
 	if err := db.PingContext(ctx); err != nil {
-		fmt.Println("ping error:", err)
+		log.Println("ping error:", err)
 		return false
 	}
-	db.Close()
+	if err := db.Close(); err != nil {
+		log.Println("close error:", err)
+	}
 	return true
 }
 
@@ -66,7 +71,11 @@ func createDB(t *testing.T, c dktest.ContainerInfo) {
 	if err = db.Ping(); err != nil {
 		t.Fatal(err)
 	}
-	defer db.Close()
+	defer func() {
+		if err := db.Close(); err != nil {
+			t.Error(err)
+		}
+	}()
 
 	if _, err = db.Exec("CREATE DATABASE migrate"); err != nil {
 		t.Fatal(err)
@@ -86,12 +95,36 @@ func Test(t *testing.T) {
 		c := &CockroachDb{}
 		d, err := c.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		dt.Test(t, d, []byte("SELECT 1"))
 	})
 }
 
+func TestMigrate(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
+		createDB(t, ci)
+
+		ip, port, err := ci.Port(26257)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fmt.Sprintf("cockroach://root@%v:%v/migrate?sslmode=disable", ip, port)
+		c := &CockroachDb{}
+		d, err := c.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "migrate", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+	})
+}
+
 func TestMultiStatement(t *testing.T) {
 	dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
 		createDB(t, ci)
@@ -105,7 +138,7 @@ func TestMultiStatement(t *testing.T) {
 		c := &CockroachDb{}
 		d, err := c.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		if err := d.Run(strings.NewReader("CREATE TABLE foo (foo text); CREATE TABLE bar (bar text);")); err != nil {
 			t.Fatalf("expected err to be nil, got %v", err)
@@ -135,7 +168,7 @@ func TestFilterCustomQuery(t *testing.T) {
 		c := &CockroachDb{}
 		_, err = c.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 	})
 }

database/datastore/README.md

@@ -0,0 +1,31 @@
+# Datastore
+
+`datastore://host:port?username=user&password=password&database=clicks&x-multi-statement=true`
+
+Hanzo Datastore is a ClickHouse-wire-compatible database engine. This driver
+connects through the native `github.com/hanzo-ds/go` driver, which registers the
+`database/sql` driver name `datastore`.
+
+| URL Query  | Description |
+|------------|-------------|
+| `x-migrations-table`| Name of the migrations table |
+| `x-migrations-table-engine`| Engine to use for the migrations table, defaults to TinyLog |
+| `x-cluster-name` | Name of cluster for creating `schema_migrations` table cluster wide |
+| `x-multi-statement` | Enable multiple statements to be ran in a single migration (See note below) |
+| `x-multi-statement-max-size` | Maximum size of a single migration in bytes when `x-multi-statement` is set (defaults to 10 MB) |
+| `database` | The name of the database to connect to |
+| `username` | The user to sign in as |
+| `password` | The user's password |
+| `host` | The host to connect to. |
+| `port` | The port to bind to. |
+
+## Notes
+
+* The Datastore driver does not natively support executing multiple statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
+  * This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
+  * The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
+* Using the default TinyLog table engine for the `schema_migrations` table prevents backing up the table. If you need to back up the database, run the migrations with `x-migrations-table-engine=MergeTree`.
+* Datastore cluster mode is not officially supported, but you can try enabling `schema_migrations` table replication by specifying a `x-cluster-name`:
+  * When `x-cluster-name` is specified, `x-migrations-table-engine` also should be specified.
+  * When `x-cluster-name` is specified, only the `schema_migrations` table is replicated across the cluster. You still need to write your migrations so that the application tables are replicated within the cluster.
+* If you want to create a database inside a migration, note that the `schema_migrations` table will live in the `default` database, so you cannot use `USE <database_name>` inside a migration. In this case you may omit the database in the connection string (example [here](examples/migrations/003_create_database.up.sql)).

database/datastore/datastore.go

@@ -0,0 +1,337 @@
+// Package datastore implements the golang-migrate database.Driver interface for
+// Hanzo Datastore (a ClickHouse-wire-compatible database engine). It registers
+// the "datastore://" URL scheme and connects through the native
+// github.com/hanzo-ds/go driver, which registers the database/sql driver name
+// "datastore".
+//
+// Usage:
+//
+//	import _ "github.com/golang-migrate/migrate/v4/database/datastore"
+//
+// Then use a URL like:
+//
+//	datastore://host:9000?username=default&database=default&x-multi-statement=true
+package datastore
+
+import (
+	"database/sql"
+	"errors"
+	"fmt"
+	"io"
+	"net/url"
+	"strconv"
+	"strings"
+	"sync/atomic"
+	"time"
+
+	"github.com/golang-migrate/migrate/v4"
+	"github.com/golang-migrate/migrate/v4/database"
+	"github.com/golang-migrate/migrate/v4/database/multistmt"
+
+	// Registers the "datastore" database/sql driver (ClickHouse-wire native).
+	_ "github.com/hanzo-ds/go"
+)
+
+var (
+	multiStmtDelimiter = []byte(";")
+
+	// DriverName is the database/sql driver registered by github.com/hanzo-ds/go.
+	DriverName = "datastore"
+
+	DefaultMigrationsTable       = "schema_migrations"
+	DefaultMigrationsTableEngine = "TinyLog"
+	DefaultMultiStatementMaxSize = 10 * 1 << 20 // 10 MB
+
+	ErrNilConfig = fmt.Errorf("no config")
+)
+
+type Config struct {
+	DatabaseName          string
+	ClusterName           string
+	MigrationsTable       string
+	MigrationsTableEngine string
+	MultiStatementEnabled bool
+	MultiStatementMaxSize int
+}
+
+func init() {
+	database.Register("datastore", &Datastore{})
+}
+
+// WithInstance creates a Datastore driver from an existing *sql.DB opened
+// against the "datastore" driver.
+func WithInstance(conn *sql.DB, config *Config) (database.Driver, error) {
+	if config == nil {
+		return nil, ErrNilConfig
+	}
+
+	if err := conn.Ping(); err != nil {
+		return nil, err
+	}
+
+	ds := &Datastore{
+		conn:   conn,
+		config: config,
+	}
+
+	if err := ds.init(); err != nil {
+		return nil, err
+	}
+
+	return ds, nil
+}
+
+// Datastore is a golang-migrate database.Driver backed by the native
+// github.com/hanzo-ds/go ClickHouse-wire driver.
+type Datastore struct {
+	conn     *sql.DB
+	config   *Config
+	isLocked atomic.Bool
+}
+
+func (ds *Datastore) Open(dsn string) (database.Driver, error) {
+	purl, err := url.Parse(dsn)
+	if err != nil {
+		return nil, err
+	}
+	q := migrate.FilterCustomQuery(purl)
+	q.Scheme = DriverName
+	conn, err := sql.Open(DriverName, q.String())
+	if err != nil {
+		return nil, err
+	}
+
+	multiStatementMaxSize := DefaultMultiStatementMaxSize
+	if s := purl.Query().Get("x-multi-statement-max-size"); len(s) > 0 {
+		multiStatementMaxSize, err = strconv.Atoi(s)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	migrationsTableEngine := DefaultMigrationsTableEngine
+	if s := purl.Query().Get("x-migrations-table-engine"); len(s) > 0 {
+		migrationsTableEngine = s
+	}
+
+	ds = &Datastore{
+		conn: conn,
+		config: &Config{
+			MigrationsTable:       purl.Query().Get("x-migrations-table"),
+			MigrationsTableEngine: migrationsTableEngine,
+			DatabaseName:          purl.Query().Get("database"),
+			ClusterName:           purl.Query().Get("x-cluster-name"),
+			MultiStatementEnabled: purl.Query().Get("x-multi-statement") == "true",
+			MultiStatementMaxSize: multiStatementMaxSize,
+		},
+	}
+
+	if err := ds.init(); err != nil {
+		return nil, err
+	}
+
+	return ds, nil
+}
+
+func (ds *Datastore) init() error {
+	if len(ds.config.DatabaseName) == 0 {
+		if err := ds.conn.QueryRow("SELECT currentDatabase()").Scan(&ds.config.DatabaseName); err != nil {
+			return err
+		}
+	}
+
+	if len(ds.config.MigrationsTable) == 0 {
+		ds.config.MigrationsTable = DefaultMigrationsTable
+	}
+
+	if ds.config.MultiStatementMaxSize <= 0 {
+		ds.config.MultiStatementMaxSize = DefaultMultiStatementMaxSize
+	}
+
+	if len(ds.config.MigrationsTableEngine) == 0 {
+		ds.config.MigrationsTableEngine = DefaultMigrationsTableEngine
+	}
+
+	return ds.ensureVersionTable()
+}
+
+func (ds *Datastore) Run(r io.Reader) error {
+	if ds.config.MultiStatementEnabled {
+		var err error
+		if e := multistmt.Parse(r, multiStmtDelimiter, ds.config.MultiStatementMaxSize, func(m []byte) bool {
+			tq := strings.TrimSpace(string(m))
+			if tq == "" {
+				return true
+			}
+			if _, e := ds.conn.Exec(string(m)); e != nil {
+				err = database.Error{OrigErr: e, Err: "migration failed", Query: m}
+				return false
+			}
+			return true
+		}); e != nil {
+			return e
+		}
+		return err
+	}
+
+	migration, err := io.ReadAll(r)
+	if err != nil {
+		return err
+	}
+
+	if _, err := ds.conn.Exec(string(migration)); err != nil {
+		return database.Error{OrigErr: err, Err: "migration failed", Query: migration}
+	}
+
+	return nil
+}
+
+func (ds *Datastore) Version() (int, bool, error) {
+	var (
+		version int
+		dirty   uint8
+		query   = "SELECT version, dirty FROM `" + ds.config.MigrationsTable + "` ORDER BY sequence DESC LIMIT 1"
+	)
+	if err := ds.conn.QueryRow(query).Scan(&version, &dirty); err != nil {
+		if err == sql.ErrNoRows {
+			return database.NilVersion, false, nil
+		}
+		return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+	return version, dirty == 1, nil
+}
+
+func (ds *Datastore) SetVersion(version int, dirty bool) error {
+	var (
+		bool = func(v bool) uint8 {
+			if v {
+				return 1
+			}
+			return 0
+		}
+		tx, err = ds.conn.Begin()
+	)
+	if err != nil {
+		return err
+	}
+
+	query := "INSERT INTO " + ds.config.MigrationsTable + " (version, dirty, sequence) VALUES (?, ?, ?)"
+	if _, err := tx.Exec(query, version, bool(dirty), time.Now().UnixNano()); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+
+	return tx.Commit()
+}
+
+// ensureVersionTable checks if versions table exists and, if not, creates it.
+// Note that this function locks the database, which deviates from the usual
+// convention of "caller locks" in the Datastore type.
+func (ds *Datastore) ensureVersionTable() (err error) {
+	if err = ds.Lock(); err != nil {
+		return err
+	}
+
+	defer func() {
+		if e := ds.Unlock(); e != nil {
+			err = errors.Join(err, e)
+		}
+	}()
+
+	var (
+		table string
+		query = "SHOW TABLES FROM " + quoteIdentifier(ds.config.DatabaseName) + " LIKE '" + ds.config.MigrationsTable + "'"
+	)
+	// check if migration table exists
+	if err := ds.conn.QueryRow(query).Scan(&table); err != nil {
+		if err != sql.ErrNoRows {
+			return &database.Error{OrigErr: err, Query: []byte(query)}
+		}
+	} else {
+		return nil
+	}
+
+	// if not, create the empty migration table
+	if len(ds.config.ClusterName) > 0 {
+		query = fmt.Sprintf(`
+			CREATE TABLE %s ON CLUSTER %s (
+				version    Int64,
+				dirty      UInt8,
+				sequence   UInt64
+			) Engine=%s`, ds.config.MigrationsTable, ds.config.ClusterName, ds.config.MigrationsTableEngine)
+	} else {
+		query = fmt.Sprintf(`
+			CREATE TABLE %s (
+				version    Int64,
+				dirty      UInt8,
+				sequence   UInt64
+			) Engine=%s`, ds.config.MigrationsTable, ds.config.MigrationsTableEngine)
+	}
+
+	if strings.HasSuffix(ds.config.MigrationsTableEngine, "Tree") {
+		query = fmt.Sprintf(`%s ORDER BY sequence`, query)
+	}
+
+	if _, err := ds.conn.Exec(query); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+	return nil
+}
+
+func (ds *Datastore) Drop() (err error) {
+	query := "SHOW TABLES FROM " + quoteIdentifier(ds.config.DatabaseName)
+	tables, err := ds.conn.Query(query)
+
+	if err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+	defer func() {
+		if errClose := tables.Close(); errClose != nil {
+			err = errors.Join(err, errClose)
+		}
+	}()
+
+	for tables.Next() {
+		var table string
+		if err := tables.Scan(&table); err != nil {
+			return err
+		}
+
+		query = "DROP TABLE IF EXISTS " + quoteIdentifier(ds.config.DatabaseName) + "." + quoteIdentifier(table)
+
+		if _, err := ds.conn.Exec(query); err != nil {
+			return &database.Error{OrigErr: err, Query: []byte(query)}
+		}
+	}
+	if err := tables.Err(); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+
+	return nil
+}
+
+func (ds *Datastore) Lock() error {
+	if !ds.isLocked.CompareAndSwap(false, true) {
+		return database.ErrLocked
+	}
+
+	return nil
+}
+
+func (ds *Datastore) Unlock() error {
+	if !ds.isLocked.CompareAndSwap(true, false) {
+		return database.ErrNotLocked
+	}
+
+	return nil
+}
+
+func (ds *Datastore) Close() error { return ds.conn.Close() }
+
+// Copied from lib/pq implementation: https://github.com/lib/pq/blob/v1.9.0/conn.go#L1611
+func quoteIdentifier(name string) string {
+	end := strings.IndexRune(name, 0)
+	if end > -1 {
+		name = name[:end]
+	}
+	return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
+}

database/datastore/datastore_test.go

@@ -0,0 +1,226 @@
+package datastore_test
+
+import (
+	"context"
+	"database/sql"
+	sqldriver "database/sql/driver"
+	"fmt"
+	"log"
+	"testing"
+
+	"github.com/dhui/dktest"
+	"github.com/golang-migrate/migrate/v4"
+	"github.com/golang-migrate/migrate/v4/database/datastore"
+	dt "github.com/golang-migrate/migrate/v4/database/testing"
+	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+
+	// Registers the "datastore" database/sql driver (ClickHouse-wire native).
+	_ "github.com/hanzo-ds/go"
+)
+
+const defaultPort = 9000
+
+var (
+	tableEngines = []string{"TinyLog", "MergeTree"}
+	opts         = dktest.Options{
+		Env:          map[string]string{"CLICKHOUSE_USER": "user", "CLICKHOUSE_PASSWORD": "password", "CLICKHOUSE_DB": "db"},
+		PortRequired: true, ReadyFunc: isReady,
+	}
+	specs = []dktesting.ContainerSpec{
+		{ImageName: "clickhouse:24.8", Options: opts},
+	}
+)
+
+func datastoreConnectionString(host, port, engine string) string {
+	if engine != "" {
+		return fmt.Sprintf(
+			"datastore://%v:%v?username=user&password=password&database=db&x-multi-statement=true&x-migrations-table-engine=%v&debug=false",
+			host, port, engine)
+	}
+
+	return fmt.Sprintf(
+		"datastore://%v:%v?username=user&password=password&database=db&x-multi-statement=true&debug=false",
+		host, port)
+}
+
+func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
+	ip, port, err := c.Port(defaultPort)
+	if err != nil {
+		return false
+	}
+
+	db, err := sql.Open("datastore", datastoreConnectionString(ip, port, ""))
+
+	if err != nil {
+		log.Println("open error", err)
+		return false
+	}
+	defer func() {
+		if err := db.Close(); err != nil {
+			log.Println("close error:", err)
+		}
+	}()
+
+	if err = db.PingContext(ctx); err != nil {
+		switch err {
+		case sqldriver.ErrBadConn:
+			return false
+		default:
+			fmt.Println(err)
+		}
+		return false
+	}
+
+	return true
+}
+
+func TestCases(t *testing.T) {
+	for _, engine := range tableEngines {
+		t.Run("Test_"+engine, func(t *testing.T) { testSimple(t, engine) })
+		t.Run("Migrate_"+engine, func(t *testing.T) { testMigrate(t, engine) })
+		t.Run("Version_"+engine, func(t *testing.T) { testVersion(t, engine) })
+		t.Run("Drop_"+engine, func(t *testing.T) { testDrop(t, engine) })
+	}
+	t.Run("WithInstanceDefaultConfigValues", func(t *testing.T) { testSimpleWithInstanceDefaultConfigValues(t) })
+}
+
+func testSimple(t *testing.T, engine string) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := datastoreConnectionString(ip, port, engine)
+		p := &datastore.Datastore{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		dt.Test(t, d, []byte("SELECT 1"))
+	})
+}
+
+func testSimpleWithInstanceDefaultConfigValues(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := datastoreConnectionString(ip, port, "")
+		conn, err := sql.Open("datastore", addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		d, err := datastore.WithInstance(conn, &datastore.Config{})
+		if err != nil {
+			_ = conn.Close()
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		dt.Test(t, d, []byte("SELECT 1"))
+	})
+}
+
+func testMigrate(t *testing.T, engine string) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := datastoreConnectionString(ip, port, engine)
+		p := &datastore.Datastore{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "db", d)
+
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+	})
+}
+
+func testVersion(t *testing.T, engine string) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		expectedVersion := 1
+
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := datastoreConnectionString(ip, port, engine)
+		p := &datastore.Datastore{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		err = d.SetVersion(expectedVersion, false)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		version, _, err := d.Version()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if version != expectedVersion {
+			t.Fatal("Version mismatch")
+		}
+	})
+}
+
+func testDrop(t *testing.T, engine string) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := datastoreConnectionString(ip, port, engine)
+		p := &datastore.Datastore{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		err = d.Drop()
+		if err != nil {
+			t.Fatal(err)
+		}
+	})
+}

database/datastore/examples/migrations/001_init.up.sql

@@ -1,3 +1,3 @@
 CREATE TABLE test_1 (
-    Date Date 
+    Date Date
 ) Engine=Memory;

database/datastore/examples/migrations/002_create_table.up.sql

@@ -1,3 +1,3 @@
 CREATE TABLE test_2 (
-    Date Date 
+    Date Date
 ) Engine=Memory;

database/datastore/examples/migrations/003_create_database.down.sql

@@ -0,0 +1,10 @@
+DROP TABLE IF EXISTS driver_ratings;
+DROP TABLE IF EXISTS user_ratings;
+DROP TABLE IF EXISTS orders;
+DROP TABLE IF EXISTS driver_ratings_queue;
+DROP TABLE IF EXISTS user_ratings_queue;
+DROP TABLE IF EXISTS orders_queue;
+DROP VIEW IF EXISTS user_ratings_queue_mv;
+DROP VIEW IF EXISTS driver_ratings_queue_mv;
+DROP VIEW IF EXISTS orders_queue_mv;
+DROP DATABASE IF EXISTS analytics;

database/datastore/examples/migrations/003_create_database.up.sql

@@ -0,0 +1,81 @@
+CREATE DATABASE IF NOT EXISTS analytics;
+
+CREATE TABLE IF NOT EXISTS analytics.driver_ratings(
+    rate UInt8,
+    userID Int64,
+    driverID String,
+    orderID String,
+    inserted_time DateTime DEFAULT now()
+) ENGINE = MergeTree
+PARTITION BY driverID
+ORDER BY (inserted_time);
+
+CREATE TABLE analytics.driver_ratings_queue(
+    rate UInt8,
+    userID Int64,
+    driverID String,
+    orderID String
+) ENGINE = Kafka
+SETTINGS kafka_broker_list = 'broker:9092',
+    kafka_topic_list = 'driver-ratings',
+    kafka_group_name = 'rating_readers',
+    kafka_format = 'Avro',
+    kafka_max_block_size = 1048576;
+
+CREATE MATERIALIZED VIEW analytics.driver_ratings_queue_mv TO analytics.driver_ratings AS
+SELECT rate, userID, driverID, orderID
+FROM analytics.driver_ratings_queue;
+
+CREATE TABLE IF NOT EXISTS analytics.user_ratings(
+    rate UInt8,
+    userID Int64,
+    driverID String,
+    orderID String,
+    inserted_time DateTime DEFAULT now()
+) ENGINE = MergeTree
+    PARTITION BY userID
+    ORDER BY (inserted_time);
+
+CREATE TABLE analytics.user_ratings_queue(
+    rate UInt8,
+    userID Int64,
+    driverID String,
+    orderID String
+) ENGINE = Kafka
+SETTINGS kafka_broker_list = 'broker:9092',
+    kafka_topic_list = 'user-ratings',
+    kafka_group_name = 'rating_readers',
+    kafka_format = 'JSON',
+    kafka_max_block_size = 1048576;
+
+CREATE MATERIALIZED VIEW analytics.user_ratings_queue_mv TO analytics.user_ratings AS
+SELECT rate, userID, driverID, orderID
+FROM analytics.user_ratings_queue;
+
+CREATE TABLE IF NOT EXISTS analytics.orders(
+    from_place String,
+    to_place String,
+    userID Int64,
+    driverID String,
+    orderID String,
+    inserted_time DateTime DEFAULT now()
+) ENGINE = MergeTree
+    PARTITION BY driverID
+    ORDER BY (inserted_time);
+
+CREATE TABLE analytics.orders_queue(
+    from_place String,
+    to_place String,
+    userID Int64,
+    driverID String,
+    orderID String
+) ENGINE = Kafka
+SETTINGS kafka_broker_list = 'broker:9092',
+    kafka_topic_list = 'orders',
+    kafka_group_name = 'order_readers',
+    kafka_format = 'Avro',
+    kafka_max_block_size = 1048576;
+
+CREATE MATERIALIZED VIEW analytics.orders_queue_mv TO orders AS
+SELECT from_place, to_place, userID, driverID, orderID
+FROM analytics.orders_queue;

database/driver.go

@@ -1,4 +1,4 @@
-// Package database provides the Database interface.
+// Package database provides the Driver interface.
 // All database drivers must implement this interface, register themselves,
 // optionally provide a `WithInstance` function and pass the tests
 // in package database/testing.
@@ -7,12 +7,14 @@ package database
 import (
 	"fmt"
 	"io"
-	nurl "net/url"
 	"sync"
+
+	iurl "github.com/golang-migrate/migrate/v4/internal/url"
 )
 
 var (
-	ErrLocked = fmt.Errorf("can't acquire lock")
+	ErrLocked    = fmt.Errorf("can't acquire lock")
+	ErrNotLocked = fmt.Errorf("can't unlock, as not currently locked")
 )
 
 const NilVersion int = -1
@@ -23,22 +25,22 @@ var drivers = make(map[string]Driver)
 // Driver is the interface every database driver must implement.
 //
 // How to implement a database driver?
-//   1. Implement this interface.
-//   2. Optionally, add a function named `WithInstance`.
-//      This function should accept an existing DB instance and a Config{} struct
-//      and return a driver instance.
-//   3. Add a test that calls database/testing.go:Test()
-//   4. Add own tests for Open(), WithInstance() (when provided) and Close().
-//      All other functions are tested by tests in database/testing.
-//      Saves you some time and makes sure all database drivers behave the same way.
-//   5. Call Register in init().
-//   6. Create a migrate/cli/build_<driver-name>.go file
-//   7. Add driver name in 'DATABASE' variable in Makefile
+//  1. Implement this interface.
+//  2. Optionally, add a function named `WithInstance`.
+//     This function should accept an existing DB instance and a Config{} struct
+//     and return a driver instance.
+//  3. Add a test that calls database/testing.go:Test()
+//  4. Add own tests for Open(), WithInstance() (when provided) and Close().
+//     All other functions are tested by tests in database/testing.
+//     Saves you some time and makes sure all database drivers behave the same way.
+//  5. Call Register in init().
+//  6. Create a internal/cli/build_<driver-name>.go file
+//  7. Add driver name in 'DATABASE' variable in Makefile
 //
 // Guidelines:
-//   * Don't try to correct user input. Don't assume things.
+//   - Don't try to correct user input. Don't assume things.
 //     When in doubt, return an error and explain the situation to the user.
-//   * All configuration input must come from the URL string in func Open()
+//   - All configuration input must come from the URL string in func Open()
 //     or the Config{} struct in WithInstance. Don't os.Getenv().
 type Driver interface {
 	// Open returns a new driver instance configured with parameters
@@ -60,7 +62,7 @@ type Driver interface {
 	// all migrations have been run.
 	Unlock() error
 
-	// Run applies a migration to the database. migration is garantueed to be not nil.
+	// Run applies a migration to the database. migration is guaranteed to be not nil.
 	Run(migration io.Reader) error
 
 	// SetVersion saves version and dirty state.
@@ -74,26 +76,23 @@ type Driver interface {
 	Version() (version int, dirty bool, err error)
 
 	// Drop deletes everything in the database.
+	// Note that this is a breaking action, a new call to Open() is necessary to
+	// ensure subsequent calls work as expected.
 	Drop() error
 }
 
 // Open returns a new driver instance.
 func Open(url string) (Driver, error) {
-	u, err := nurl.Parse(url)
+	scheme, err := iurl.SchemeFromURL(url)
 	if err != nil {
-		return nil, fmt.Errorf("Unable to parse URL. Did you escape all reserved URL characters? "+
-			"See: https://github.com/golang-migrate/migrate#database-urls Error: %v", err)
-	}
-
-	if u.Scheme == "" {
-		return nil, fmt.Errorf("database driver: invalid URL scheme")
+		return nil, err
 	}
 
 	driversMu.RLock()
-	d, ok := drivers[u.Scheme]
+	d, ok := drivers[scheme]
 	driversMu.RUnlock()
 	if !ok {
-		return nil, fmt.Errorf("database driver: unknown driver %v (forgotten import?)", u.Scheme)
+		return nil, fmt.Errorf("database driver: unknown driver %v (forgotten import?)", scheme)
 	}
 
 	return d.Open(url)

database/driver_test.go

@@ -1,8 +1,115 @@
 package database
 
+import (
+	"io"
+	"testing"
+)
+
 func ExampleDriver() {
 	// see database/stub for an example
 
 	// database/stub/stub.go has the driver implementation
 	// database/stub/stub_test.go runs database/testing/test.go:Test
 }
+
+// Using database/stub here is not possible as it
+// results in an import cycle.
+type mockDriver struct {
+	url string
+}
+
+func (m *mockDriver) Open(url string) (Driver, error) {
+	return &mockDriver{
+		url: url,
+	}, nil
+}
+
+func (m *mockDriver) Close() error {
+	return nil
+}
+
+func (m *mockDriver) Lock() error {
+	return nil
+}
+
+func (m *mockDriver) Unlock() error {
+	return nil
+}
+
+func (m *mockDriver) Run(migration io.Reader) error {
+	return nil
+}
+
+func (m *mockDriver) SetVersion(version int, dirty bool) error {
+	return nil
+}
+
+func (m *mockDriver) Version() (version int, dirty bool, err error) {
+	return 0, false, nil
+}
+
+func (m *mockDriver) Drop() error {
+	return nil
+}
+
+func TestRegisterTwice(t *testing.T) {
+	Register("mock", &mockDriver{})
+
+	var err interface{}
+	func() {
+		defer func() {
+			err = recover()
+		}()
+		Register("mock", &mockDriver{})
+	}()
+
+	if err == nil {
+		t.Fatal("expected a panic when calling Register twice")
+	}
+}
+
+func TestOpen(t *testing.T) {
+	// Make sure the driver is registered.
+	// But if the previous test already registered it just ignore the panic.
+	// If we don't do this it will be impossible to run this test standalone.
+	func() {
+		defer func() {
+			_ = recover()
+		}()
+		Register("mock", &mockDriver{})
+	}()
+
+	cases := []struct {
+		url string
+		err bool
+	}{
+		{
+			"mock://user:pass@tcp(host:1337)/db",
+			false,
+		},
+		{
+			"unknown://bla",
+			true,
+		},
+	}
+
+	for _, c := range cases {
+		t.Run(c.url, func(t *testing.T) {
+			d, err := Open(c.url)
+
+			if err == nil {
+				if c.err {
+					t.Fatal("expected an error for an unknown driver")
+				} else {
+					if md, ok := d.(*mockDriver); !ok {
+						t.Fatalf("expected *mockDriver got %T", d)
+					} else if md.url != c.url {
+						t.Fatalf("expected %q got %q", c.url, md.url)
+					}
+				}
+			} else if !c.err {
+				t.Fatalf("did not expect %q", err)
+			}
+		})
+	}
+}

database/firebird/README.md

@@ -0,0 +1,12 @@
+# firebird
+
+`firebirdsql://user:password@servername[:port_number]/database_name_or_file[?params1=value1[&param2=value2]...]`
+
+| URL Query  | WithInstance Config | Description |
+|------------|---------------------|-------------|
+| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
+| `auth_plugin_name` | | Authentication plugin name. Srp256/Srp/Legacy_Auth are available. (default is Srp) |
+| `column_name_to_lower` | | Force column name to lower. (default is false) |
+| `role` | | Role name |
+| `tzname` | | Time Zone name. (For Firebird 4.0+) |
+| `wire_crypt` | | Enable wire data encryption or not. For Firebird 3.0+ (default is true) |

database/firebird/examples/migrations/1085649617_create_users_table.down.sql

@@ -0,0 +1 @@
+DROP TABLE users;

database/firebird/examples/migrations/1085649617_create_users_table.up.sql

@@ -0,0 +1,5 @@
+CREATE TABLE users (
+  user_id integer unique,
+  name    varchar(40),
+  email   varchar(40)
+);

database/firebird/examples/migrations/1185749658_add_city_to_users.down.sql

@@ -0,0 +1 @@
+ALTER TABLE users DROP city;

database/firebird/examples/migrations/1185749658_add_city_to_users.up.sql

@@ -0,0 +1,3 @@
+ALTER TABLE users ADD city varchar(100);
+
+

database/firebird/examples/migrations/1285849751_add_index_on_user_emails.down.sql

@@ -0,0 +1 @@
+DROP INDEX users_email_index;

database/firebird/examples/migrations/1285849751_add_index_on_user_emails.up.sql

@@ -0,0 +1,3 @@
+CREATE UNIQUE INDEX users_email_index ON users (email);
+
+-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.

database/firebird/examples/migrations/1385949617_create_books_table.down.sql

@@ -0,0 +1 @@
+DROP TABLE books;

database/firebird/examples/migrations/1385949617_create_books_table.up.sql

@@ -0,0 +1,5 @@
+CREATE TABLE books (
+  user_id integer,
+  name    varchar(40),
+  author  varchar(40)
+);

database/firebird/examples/migrations/1485949617_create_movies_table.down.sql

@@ -0,0 +1 @@
+DROP TABLE movies;

database/firebird/examples/migrations/1485949617_create_movies_table.up.sql

@@ -0,0 +1,5 @@
+CREATE TABLE movies (
+  user_id   integer,
+  name      varchar(40),
+  director  varchar(40)
+);

database/firebird/firebird.go

@@ -0,0 +1,254 @@
+//go:build go1.9
+
+package firebird
+
+import (
+	"context"
+	"database/sql"
+	"errors"
+	"fmt"
+	"io"
+	nurl "net/url"
+	"sync/atomic"
+
+	"github.com/golang-migrate/migrate/v4"
+	"github.com/golang-migrate/migrate/v4/database"
+	_ "github.com/nakagami/firebirdsql"
+)
+
+func init() {
+	db := Firebird{}
+	database.Register("firebird", &db)
+	database.Register("firebirdsql", &db)
+}
+
+var DefaultMigrationsTable = "schema_migrations"
+
+var (
+	ErrNilConfig = fmt.Errorf("no config")
+)
+
+type Config struct {
+	DatabaseName    string
+	MigrationsTable string
+}
+
+type Firebird struct {
+	// Locking and unlocking need to use the same connection
+	conn     *sql.Conn
+	db       *sql.DB
+	isLocked atomic.Bool
+
+	// Open and WithInstance need to guarantee that config is never nil
+	config *Config
+}
+
+func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
+	if config == nil {
+		return nil, ErrNilConfig
+	}
+
+	if err := instance.Ping(); err != nil {
+		return nil, err
+	}
+
+	if len(config.MigrationsTable) == 0 {
+		config.MigrationsTable = DefaultMigrationsTable
+	}
+
+	conn, err := instance.Conn(context.Background())
+	if err != nil {
+		return nil, err
+	}
+
+	fb := &Firebird{
+		conn:   conn,
+		db:     instance,
+		config: config,
+	}
+
+	if err := fb.ensureVersionTable(); err != nil {
+		return nil, err
+	}
+
+	return fb, nil
+}
+
+func (f *Firebird) Open(dsn string) (database.Driver, error) {
+	purl, err := nurl.Parse(dsn)
+	if err != nil {
+		return nil, err
+	}
+
+	db, err := sql.Open("firebirdsql", migrate.FilterCustomQuery(purl).String())
+	if err != nil {
+		return nil, err
+	}
+
+	px, err := WithInstance(db, &Config{
+		MigrationsTable: purl.Query().Get("x-migrations-table"),
+		DatabaseName:    purl.Path,
+	})
+
+	if err != nil {
+		return nil, err
+	}
+
+	return px, nil
+}
+
+func (f *Firebird) Close() error {
+	connErr := f.conn.Close()
+	dbErr := f.db.Close()
+	if connErr != nil || dbErr != nil {
+		return fmt.Errorf("conn: %v, db: %v", connErr, dbErr)
+	}
+	return nil
+}
+
+func (f *Firebird) Lock() error {
+	if !f.isLocked.CompareAndSwap(false, true) {
+		return database.ErrLocked
+	}
+	return nil
+}
+
+func (f *Firebird) Unlock() error {
+	if !f.isLocked.CompareAndSwap(true, false) {
+		return database.ErrNotLocked
+	}
+	return nil
+}
+
+func (f *Firebird) Run(migration io.Reader) error {
+	migr, err := io.ReadAll(migration)
+	if err != nil {
+		return err
+	}
+
+	// run migration
+	query := string(migr[:])
+	if _, err := f.conn.ExecContext(context.Background(), query); err != nil {
+		return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
+	}
+
+	return nil
+}
+
+func (f *Firebird) SetVersion(version int, dirty bool) error {
+	// Always re-write the schema version to prevent empty schema version
+	// for failed down migration on the first migration
+	// See: https://github.com/golang-migrate/migrate/issues/330
+
+	// TODO: parameterize this SQL statement
+	//       https://firebirdsql.org/refdocs/langrefupd20-execblock.html
+	//       VALUES (?, ?) doesn't work
+	query := fmt.Sprintf(`EXECUTE BLOCK AS BEGIN
+					DELETE FROM "%v";
+					INSERT INTO "%v" (version, dirty) VALUES (%v, %v);
+				END;`,
+		f.config.MigrationsTable, f.config.MigrationsTable, version, btoi(dirty))
+
+	if _, err := f.conn.ExecContext(context.Background(), query); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+
+	return nil
+}
+
+func (f *Firebird) Version() (version int, dirty bool, err error) {
+	var d int
+	query := fmt.Sprintf(`SELECT FIRST 1 version, dirty FROM "%v"`, f.config.MigrationsTable)
+	err = f.conn.QueryRowContext(context.Background(), query).Scan(&version, &d)
+	switch {
+	case err == sql.ErrNoRows:
+		return database.NilVersion, false, nil
+	case err != nil:
+		return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
+
+	default:
+		return version, itob(d), nil
+	}
+}
+
+func (f *Firebird) Drop() (err error) {
+	// select all tables
+	query := `SELECT rdb$relation_name FROM rdb$relations WHERE rdb$view_blr IS NULL AND (rdb$system_flag IS NULL OR rdb$system_flag = 0);`
+	tables, err := f.conn.QueryContext(context.Background(), query)
+	if err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+	defer func() {
+		if errClose := tables.Close(); errClose != nil {
+			err = errors.Join(err, errClose)
+		}
+	}()
+
+	// delete one table after another
+	tableNames := make([]string, 0)
+	for tables.Next() {
+		var tableName string
+		if err := tables.Scan(&tableName); err != nil {
+			return err
+		}
+		if len(tableName) > 0 {
+			tableNames = append(tableNames, tableName)
+		}
+	}
+	if err := tables.Err(); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+
+	// delete one by one ...
+	for _, t := range tableNames {
+		query := fmt.Sprintf(`EXECUTE BLOCK AS BEGIN
+						if (not exists(select 1 from rdb$relations where rdb$relation_name = '%v')) then
+						execute statement 'drop table "%v"';
+					END;`,
+			t, t)
+
+		if _, err := f.conn.ExecContext(context.Background(), query); err != nil {
+			return &database.Error{OrigErr: err, Query: []byte(query)}
+		}
+	}
+
+	return nil
+}
+
+// ensureVersionTable checks if versions table exists and, if not, creates it.
+func (f *Firebird) ensureVersionTable() (err error) {
+	if err = f.Lock(); err != nil {
+		return err
+	}
+
+	defer func() {
+		if e := f.Unlock(); e != nil {
+			err = errors.Join(err, e)
+		}
+	}()
+
+	query := fmt.Sprintf(`EXECUTE BLOCK AS BEGIN
+			if (not exists(select 1 from rdb$relations where rdb$relation_name = '%v')) then
+			execute statement 'create table "%v" (version bigint not null primary key, dirty smallint not null)';
+		END;`,
+		f.config.MigrationsTable, f.config.MigrationsTable)
+
+	if _, err = f.conn.ExecContext(context.Background(), query); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
+
+	return nil
+}
+
+// btoi converts bool to int
+func btoi(v bool) int {
+	if v {
+		return 1
+	}
+	return 0
+}
+
+// itob converts int to bool
+func itob(v int) bool {
+	return v != 0
+}

database/firebird/firebird_test.go

@@ -0,0 +1,227 @@
+package firebird
+
+import (
+	"context"
+	"database/sql"
+	sqldriver "database/sql/driver"
+	"fmt"
+	"log"
+
+	"github.com/golang-migrate/migrate/v4"
+	"io"
+	"strings"
+	"testing"
+
+	"github.com/dhui/dktest"
+
+	dt "github.com/golang-migrate/migrate/v4/database/testing"
+	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+
+	_ "github.com/nakagami/firebirdsql"
+)
+
+const (
+	user     = "test_user"
+	password = "123456"
+	dbName   = "test.fdb"
+)
+
+var (
+	opts = dktest.Options{
+		PortRequired: true,
+		ReadyFunc:    isReady,
+		Env: map[string]string{
+			"FIREBIRD_DATABASE": dbName,
+			"FIREBIRD_USER":     user,
+			"FIREBIRD_PASSWORD": password,
+		},
+	}
+	specs = []dktesting.ContainerSpec{
+		{ImageName: "jacobalberty/firebird:v3.0", Options: opts},
+		{ImageName: "jacobalberty/firebird:v4.0", Options: opts},
+		{ImageName: "jacobalberty/firebird:v5.0", Options: opts},
+	}
+)
+
+func fbConnectionString(host, port string) string {
+	//firebird://user:password@servername[:port_number]/database_name_or_file[?params1=value1[&param2=value2]...]
+	return fmt.Sprintf("firebird://%s:%s@%s:%s//firebird/data/%s", user, password, host, port, dbName)
+}
+
+func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
+	ip, port, err := c.FirstPort()
+	if err != nil {
+		return false
+	}
+
+	db, err := sql.Open("firebirdsql", fbConnectionString(ip, port))
+	if err != nil {
+		log.Println("open error:", err)
+		return false
+	}
+	defer func() {
+		if err := db.Close(); err != nil {
+			log.Println("close error:", err)
+		}
+	}()
+	if err = db.PingContext(ctx); err != nil {
+		switch err {
+		case sqldriver.ErrBadConn, io.EOF:
+			return false
+		default:
+			log.Println(err)
+		}
+		return false
+	}
+
+	return true
+}
+
+func Test(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fbConnectionString(ip, port)
+		p := &Firebird{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+		dt.Test(t, d, []byte("SELECT Count(*) FROM rdb$relations"))
+	})
+}
+
+func TestMigrate(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fbConnectionString(ip, port)
+		p := &Firebird{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "firebirdsql", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+	})
+}
+
+func TestErrorParsing(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fbConnectionString(ip, port)
+		p := &Firebird{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		wantErr := `migration failed in line 0: CREATE TABLEE foo (foo varchar(40)); (details: Dynamic SQL Error
+SQL error code = -104
+Token unknown - line 1, column 8
+TABLEE
+)`
+
+		if err := d.Run(strings.NewReader("CREATE TABLEE foo (foo varchar(40));")); err == nil {
+			t.Fatal("expected err but got nil")
+		} else if err.Error() != wantErr {
+			msg := err.Error()
+			t.Fatalf("expected '%s' but got '%s'", wantErr, msg)
+		}
+	})
+}
+
+func TestFilterCustomQuery(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fbConnectionString(ip, port) + "?sslmode=disable&x-custom=foobar"
+		p := &Firebird{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+	})
+}
+
+func Test_Lock(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fbConnectionString(ip, port)
+		p := &Firebird{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		dt.Test(t, d, []byte("SELECT Count(*) FROM rdb$relations"))
+
+		ps := d.(*Firebird)
+
+		err = ps.Lock()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		err = ps.Unlock()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		err = ps.Lock()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		err = ps.Unlock()
+		if err != nil {
+			t.Fatal(err)
+		}
+	})
+}

database/mongodb/README.md

@@ -7,12 +7,16 @@
 
 # Usage
 
-`mongodb://user:password@host:port/dbname?query`
+`mongodb://user:password@host:port/dbname?query` (`mongodb+srv://` also works, but behaves a bit differently. See [docs](https://docs.mongodb.com/manual/reference/connection-string/#dns-seedlist-connection-format) for more information)
 
 | URL Query  | WithInstance Config | Description |
 |------------|---------------------|-------------|
-| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
+| `x-migrations-collection` | `MigrationsCollection` | Name of the migrations collection |
 | `x-transaction-mode` | `TransactionMode` | If set to `true` wrap commands in [transaction](https://docs.mongodb.com/manual/core/transactions). Available only for replica set. Driver is using [strconv.ParseBool](https://golang.org/pkg/strconv/#ParseBool) for parsing|
+| `x-advisory-locking` | `true` | Feature flag for advisory locking, if set to false, disable advisory locking |
+| `x-advisory-lock-collection` | `migrate_advisory_lock` | The name of the collection to use for advisory locking.|
+| `x-advisory-lock-timeout` | `15` | The max time in seconds that migrate will wait to acquire a lock before failing. |
+| `x-advisory-lock-timeout-interval` | `10` | The max time in seconds between attempts to acquire the advisory lock, the lock is attempted to be acquired using an exponential backoff algorithm. |
 | `dbname` | `DatabaseName` | The name of the database to connect to |
 | `user` | | The user to sign in as. Can be omitted |
 | `password` | | The user's password. Can be omitted | 

database/mongodb/examples/migrations/003_add_new_field.down.json

@@ -0,0 +1,16 @@
+[
+    {
+        "update": "users",
+        "updates": [
+            {
+                "q": {},
+                "u": {
+                    "$unset": {
+                        "status": ""
+                    }
+                },
+                "multi": true
+            }
+        ]
+    }
+]

database/mongodb/examples/migrations/003_add_new_field.up.json

@@ -0,0 +1,16 @@
+[
+    {
+        "update": "users",
+        "updates": [
+            {
+                "q": {},
+                "u": {
+                    "$set": {
+                        "status": "active"
+                    }
+                },
+                "multi": true
+            }
+        ]
+    }
+]

database/mongodb/examples/migrations/004_replace_field_value_from_another_field.down.json

@@ -0,0 +1,14 @@
+[
+    {
+        "update": "users",
+        "updates": [
+            {
+                "q": {},
+                "u": {
+                    "fullname": ""
+                },
+                "multi": true
+            }
+        ]
+    }
+]

database/mongodb/examples/migrations/004_replace_field_value_from_another_field.up.json

@@ -0,0 +1,23 @@
+[
+    {
+        "aggregate": "users",
+        "pipeline": [
+            {
+                "$project": {
+                    "_id": 1,
+                    "firstname": 1,
+                    "lastname": 1,
+                    "username": 1,
+                    "password": 1,
+                    "email": 1,
+                    "active": 1,
+                    "fullname": { "$concat": ["$firstname", " ", "$lastname"] }
+                }
+            },
+            {
+                "$out": "users"
+            }
+        ],
+        "cursor": {}
+    }
+]

database/mongodb/mongodb.go

@@ -2,48 +2,79 @@ package mongodb
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"io"
-	"io/ioutil"
 	"net/url"
+	"os"
 	"strconv"
+	"sync/atomic"
+	"time"
 
-	"github.com/golang-migrate/migrate/v4"
+	"github.com/cenkalti/backoff/v4"
 	"github.com/golang-migrate/migrate/v4/database"
-	"github.com/mongodb/mongo-go-driver/bson"
-	"github.com/mongodb/mongo-go-driver/mongo"
-	"github.com/mongodb/mongo-go-driver/x/network/connstring"
+	"go.mongodb.org/mongo-driver/bson"
+	"go.mongodb.org/mongo-driver/mongo"
+	"go.mongodb.org/mongo-driver/mongo/options"
+	"go.mongodb.org/mongo-driver/x/mongo/driver/connstring"
 )
 
 func init() {
-	database.Register("mongodb", &Mongo{})
+	db := Mongo{}
+	database.Register("mongodb", &db)
+	database.Register("mongodb+srv", &db)
 }
 
 var DefaultMigrationsCollection = "schema_migrations"
 
+const DefaultLockingCollection = "migrate_advisory_lock" // the collection to use for advisory locking by default.
+const lockKeyUniqueValue = 0                             // the unique value to lock on. If multiple clients try to insert the same key, it will fail (locked).
+const DefaultLockTimeout = 15                            // the default maximum time to wait for a lock to be released.
+const DefaultLockTimeoutInterval = 10                    // the default maximum intervals time for the locking timout.
+const DefaultAdvisoryLockingFlag = true                  // the default value for the advisory locking feature flag. Default is true.
+const LockIndexName = "lock_unique_key"                  // the name of the index which adds unique constraint to the locking_key field.
+const contextWaitTimeout = 5 * time.Second               // how long to wait for the request to mongo to block/wait for.
+
 var (
-	ErrNoDatabaseName = fmt.Errorf("no database name")
-	ErrNilConfig      = fmt.Errorf("no config")
+	ErrNoDatabaseName            = fmt.Errorf("no database name")
+	ErrNilConfig                 = fmt.Errorf("no config")
+	ErrLockTimeoutConfigConflict = fmt.Errorf("both x-advisory-lock-timeout-interval and x-advisory-lock-timout-interval were specified")
 )
 
 type Mongo struct {
-	client *mongo.Client
-	db     *mongo.Database
-
-	config *Config
+	client   *mongo.Client
+	db       *mongo.Database
+	config   *Config
+	isLocked atomic.Bool
 }
 
+type Locking struct {
+	CollectionName string
+	Timeout        int
+	Enabled        bool
+	Interval       int
+}
 type Config struct {
 	DatabaseName         string
 	MigrationsCollection string
 	TransactionMode      bool
+	Locking              Locking
 }
-
 type versionInfo struct {
 	Version int  `bson:"version"`
 	Dirty   bool `bson:"dirty"`
 }
 
+type lockObj struct {
+	Key       int       `bson:"locking_key"`
+	Pid       int       `bson:"pid"`
+	Hostname  string    `bson:"hostname"`
+	CreatedAt time.Time `bson:"created_at"`
+}
+type findFilter struct {
+	Key int `bson:"locking_key"`
+}
+
 func WithInstance(instance *mongo.Client, config *Config) (database.Driver, error) {
 	if config == nil {
 		return nil, ErrNilConfig
@@ -54,16 +85,36 @@ func WithInstance(instance *mongo.Client, config *Config) (database.Driver, erro
 	if len(config.MigrationsCollection) == 0 {
 		config.MigrationsCollection = DefaultMigrationsCollection
 	}
+	if len(config.Locking.CollectionName) == 0 {
+		config.Locking.CollectionName = DefaultLockingCollection
+	}
+	if config.Locking.Timeout <= 0 {
+		config.Locking.Timeout = DefaultLockTimeout
+	}
+	if config.Locking.Interval <= 0 {
+		config.Locking.Interval = DefaultLockTimeoutInterval
+	}
+
 	mc := &Mongo{
 		client: instance,
 		db:     instance.Database(config.DatabaseName),
 		config: config,
 	}
+
+	if mc.config.Locking.Enabled {
+		if err := mc.ensureLockTable(); err != nil {
+			return nil, err
+		}
+	}
+	if err := mc.ensureVersionTable(); err != nil {
+		return nil, err
+	}
+
 	return mc, nil
 }
 
 func (m *Mongo) Open(dsn string) (database.Driver, error) {
-	//connsting is experimental package, but it used for parse connection string in mongo.Connect function
+	// connstring is experimental package, but it used for parse connection string in mongo.Connect function
 	uri, err := connstring.Parse(dsn)
 	if err != nil {
 		return nil, err
@@ -71,25 +122,46 @@ func (m *Mongo) Open(dsn string) (database.Driver, error) {
 	if len(uri.Database) == 0 {
 		return nil, ErrNoDatabaseName
 	}
+	unknown := url.Values(uri.UnknownOptions)
 
-	purl, err := url.Parse(dsn)
+	migrationsCollection := unknown.Get("x-migrations-collection")
+	lockCollection := unknown.Get("x-advisory-lock-collection")
+	transactionMode, err := parseBoolean(unknown.Get("x-transaction-mode"), false)
 	if err != nil {
 		return nil, err
 	}
-	migrationsCollection := purl.Query().Get("x-migrations-collection")
-	if len(migrationsCollection) == 0 {
-		migrationsCollection = DefaultMigrationsCollection
-	}
-
-	transactionMode, _ := strconv.ParseBool(purl.Query().Get("x-transaction-mode"))
-
-	q := migrate.FilterCustomQuery(purl)
-	q.Scheme = "mongodb"
-
-	client, err := mongo.Connect(context.TODO(), q.String())
+	advisoryLockingFlag, err := parseBoolean(unknown.Get("x-advisory-locking"), DefaultAdvisoryLockingFlag)
 	if err != nil {
 		return nil, err
 	}
+	lockingTimout, err := parseInt(unknown.Get("x-advisory-lock-timeout"), DefaultLockTimeout)
+	if err != nil {
+		return nil, err
+	}
+
+	lockTimeoutIntervalValue := unknown.Get("x-advisory-lock-timeout-interval")
+	// The initial release had a typo for this argument but for backwards compatibility sake, we will keep supporting it
+	// and we will error out if both values are set.
+	lockTimeoutIntervalValueFromTypo := unknown.Get("x-advisory-lock-timout-interval")
+
+	lockTimeout := lockTimeoutIntervalValue
+
+	if lockTimeoutIntervalValue != "" && lockTimeoutIntervalValueFromTypo != "" {
+		return nil, ErrLockTimeoutConfigConflict
+	} else if lockTimeoutIntervalValueFromTypo != "" {
+		lockTimeout = lockTimeoutIntervalValueFromTypo
+	}
+
+	maxLockCheckInterval, err := parseInt(lockTimeout, DefaultLockTimeoutInterval)
+
+	if err != nil {
+		return nil, err
+	}
+	client, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(dsn))
+	if err != nil {
+		return nil, err
+	}
+
 	if err = client.Ping(context.TODO(), nil); err != nil {
 		return nil, err
 	}
@@ -97,6 +169,12 @@ func (m *Mongo) Open(dsn string) (database.Driver, error) {
 		DatabaseName:         uri.Database,
 		MigrationsCollection: migrationsCollection,
 		TransactionMode:      transactionMode,
+		Locking: Locking{
+			CollectionName: lockCollection,
+			Timeout:        lockingTimout,
+			Enabled:        advisoryLockingFlag,
+			Interval:       maxLockCheckInterval,
+		},
 	})
 	if err != nil {
 		return nil, err
@@ -104,6 +182,39 @@ func (m *Mongo) Open(dsn string) (database.Driver, error) {
 	return mc, nil
 }
 
+// Parse the url param, convert it to boolean
+// returns error if param invalid. returns defaultValue if param not present
+func parseBoolean(urlParam string, defaultValue bool) (bool, error) {
+
+	// if parameter passed, parse it (otherwise return default value)
+	if urlParam != "" {
+		result, err := strconv.ParseBool(urlParam)
+		if err != nil {
+			return false, err
+		}
+		return result, nil
+	}
+
+	// if no url Param passed, return default value
+	return defaultValue, nil
+}
+
+// Parse the url param, convert it to int
+// returns error if param invalid. returns defaultValue if param not present
+func parseInt(urlParam string, defaultValue int) (int, error) {
+
+	// if parameter passed, parse it (otherwise return default value)
+	if urlParam != "" {
+		result, err := strconv.Atoi(urlParam)
+		if err != nil {
+			return -1, err
+		}
+		return result, nil
+	}
+
+	// if no url Param passed, return default value
+	return defaultValue, nil
+}
 func (m *Mongo) SetVersion(version int, dirty bool) error {
 	migrationsCollection := m.db.Collection(m.config.MigrationsCollection)
 	if err := migrationsCollection.Drop(context.TODO()); err != nil {
@@ -130,7 +241,7 @@ func (m *Mongo) Version() (version int, dirty bool, err error) {
 }
 
 func (m *Mongo) Run(migration io.Reader) error {
-	migr, err := ioutil.ReadAll(migration)
+	migr, err := io.ReadAll(migration)
 	if err != nil {
 		return err
 	}
@@ -157,8 +268,8 @@ func (m *Mongo) executeCommandsWithTransaction(ctx context.Context, cmds []bson.
 			return &database.Error{OrigErr: err, Err: "failed to start transaction"}
 		}
 		if err := m.executeCommands(sessionContext, cmds); err != nil {
-			//When command execution is failed, it's aborting transaction
-			//If you tried to call abortTransaction, it`s return error that transaction already aborted
+			// When command execution is failed, it's aborting transaction
+			// If you tried to call abortTransaction, it`s return error that transaction already aborted
 			return err
 		}
 		if err := sessionContext.CommitTransaction(sessionContext); err != nil {
@@ -190,10 +301,100 @@ func (m *Mongo) Drop() error {
 	return m.db.Drop(context.TODO())
 }
 
-func (m *Mongo) Lock() error {
+func (m *Mongo) ensureLockTable() error {
+	indexes := m.db.Collection(m.config.Locking.CollectionName).Indexes()
+
+	indexOptions := options.Index().SetUnique(true).SetName(LockIndexName)
+	_, err := indexes.CreateOne(context.TODO(), mongo.IndexModel{
+		Options: indexOptions,
+		Keys:    findFilter{Key: -1},
+	})
+	if err != nil {
+		return err
+	}
 	return nil
 }
 
-func (m *Mongo) Unlock() error {
+// ensureVersionTable checks if versions table exists and, if not, creates it.
+// Note that this function locks the database, which deviates from the usual
+// convention of "caller locks" in the MongoDb type.
+func (m *Mongo) ensureVersionTable() (err error) {
+	if err = m.Lock(); err != nil {
+		return err
+	}
+
+	defer func() {
+		if e := m.Unlock(); e != nil {
+			err = errors.Join(err, e)
+		}
+	}()
+
+	if err != nil {
+		return err
+	}
+	if _, _, err = m.Version(); err != nil {
+		return err
+	}
 	return nil
 }
+
+// Utilizes advisory locking on the config.LockingCollection collection
+// This uses a unique index on the `locking_key` field.
+func (m *Mongo) Lock() error {
+	return database.CasRestoreOnErr(&m.isLocked, false, true, database.ErrLocked, func() error {
+		if !m.config.Locking.Enabled {
+			return nil
+		}
+
+		pid := os.Getpid()
+		hostname, err := os.Hostname()
+		if err != nil {
+			hostname = fmt.Sprintf("Could not determine hostname. Error: %s", err.Error())
+		}
+
+		newLockObj := lockObj{
+			Key:       lockKeyUniqueValue,
+			Pid:       pid,
+			Hostname:  hostname,
+			CreatedAt: time.Now(),
+		}
+		operation := func() error {
+			timeout, cancelFunc := context.WithTimeout(context.Background(), contextWaitTimeout)
+			_, err := m.db.Collection(m.config.Locking.CollectionName).InsertOne(timeout, newLockObj)
+			defer cancelFunc()
+			return err
+		}
+		exponentialBackOff := backoff.NewExponentialBackOff()
+		duration := time.Duration(m.config.Locking.Timeout) * time.Second
+		exponentialBackOff.MaxElapsedTime = duration
+		exponentialBackOff.MaxInterval = time.Duration(m.config.Locking.Interval) * time.Second
+
+		err = backoff.Retry(operation, exponentialBackOff)
+		if err != nil {
+			return database.ErrLocked
+		}
+
+		return nil
+	})
+}
+
+func (m *Mongo) Unlock() error {
+	return database.CasRestoreOnErr(&m.isLocked, true, false, database.ErrNotLocked, func() error {
+		if !m.config.Locking.Enabled {
+			return nil
+		}
+
+		filter := findFilter{
+			Key: lockKeyUniqueValue,
+		}
+
+		ctx, cancel := context.WithTimeout(context.Background(), contextWaitTimeout)
+		_, err := m.db.Collection(m.config.Locking.CollectionName).DeleteMany(ctx, filter)
+		defer cancel()
+
+		if err != nil {
+			return err
+		}
+		return nil
+	})
+}

database/mongodb/mongodb_test.go

@@ -4,6 +4,10 @@ import (
 	"bytes"
 	"context"
 	"fmt"
+
+	"log"
+
+	"github.com/golang-migrate/migrate/v4"
 	"io"
 	"os"
 	"strconv"
@@ -13,29 +17,32 @@ import (
 
 import (
 	"github.com/dhui/dktest"
-	"github.com/mongodb/mongo-go-driver/bson"
-	"github.com/mongodb/mongo-go-driver/mongo"
+	"go.mongodb.org/mongo-driver/bson"
+	"go.mongodb.org/mongo-driver/mongo"
+	"go.mongodb.org/mongo-driver/mongo/options"
 )
 
 import (
 	dt "github.com/golang-migrate/migrate/v4/database/testing"
 	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
 )
 
 var (
 	opts = dktest.Options{PortRequired: true, ReadyFunc: isReady}
 	// Supported versions: https://www.mongodb.com/support-policy
 	specs = []dktesting.ContainerSpec{
-		{ImageName: "mongo:3.4", Options: opts},
-		{ImageName: "mongo:3.6", Options: opts},
-		{ImageName: "mongo:4.0", Options: opts},
+		{ImageName: "mongo:5.0", Options: opts},
+		{ImageName: "mongo:6.0", Options: opts},
+		{ImageName: "mongo:7.0", Options: opts},
+		{ImageName: "mongo:8.0", Options: opts},
 	}
 )
 
 func mongoConnectionString(host, port string) string {
 	// there is connect option for excluding serverConnection algorithm
 	// it's let avoid errors with mongo replica set connection in docker container
-	return fmt.Sprintf("mongodb://%s:%s/testMigration?connect=single", host, port)
+	return fmt.Sprintf("mongodb://%s:%s/testMigration?connect=direct", host, port)
 }
 
 func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
@@ -44,17 +51,22 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
 		return false
 	}
 
-	client, err := mongo.Connect(ctx, mongoConnectionString(ip, port))
+	client, err := mongo.Connect(ctx, options.Client().ApplyURI(mongoConnectionString(ip, port)))
 	if err != nil {
 		return false
 	}
-	defer client.Disconnect(ctx)
+	defer func() {
+		if err := client.Disconnect(ctx); err != nil {
+			log.Println("close error:", err)
+		}
+	}()
+
 	if err = client.Ping(ctx, nil); err != nil {
 		switch err {
 		case io.EOF:
 			return false
 		default:
-			fmt.Println(err)
+			log.Println(err)
 		}
 		return false
 	}
@@ -62,6 +74,22 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
 }
 
 func Test(t *testing.T) {
+	t.Run("test", test)
+	t.Run("testMigrate", testMigrate)
+	t.Run("testWithAuth", testWithAuth)
+	t.Run("testLockWorks", testLockWorks)
+
+	t.Cleanup(func() {
+		for _, spec := range specs {
+			t.Log("Cleaning up ", spec.ImageName)
+			if err := spec.Cleanup(); err != nil {
+				t.Error("Error removing ", spec.ImageName, "error:", err)
+			}
+		}
+	})
+}
+
+func test(t *testing.T) {
 	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
 		ip, port, err := c.FirstPort()
 		if err != nil {
@@ -72,18 +100,22 @@ func Test(t *testing.T) {
 		p := &Mongo{}
 		d, err := p.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
-		defer d.Close()
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
 		dt.TestNilVersion(t, d)
-		//TestLockAndUnlock(t, d) driver doesn't support lock on database level
+		dt.TestLockAndUnlock(t, d)
 		dt.TestRun(t, d, bytes.NewReader([]byte(`[{"insert":"hello","documents":[{"wild":"world"}]}]`)))
 		dt.TestSetVersion(t, d)
 		dt.TestDrop(t, d)
 	})
 }
 
-func TestWithAuth(t *testing.T) {
+func testMigrate(t *testing.T) {
 	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
 		ip, port, err := c.FirstPort()
 		if err != nil {
@@ -94,13 +126,43 @@ func TestWithAuth(t *testing.T) {
 		p := &Mongo{}
 		d, err := p.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
-		defer d.Close()
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+	})
+}
+
+func testWithAuth(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := mongoConnectionString(ip, port)
+		p := &Mongo{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
 		createUserCMD := []byte(`[{"createUser":"deminem","pwd":"gogo","roles":[{"role":"readWrite","db":"testMigration"}]}]`)
 		err = d.Run(bytes.NewReader(createUserCMD))
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		testcases := []struct {
 			name            string
@@ -110,20 +172,19 @@ func TestWithAuth(t *testing.T) {
 			{"right auth data", "mongodb://deminem:gogo@%s:%v/testMigration", false},
 			{"wrong auth data", "mongodb://wrong:auth@%s:%v/testMigration", true},
 		}
-		insertCMD := []byte(`[{"insert":"hello","documents":[{"wild":"world"}]}]`)
 
 		for _, tcase := range testcases {
-			//With wrong authenticate `Open` func doesn't return auth error
-			//Because at the moment golang mongo driver doesn't support auth during connection
-			//For getting auth error we should execute database command
 			t.Run(tcase.name, func(t *testing.T) {
 				mc := &Mongo{}
 				d, err := mc.Open(fmt.Sprintf(tcase.connectUri, ip, port))
-				if err != nil {
-					t.Fatalf("%v", err)
+				if err == nil {
+					defer func() {
+						if err := d.Close(); err != nil {
+							t.Error(err)
+						}
+					}()
 				}
-				defer d.Close()
-				err = d.Run(bytes.NewReader(insertCMD))
+
 				switch {
 				case tcase.isErrorExpected && err == nil:
 					t.Fatalf("no error when expected")
@@ -135,38 +196,110 @@ func TestWithAuth(t *testing.T) {
 	})
 }
 
+func testLockWorks(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.FirstPort()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := mongoConnectionString(ip, port)
+		p := &Mongo{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		dt.TestRun(t, d, bytes.NewReader([]byte(`[{"insert":"hello","documents":[{"wild":"world"}]}]`)))
+
+		mc := d.(*Mongo)
+
+		err = mc.Lock()
+		if err != nil {
+			t.Fatal(err)
+		}
+		err = mc.Unlock()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		err = mc.Lock()
+		if err != nil {
+			t.Fatal(err)
+		}
+		err = mc.Unlock()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		// enable locking,
+		//try to hit a lock conflict
+		mc.config.Locking.Enabled = true
+		mc.config.Locking.Timeout = 1
+		err = mc.Lock()
+		if err != nil {
+			t.Fatal(err)
+		}
+		err = mc.Lock()
+		if err == nil {
+			t.Fatal("should have failed, mongo should be locked already")
+		}
+	})
+}
+
 func TestTransaction(t *testing.T) {
 	transactionSpecs := []dktesting.ContainerSpec{
 		{ImageName: "mongo:4", Options: dktest.Options{PortRequired: true, ReadyFunc: isReady,
 			Cmd: []string{"mongod", "--bind_ip_all", "--replSet", "rs0"}}},
 	}
+	t.Cleanup(func() {
+		for _, spec := range transactionSpecs {
+			t.Log("Cleaning up ", spec.ImageName)
+			if err := spec.Cleanup(); err != nil {
+				t.Error("Error removing ", spec.ImageName, "error:", err)
+			}
+		}
+	})
+
 	dktesting.ParallelTest(t, transactionSpecs, func(t *testing.T, c dktest.ContainerInfo) {
 		ip, port, err := c.FirstPort()
 		if err != nil {
 			t.Fatal(err)
 		}
 
-		client, err := mongo.Connect(context.TODO(), mongoConnectionString(ip, port))
+		client, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(mongoConnectionString(ip, port)))
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		err = client.Ping(context.TODO(), nil)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		//rs.initiate()
-		err = client.Database("admin").RunCommand(context.TODO(), bson.D{{"replSetInitiate", bson.D{}}}).Err()
+		err = client.Database("admin").RunCommand(context.TODO(), bson.D{bson.E{Key: "replSetInitiate", Value: bson.D{}}}).Err()
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		err = waitForReplicaInit(client)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		d, err := WithInstance(client, &Config{
 			DatabaseName: "testMigration",
 		})
-		defer d.Close()
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
 		//We have to create collection
 		//transactions don't support operations with creating new dbs, collections
 		//Unique index need for checking transaction aborting
@@ -184,7 +317,7 @@ func TestTransaction(t *testing.T) {
 			}]`)
 		err = d.Run(bytes.NewReader(insertCMD))
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		testcases := []struct {
 			name            string
@@ -219,31 +352,35 @@ func TestTransaction(t *testing.T) {
 		}
 		for _, tcase := range testcases {
 			t.Run(tcase.name, func(t *testing.T) {
-				client, err := mongo.Connect(context.TODO(), mongoConnectionString(ip, port))
+				client, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(mongoConnectionString(ip, port)))
 				if err != nil {
-					t.Fatalf("%v", err)
+					t.Fatal(err)
 				}
 				err = client.Ping(context.TODO(), nil)
 				if err != nil {
-					t.Fatalf("%v", err)
+					t.Fatal(err)
 				}
 				d, err := WithInstance(client, &Config{
 					DatabaseName:    "testMigration",
 					TransactionMode: true,
 				})
 				if err != nil {
-					t.Fatalf("%v", err)
+					t.Fatal(err)
 				}
-				defer d.Close()
+				defer func() {
+					if err := d.Close(); err != nil {
+						t.Error(err)
+					}
+				}()
 				runErr := d.Run(bytes.NewReader(tcase.cmds))
 				if runErr != nil {
 					if !tcase.isErrorExpected {
-						t.Fatalf("%v", runErr)
+						t.Fatal(runErr)
 					}
 				}
-				documentsCount, err := client.Database("testMigration").Collection("hello").Count(context.TODO(), nil)
+				documentsCount, err := client.Database("testMigration").Collection("hello").CountDocuments(context.TODO(), bson.M{})
 				if err != nil {
-					t.Fatalf("%v", err)
+					t.Fatal(err)
 				}
 				if tcase.documentsCount != documentsCount {
 					t.Fatalf("expected %d and actual %d documents count not equal. run migration error:%s", tcase.documentsCount, documentsCount, runErr)
@@ -273,7 +410,7 @@ func waitForReplicaInit(client *mongo.Client) error {
 			//Check that node is primary because
 			//during replica set initialization, the first node first becomes a secondary and then becomes the primary
 			//should consider that initialization is completed only after the node has become the primary
-			result := client.Database("admin").RunCommand(context.TODO(), bson.D{{"isMaster", 1}})
+			result := client.Database("admin").RunCommand(context.TODO(), bson.D{bson.E{Key: "isMaster", Value: 1}})
 			r, err := result.DecodeBytes()
 			if err != nil {
 				return err

database/multistmt/parse.go

@@ -0,0 +1,46 @@
+// Package multistmt provides methods for parsing multi-statement database migrations
+package multistmt
+
+import (
+	"bufio"
+	"bytes"
+	"io"
+)
+
+// StartBufSize is the default starting size of the buffer used to scan and parse multi-statement migrations
+var StartBufSize = 4096
+
+// Handler handles a single migration parsed from a multi-statement migration.
+// It's given the single migration to handle and returns whether or not further statements
+// from the multi-statement migration should be parsed and handled.
+type Handler func(migration []byte) bool
+
+func splitWithDelimiter(delimiter []byte) func(d []byte, atEOF bool) (int, []byte, error) {
+	return func(d []byte, atEOF bool) (int, []byte, error) {
+		// SplitFunc inspired by bufio.ScanLines() implementation
+		if atEOF {
+			if len(d) == 0 {
+				return 0, nil, nil
+			}
+			return len(d), d, nil
+		}
+		if i := bytes.Index(d, delimiter); i >= 0 {
+			return i + len(delimiter), d[:i+len(delimiter)], nil
+		}
+		return 0, nil, nil
+	}
+}
+
+// Parse parses the given multi-statement migration
+func Parse(reader io.Reader, delimiter []byte, maxMigrationSize int, h Handler) error {
+	scanner := bufio.NewScanner(reader)
+	scanner.Buffer(make([]byte, 0, StartBufSize), maxMigrationSize)
+	scanner.Split(splitWithDelimiter(delimiter))
+	for scanner.Scan() {
+		cont := h(scanner.Bytes())
+		if !cont {
+			break
+		}
+	}
+	return scanner.Err()
+}

database/multistmt/parse_test.go

@@ -0,0 +1,57 @@
+package multistmt_test
+
+import (
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+
+	"github.com/golang-migrate/migrate/v4/database/multistmt"
+)
+
+const maxMigrationSize = 1024
+
+func TestParse(t *testing.T) {
+	testCases := []struct {
+		name        string
+		multiStmt   string
+		delimiter   string
+		expected    []string
+		expectedErr error
+	}{
+		{name: "single statement, no delimiter", multiStmt: "single statement, no delimiter", delimiter: ";",
+			expected: []string{"single statement, no delimiter"}, expectedErr: nil},
+		{name: "single statement, one delimiter", multiStmt: "single statement, one delimiter;", delimiter: ";",
+			expected: []string{"single statement, one delimiter;"}, expectedErr: nil},
+		{name: "two statements, no trailing delimiter", multiStmt: "statement one; statement two", delimiter: ";",
+			expected: []string{"statement one;", " statement two"}, expectedErr: nil},
+		{name: "two statements, with trailing delimiter", multiStmt: "statement one; statement two;", delimiter: ";",
+			expected: []string{"statement one;", " statement two;"}, expectedErr: nil},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			stmts := make([]string, 0, len(tc.expected))
+			err := multistmt.Parse(strings.NewReader(tc.multiStmt), []byte(tc.delimiter), maxMigrationSize, func(b []byte) bool {
+				stmts = append(stmts, string(b))
+				return true
+			})
+			assert.Equal(t, tc.expectedErr, err)
+			assert.Equal(t, tc.expected, stmts)
+		})
+	}
+}
+
+func TestParseDiscontinue(t *testing.T) {
+	multiStmt := "statement one; statement two"
+	delimiter := ";"
+	expected := []string{"statement one;"}
+
+	stmts := make([]string, 0, len(expected))
+	err := multistmt.Parse(strings.NewReader(multiStmt), []byte(delimiter), maxMigrationSize, func(b []byte) bool {
+		stmts = append(stmts, string(b))
+		return false
+	})
+	assert.Nil(t, err)
+	assert.Equal(t, expected, stmts)
+}

database/mysql/README.md

@@ -5,6 +5,8 @@
 | URL Query  | WithInstance Config | Description |
 |------------|---------------------|-------------|
 | `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
+| `x-no-lock` | `NoLock` | Set to `true` to skip `GET_LOCK`/`RELEASE_LOCK` statements. Useful for [multi-master MySQL flavors](https://www.percona.com/doc/percona-xtradb-cluster/LATEST/features/pxc-strict-mode.html#explicit-table-locking). Only run migrations from one host when this is enabled. |
+| `x-statement-timeout` | `StatementTimeout` | Abort any statement that takes more than the specified number of milliseconds, functionally similar to [Server-side SELECT statement timeouts](https://dev.mysql.com/blog-archive/server-side-select-statement-timeouts/) but enforced by the client. Available for all versions of MySQL, not just >=5.7. | 
 | `dbname` | `DatabaseName` | The name of the database to connect to |
 | `user` | | The user to sign in as |
 | `password` | | The user's password | 
@@ -12,7 +14,7 @@
 | `port` | | The port to bind to. |
 | `tls`  | | TLS / SSL encrypted connection parameter; see [go-sql-driver](https://github.com/go-sql-driver/mysql#tls). Use any name (e.g. `migrate`) if you want to use a custom TLS config (`x-tls-` queries). |
 | `x-tls-ca` | | The location of the CA (certificate authority) file. |
-| `x-tls-cert` | | The location of the client certicicate file. Must be used with `x-tls-key`. |
+| `x-tls-cert` | | The location of the client certificate file. Must be used with `x-tls-key`. |
 | `x-tls-key` | | The location of the private key file. Must be used with `x-tls-cert`. |
 | `x-tls-insecure-skip-verify` | | Whether or not to use SSL (true\|false) | 
 
@@ -27,9 +29,9 @@ import (
     "database/sql"
     
     _ "github.com/go-sql-driver/mysql"
-    "github.com/golang-migrate/migrate"
-    "github.com/golang-migrate/migrate/database/mysql"
-    _ "github.com/golang-migrate/migrate/source/file"
+    "github.com/golang-migrate/migrate/v4"
+    "github.com/golang-migrate/migrate/v4/database/mysql"
+    _ "github.com/golang-migrate/migrate/v4/source/file"
 )
 
 func main() {

database/mysql/examples/migrations/1_init.down.sql

@@ -0,0 +1 @@
+DROP TABLE IF EXISTS test;

database/mysql/examples/migrations/1_init.up.sql

@@ -0,0 +1,3 @@
+CREATE TABLE IF NOT EXISTS test (
+  firstname VARCHAR(16)
+);

database/mysql/mysql.go

@@ -1,4 +1,4 @@
-// +build go1.9
+//go:build go1.9
 
 package mysql
 
@@ -7,23 +7,22 @@ import (
 	"crypto/tls"
 	"crypto/x509"
 	"database/sql"
+	"errors"
 	"fmt"
 	"io"
-	"io/ioutil"
 	nurl "net/url"
+	"os"
 	"strconv"
 	"strings"
-)
+	"sync/atomic"
+	"time"
 
-import (
 	"github.com/go-sql-driver/mysql"
-)
-
-import (
-	"github.com/golang-migrate/migrate/v4"
 	"github.com/golang-migrate/migrate/v4/database"
 )
 
+var _ database.Driver = (*Mysql)(nil) // explicit compile time type check
+
 func init() {
 	database.Register("mysql", &Mysql{})
 }
@@ -35,12 +34,14 @@ var (
 	ErrNilConfig        = fmt.Errorf("no config")
 	ErrNoDatabaseName   = fmt.Errorf("no database name")
 	ErrAppendPEM        = fmt.Errorf("failed to append PEM")
-	ErrTLSCertKeyConfig = fmt.Errorf("To use TLS client authentication, both x-tls-cert and x-tls-key must not be empty")
+	ErrTLSCertKeyConfig = fmt.Errorf("to use TLS client authentication, both x-tls-cert and x-tls-key must not be empty")
 )
 
 type Config struct {
-	MigrationsTable string
-	DatabaseName    string
+	MigrationsTable  string
+	DatabaseName     string
+	NoLock           bool
+	StatementTimeout time.Duration
 }
 
 type Mysql struct {
@@ -48,48 +49,45 @@ type Mysql struct {
 	// just do everything over a single conn anyway.
 	conn     *sql.Conn
 	db       *sql.DB
-	isLocked bool
+	isLocked atomic.Bool
 
 	config *Config
 }
 
-// instance must have `multiStatements` set to true
-func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
+// connection instance must have `multiStatements` set to true
+func WithConnection(ctx context.Context, conn *sql.Conn, config *Config) (*Mysql, error) {
 	if config == nil {
 		return nil, ErrNilConfig
 	}
 
-	if err := instance.Ping(); err != nil {
-		return nil, err
-	}
-
-	query := `SELECT DATABASE()`
-	var databaseName sql.NullString
-	if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
-		return nil, &database.Error{OrigErr: err, Query: []byte(query)}
-	}
-
-	if len(databaseName.String) == 0 {
-		return nil, ErrNoDatabaseName
-	}
-
-	config.DatabaseName = databaseName.String
-
-	if len(config.MigrationsTable) == 0 {
-		config.MigrationsTable = DefaultMigrationsTable
-	}
-
-	conn, err := instance.Conn(context.Background())
-	if err != nil {
+	if err := conn.PingContext(ctx); err != nil {
 		return nil, err
 	}
 
 	mx := &Mysql{
 		conn:   conn,
-		db:     instance,
+		db:     nil,
 		config: config,
 	}
 
+	if config.DatabaseName == "" {
+		query := `SELECT DATABASE()`
+		var databaseName sql.NullString
+		if err := conn.QueryRowContext(ctx, query).Scan(&databaseName); err != nil {
+			return nil, &database.Error{OrigErr: err, Query: []byte(query)}
+		}
+
+		if len(databaseName.String) == 0 {
+			return nil, ErrNoDatabaseName
+		}
+
+		config.DatabaseName = databaseName.String
+	}
+
+	if len(config.MigrationsTable) == 0 {
+		config.MigrationsTable = DefaultMigrationsTable
+	}
+
 	if err := mx.ensureVersionTable(); err != nil {
 		return nil, err
 	}
@@ -97,95 +95,172 @@ func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
 	return mx, nil
 }
 
-// urlToMySQLConfig takes a net/url URL and returns a go-sql-driver/mysql Config.
-// Manually sets username and password to avoid net/url from url-encoding the reserved URL characters
-func urlToMySQLConfig(u nurl.URL) (*mysql.Config, error) {
-	origUserInfo := u.User
-	u.User = nil
+// instance must have `multiStatements` set to true
+func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
+	ctx := context.Background()
 
-	c, err := mysql.ParseDSN(strings.TrimPrefix(u.String(), "mysql://"))
+	if err := instance.Ping(); err != nil {
+		return nil, err
+	}
+
+	conn, err := instance.Conn(ctx)
 	if err != nil {
 		return nil, err
 	}
-	if origUserInfo != nil {
-		c.User = origUserInfo.Username()
-		if p, ok := origUserInfo.Password(); ok {
-			c.Passwd = p
+
+	mx, err := WithConnection(ctx, conn, config)
+	if err != nil {
+		return nil, err
+	}
+
+	mx.db = instance
+
+	return mx, nil
+}
+
+// extractCustomQueryParams extracts the custom query params (ones that start with "x-") from
+// mysql.Config.Params (connection parameters) as to not interfere with connecting to MySQL
+func extractCustomQueryParams(c *mysql.Config) (map[string]string, error) {
+	if c == nil {
+		return nil, ErrNilConfig
+	}
+	customQueryParams := map[string]string{}
+
+	for k, v := range c.Params {
+		if strings.HasPrefix(k, "x-") {
+			customQueryParams[k] = v
+			delete(c.Params, k)
 		}
 	}
-	return c, nil
+	return customQueryParams, nil
+}
+
+func urlToMySQLConfig(url string) (*mysql.Config, error) {
+	// Need to parse out custom TLS parameters and call
+	// mysql.RegisterTLSConfig() before mysql.ParseDSN() is called
+	// which consumes the registered tls.Config
+	// Fixes: https://github.com/golang-migrate/migrate/issues/411
+	//
+	// Can't use url.Parse() since it fails to parse MySQL DSNs
+	// mysql.ParseDSN() also searches for "?" to find query parameters:
+	// https://github.com/go-sql-driver/mysql/blob/46351a8/dsn.go#L344
+	if idx := strings.LastIndex(url, "?"); idx > 0 {
+		rawParams := url[idx+1:]
+		parsedParams, err := nurl.ParseQuery(rawParams)
+		if err != nil {
+			return nil, err
+		}
+
+		ctls := parsedParams.Get("tls")
+		if len(ctls) > 0 {
+			if _, isBool := readBool(ctls); !isBool && strings.ToLower(ctls) != "skip-verify" {
+				rootCertPool := x509.NewCertPool()
+				pem, err := os.ReadFile(parsedParams.Get("x-tls-ca"))
+				if err != nil {
+					return nil, err
+				}
+
+				if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
+					return nil, ErrAppendPEM
+				}
+
+				clientCert := make([]tls.Certificate, 0, 1)
+				if ccert, ckey := parsedParams.Get("x-tls-cert"), parsedParams.Get("x-tls-key"); ccert != "" || ckey != "" {
+					if ccert == "" || ckey == "" {
+						return nil, ErrTLSCertKeyConfig
+					}
+					certs, err := tls.LoadX509KeyPair(ccert, ckey)
+					if err != nil {
+						return nil, err
+					}
+					clientCert = append(clientCert, certs)
+				}
+
+				insecureSkipVerify := false
+				insecureSkipVerifyStr := parsedParams.Get("x-tls-insecure-skip-verify")
+				if len(insecureSkipVerifyStr) > 0 {
+					x, err := strconv.ParseBool(insecureSkipVerifyStr)
+					if err != nil {
+						return nil, err
+					}
+					insecureSkipVerify = x
+				}
+
+				err = mysql.RegisterTLSConfig(ctls, &tls.Config{
+					RootCAs:            rootCertPool,
+					Certificates:       clientCert,
+					InsecureSkipVerify: insecureSkipVerify,
+				})
+				if err != nil {
+					return nil, err
+				}
+			}
+		}
+	}
+
+	config, err := mysql.ParseDSN(strings.TrimPrefix(url, "mysql://"))
+	if err != nil {
+		return nil, err
+	}
+
+	config.MultiStatements = true
+
+	// Keep backwards compatibility from when we used net/url.Parse() to parse the DSN.
+	// net/url.Parse() would automatically unescape it for us.
+	// See: https://play.golang.org/p/q9j1io-YICQ
+	user, err := nurl.QueryUnescape(config.User)
+	if err != nil {
+		return nil, err
+	}
+	config.User = user
+
+	password, err := nurl.QueryUnescape(config.Passwd)
+	if err != nil {
+		return nil, err
+	}
+	config.Passwd = password
+
+	return config, nil
 }
 
 func (m *Mysql) Open(url string) (database.Driver, error) {
-	purl, err := nurl.Parse(url)
+	config, err := urlToMySQLConfig(url)
 	if err != nil {
 		return nil, err
 	}
 
-	q := purl.Query()
-	q.Set("multiStatements", "true")
-	purl.RawQuery = q.Encode()
-
-	migrationsTable := purl.Query().Get("x-migrations-table")
-	if len(migrationsTable) == 0 {
-		migrationsTable = DefaultMigrationsTable
+	customParams, err := extractCustomQueryParams(config)
+	if err != nil {
+		return nil, err
 	}
 
-	// use custom TLS?
-	ctls := purl.Query().Get("tls")
-	if len(ctls) > 0 {
-		if _, isBool := readBool(ctls); !isBool && strings.ToLower(ctls) != "skip-verify" {
-			rootCertPool := x509.NewCertPool()
-			pem, err := ioutil.ReadFile(purl.Query().Get("x-tls-ca"))
-			if err != nil {
-				return nil, err
-			}
-
-			if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
-				return nil, ErrAppendPEM
-			}
-
-			clientCert := make([]tls.Certificate, 0, 1)
-			if ccert, ckey := purl.Query().Get("x-tls-cert"), purl.Query().Get("x-tls-key"); ccert != "" || ckey != "" {
-				if ccert == "" || ckey == "" {
-					return nil, ErrTLSCertKeyConfig
-				}
-				certs, err := tls.LoadX509KeyPair(ccert, ckey)
-				if err != nil {
-					return nil, err
-				}
-				clientCert = append(clientCert, certs)
-			}
-
-			insecureSkipVerify := false
-			if len(purl.Query().Get("x-tls-insecure-skip-verify")) > 0 {
-				x, err := strconv.ParseBool(purl.Query().Get("x-tls-insecure-skip-verify"))
-				if err != nil {
-					return nil, err
-				}
-				insecureSkipVerify = x
-			}
-
-			mysql.RegisterTLSConfig(ctls, &tls.Config{
-				RootCAs:            rootCertPool,
-				Certificates:       clientCert,
-				InsecureSkipVerify: insecureSkipVerify,
-			})
+	noLockParam, noLock := customParams["x-no-lock"], false
+	if noLockParam != "" {
+		noLock, err = strconv.ParseBool(noLockParam)
+		if err != nil {
+			return nil, fmt.Errorf("could not parse x-no-lock as bool: %w", err)
 		}
 	}
 
-	c, err := urlToMySQLConfig(*migrate.FilterCustomQuery(purl))
-	if err != nil {
-		return nil, err
+	statementTimeoutParam := customParams["x-statement-timeout"]
+	statementTimeout := 0
+	if statementTimeoutParam != "" {
+		statementTimeout, err = strconv.Atoi(statementTimeoutParam)
+		if err != nil {
+			return nil, fmt.Errorf("could not parse x-statement-timeout as float: %w", err)
+		}
 	}
-	db, err := sql.Open("mysql", c.FormatDSN())
+
+	db, err := sql.Open("mysql", config.FormatDSN())
 	if err != nil {
 		return nil, err
 	}
 
 	mx, err := WithInstance(db, &Config{
-		DatabaseName:    purl.Path,
-		MigrationsTable: migrationsTable,
+		DatabaseName:     config.DBName,
+		MigrationsTable:  customParams["x-migrations-table"],
+		NoLock:           noLock,
+		StatementTimeout: time.Duration(statementTimeout) * time.Millisecond,
 	})
 	if err != nil {
 		return nil, err
@@ -196,7 +271,11 @@ func (m *Mysql) Open(url string) (database.Driver, error) {
 
 func (m *Mysql) Close() error {
 	connErr := m.conn.Close()
-	dbErr := m.db.Close()
+	var dbErr error
+	if m.db != nil {
+		dbErr = m.db.Close()
+	}
+
 	if connErr != nil || dbErr != nil {
 		return fmt.Errorf("conn: %v, db: %v", connErr, dbErr)
 	}
@@ -204,62 +283,70 @@ func (m *Mysql) Close() error {
 }
 
 func (m *Mysql) Lock() error {
-	if m.isLocked {
-		return database.ErrLocked
-	}
+	return database.CasRestoreOnErr(&m.isLocked, false, true, database.ErrLocked, func() error {
+		if m.config.NoLock {
+			return nil
+		}
+		aid, err := database.GenerateAdvisoryLockId(
+			fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable))
+		if err != nil {
+			return err
+		}
 
-	aid, err := database.GenerateAdvisoryLockId(
-		fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable))
-	if err != nil {
-		return err
-	}
+		query := "SELECT GET_LOCK(?, 10)"
+		var success bool
+		if err := m.conn.QueryRowContext(context.Background(), query, aid).Scan(&success); err != nil {
+			return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
+		}
 
-	query := "SELECT GET_LOCK(?, 10)"
-	var success bool
-	if err := m.conn.QueryRowContext(context.Background(), query, aid).Scan(&success); err != nil {
-		return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
-	}
+		if !success {
+			return database.ErrLocked
+		}
 
-	if success {
-		m.isLocked = true
 		return nil
-	}
-
-	return database.ErrLocked
+	})
 }
 
 func (m *Mysql) Unlock() error {
-	if !m.isLocked {
+	return database.CasRestoreOnErr(&m.isLocked, true, false, database.ErrNotLocked, func() error {
+		if m.config.NoLock {
+			return nil
+		}
+
+		aid, err := database.GenerateAdvisoryLockId(
+			fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable))
+		if err != nil {
+			return err
+		}
+
+		query := `SELECT RELEASE_LOCK(?)`
+		if _, err := m.conn.ExecContext(context.Background(), query, aid); err != nil {
+			return &database.Error{OrigErr: err, Query: []byte(query)}
+		}
+
+		// NOTE: RELEASE_LOCK could return NULL or (or 0 if the code is changed),
+		// in which case isLocked should be true until the timeout expires -- synchronizing
+		// these states is likely not worth trying to do; reconsider the necessity of isLocked.
+
 		return nil
-	}
-
-	aid, err := database.GenerateAdvisoryLockId(
-		fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable))
-	if err != nil {
-		return err
-	}
-
-	query := `SELECT RELEASE_LOCK(?)`
-	if _, err := m.conn.ExecContext(context.Background(), query, aid); err != nil {
-		return &database.Error{OrigErr: err, Query: []byte(query)}
-	}
-
-	// NOTE: RELEASE_LOCK could return NULL or (or 0 if the code is changed),
-	// in which case isLocked should be true until the timeout expires -- synchronizing
-	// these states is likely not worth trying to do; reconsider the necessity of isLocked.
-
-	m.isLocked = false
-	return nil
+	})
 }
 
 func (m *Mysql) Run(migration io.Reader) error {
-	migr, err := ioutil.ReadAll(migration)
+	migr, err := io.ReadAll(migration)
 	if err != nil {
 		return err
 	}
 
+	ctx := context.Background()
+	if m.config.StatementTimeout != 0 {
+		var cancel context.CancelFunc
+		ctx, cancel = context.WithTimeout(ctx, m.config.StatementTimeout)
+		defer cancel()
+	}
+
 	query := string(migr[:])
-	if _, err := m.conn.ExecContext(context.Background(), query); err != nil {
+	if _, err := m.conn.ExecContext(ctx, query); err != nil {
 		return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
 	}
 
@@ -267,21 +354,28 @@ func (m *Mysql) Run(migration io.Reader) error {
 }
 
 func (m *Mysql) SetVersion(version int, dirty bool) error {
-	tx, err := m.conn.BeginTx(context.Background(), &sql.TxOptions{})
+	tx, err := m.conn.BeginTx(context.Background(), &sql.TxOptions{Isolation: sql.LevelSerializable})
 	if err != nil {
 		return &database.Error{OrigErr: err, Err: "transaction start failed"}
 	}
 
-	query := "TRUNCATE `" + m.config.MigrationsTable + "`"
+	query := "DELETE FROM `" + m.config.MigrationsTable + "` LIMIT 1"
 	if _, err := tx.ExecContext(context.Background(), query); err != nil {
-		tx.Rollback()
+		if errRollback := tx.Rollback(); errRollback != nil {
+			err = errors.Join(err, errRollback)
+		}
 		return &database.Error{OrigErr: err, Query: []byte(query)}
 	}
 
-	if version >= 0 {
+	// Also re-write the schema version for nil dirty versions to prevent
+	// empty schema version for failed down migration on the first migration
+	// See: https://github.com/golang-migrate/migrate/issues/330
+	if version >= 0 || (version == database.NilVersion && dirty) {
 		query := "INSERT INTO `" + m.config.MigrationsTable + "` (version, dirty) VALUES (?, ?)"
 		if _, err := tx.ExecContext(context.Background(), query, version, dirty); err != nil {
-			tx.Rollback()
+			if errRollback := tx.Rollback(); errRollback != nil {
+				err = errors.Join(err, errRollback)
+			}
 			return &database.Error{OrigErr: err, Query: []byte(query)}
 		}
 	}
@@ -313,14 +407,18 @@ func (m *Mysql) Version() (version int, dirty bool, err error) {
 	}
 }
 
-func (m *Mysql) Drop() error {
+func (m *Mysql) Drop() (err error) {
 	// select all tables
 	query := `SHOW TABLES LIKE '%'`
 	tables, err := m.conn.QueryContext(context.Background(), query)
 	if err != nil {
 		return &database.Error{OrigErr: err, Query: []byte(query)}
 	}
-	defer tables.Close()
+	defer func() {
+		if errClose := tables.Close(); errClose != nil {
+			err = errors.Join(err, errClose)
+		}
+	}()
 
 	// delete one table after another
 	tableNames := make([]string, 0)
@@ -333,27 +431,51 @@ func (m *Mysql) Drop() error {
 			tableNames = append(tableNames, tableName)
 		}
 	}
+	if err := tables.Err(); err != nil {
+		return &database.Error{OrigErr: err, Query: []byte(query)}
+	}
 
 	if len(tableNames) > 0 {
+		// disable checking foreign key constraints until finished
+		query = `SET foreign_key_checks = 0`
+		if _, err := m.conn.ExecContext(context.Background(), query); err != nil {
+			return &database.Error{OrigErr: err, Query: []byte(query)}
+		}
+
+		defer func() {
+			// enable foreign key checks
+			_, _ = m.conn.ExecContext(context.Background(), `SET foreign_key_checks = 1`)
+		}()
+
 		// delete one by one ...
 		for _, t := range tableNames {
-			query = "DROP TABLE IF EXISTS `" + t + "` CASCADE"
+			query = "DROP TABLE IF EXISTS `" + t + "`"
 			if _, err := m.conn.ExecContext(context.Background(), query); err != nil {
 				return &database.Error{OrigErr: err, Query: []byte(query)}
 			}
 		}
-		if err := m.ensureVersionTable(); err != nil {
-			return err
-		}
 	}
 
 	return nil
 }
 
-func (m *Mysql) ensureVersionTable() error {
+// ensureVersionTable checks if versions table exists and, if not, creates it.
+// Note that this function locks the database, which deviates from the usual
+// convention of "caller locks" in the Mysql type.
+func (m *Mysql) ensureVersionTable() (err error) {
+	if err = m.Lock(); err != nil {
+		return err
+	}
+
+	defer func() {
+		if e := m.Unlock(); e != nil {
+			err = errors.Join(err, e)
+		}
+	}()
+
 	// check if migration table exists
 	var result string
-	query := `SHOW TABLES LIKE "` + m.config.MigrationsTable + `"`
+	query := `SHOW TABLES LIKE '` + m.config.MigrationsTable + `'`
 	if err := m.conn.QueryRowContext(context.Background(), query).Scan(&result); err != nil {
 		if err != sql.ErrNoRows {
 			return &database.Error{OrigErr: err, Query: []byte(query)}

database/mysql/mysql_test.go

@@ -2,21 +2,28 @@ package mysql
 
 import (
 	"context"
+	"crypto/ed25519"
+	"crypto/x509"
 	"database/sql"
 	sqldriver "database/sql/driver"
+	"encoding/pem"
+	"errors"
 	"fmt"
+	"log"
+	"math/big"
+	"math/rand"
 	"net/url"
+	"os"
+	"strconv"
 	"testing"
-)
 
-import (
 	"github.com/dhui/dktest"
 	"github.com/go-sql-driver/mysql"
-)
-
-import (
+	"github.com/golang-migrate/migrate/v4"
 	dt "github.com/golang-migrate/migrate/v4/database/testing"
 	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+	"github.com/stretchr/testify/assert"
 )
 
 const defaultPort = 3306
@@ -26,12 +33,21 @@ var (
 		Env:          map[string]string{"MYSQL_ROOT_PASSWORD": "root", "MYSQL_DATABASE": "public"},
 		PortRequired: true, ReadyFunc: isReady,
 	}
+	optsAnsiQuotes = dktest.Options{
+		Env:          map[string]string{"MYSQL_ROOT_PASSWORD": "root", "MYSQL_DATABASE": "public"},
+		PortRequired: true, ReadyFunc: isReady,
+		Cmd: []string{"--sql-mode=ANSI_QUOTES"},
+	}
 	// Supported versions: https://www.mysql.com/support/supportedplatforms/database.html
 	specs = []dktesting.ContainerSpec{
-		{ImageName: "mysql:5.5", Options: opts},
-		{ImageName: "mysql:5.6", Options: opts},
-		{ImageName: "mysql:5.7", Options: opts},
-		{ImageName: "mysql:8", Options: opts},
+		{ImageName: "mysql:8.0", Options: opts},
+		{ImageName: "mysql:8.4", Options: opts},
+		{ImageName: "mysql:9.0", Options: opts},
+	}
+	specsAnsiQuotes = []dktesting.ContainerSpec{
+		{ImageName: "mysql:8.0", Options: optsAnsiQuotes},
+		{ImageName: "mysql:8.4", Options: optsAnsiQuotes},
+		{ImageName: "mysql:9.0", Options: optsAnsiQuotes},
 	}
 )
 
@@ -45,7 +61,11 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
 	if err != nil {
 		return false
 	}
-	defer db.Close()
+	defer func() {
+		if err := db.Close(); err != nil {
+			log.Println("close error:", err)
+		}
+	}()
 	if err = db.PingContext(ctx); err != nil {
 		switch err {
 		case sqldriver.ErrBadConn, mysql.ErrInvalidConn:
@@ -60,7 +80,7 @@ func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
 }
 
 func Test(t *testing.T) {
-	// mysql.SetLogger(mysql.Logger(log.New(ioutil.Discard, "", log.Ltime)))
+	// mysql.SetLogger(mysql.Logger(log.New(io.Discard, "", log.Ltime)))
 
 	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
 		ip, port, err := c.Port(defaultPort)
@@ -72,9 +92,13 @@ func Test(t *testing.T) {
 		p := &Mysql{}
 		d, err := p.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
-		defer d.Close()
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
 		dt.Test(t, d, []byte("SELECT 1"))
 
 		// check ensureVersionTable
@@ -88,6 +112,82 @@ func Test(t *testing.T) {
 	})
 }
 
+func TestMigrate(t *testing.T) {
+	// mysql.SetLogger(mysql.Logger(log.New(io.Discard, "", log.Ltime)))
+
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fmt.Sprintf("mysql://root:root@tcp(%v:%v)/public", ip, port)
+		p := &Mysql{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "public", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+
+		// check ensureVersionTable
+		if err := d.(*Mysql).ensureVersionTable(); err != nil {
+			t.Fatal(err)
+		}
+		// check again
+		if err := d.(*Mysql).ensureVersionTable(); err != nil {
+			t.Fatal(err)
+		}
+	})
+}
+
+func TestMigrateAnsiQuotes(t *testing.T) {
+	// mysql.SetLogger(mysql.Logger(log.New(io.Discard, "", log.Ltime)))
+
+	dktesting.ParallelTest(t, specsAnsiQuotes, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fmt.Sprintf("mysql://root:root@tcp(%v:%v)/public", ip, port)
+		p := &Mysql{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "public", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+
+		// check ensureVersionTable
+		if err := d.(*Mysql).ensureVersionTable(); err != nil {
+			t.Fatal(err)
+		}
+		// check again
+		if err := d.(*Mysql).ensureVersionTable(); err != nil {
+			t.Fatal(err)
+		}
+	})
+}
+
 func TestLockWorks(t *testing.T) {
 	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
 		ip, port, err := c.Port(defaultPort)
@@ -99,7 +199,7 @@ func TestLockWorks(t *testing.T) {
 		p := &Mysql{}
 		d, err := p.Open(addr)
 		if err != nil {
-			t.Fatalf("%v", err)
+			t.Fatal(err)
 		}
 		dt.Test(t, d, []byte("SELECT 1"))
 
@@ -126,7 +226,149 @@ func TestLockWorks(t *testing.T) {
 	})
 }
 
+func TestNoLockParamValidation(t *testing.T) {
+	ip := "127.0.0.1"
+	port := 3306
+	addr := fmt.Sprintf("mysql://root:root@tcp(%v:%v)/public", ip, port)
+	p := &Mysql{}
+	_, err := p.Open(addr + "?x-no-lock=not-a-bool")
+	if !errors.Is(err, strconv.ErrSyntax) {
+		t.Fatal("Expected syntax error when passing a non-bool as x-no-lock parameter")
+	}
+}
+
+func TestNoLockWorks(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(defaultPort)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		addr := fmt.Sprintf("mysql://root:root@tcp(%v:%v)/public", ip, port)
+		p := &Mysql{}
+		d, err := p.Open(addr)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		lock := d.(*Mysql)
+
+		p = &Mysql{}
+		d, err = p.Open(addr + "?x-no-lock=true")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		noLock := d.(*Mysql)
+
+		// Should be possible to take real lock and no-lock at the same time
+		if err = lock.Lock(); err != nil {
+			t.Fatal(err)
+		}
+		if err = noLock.Lock(); err != nil {
+			t.Fatal(err)
+		}
+		if err = lock.Unlock(); err != nil {
+			t.Fatal(err)
+		}
+		if err = noLock.Unlock(); err != nil {
+			t.Fatal(err)
+		}
+	})
+}
+
+func TestExtractCustomQueryParams(t *testing.T) {
+	testcases := []struct {
+		name                 string
+		config               *mysql.Config
+		expectedParams       map[string]string
+		expectedCustomParams map[string]string
+		expectedErr          error
+	}{
+		{name: "nil config", expectedErr: ErrNilConfig},
+		{
+			name:                 "no params",
+			config:               mysql.NewConfig(),
+			expectedCustomParams: map[string]string{},
+		},
+		{
+			name:                 "no custom params",
+			config:               &mysql.Config{Params: map[string]string{"hello": "world"}},
+			expectedParams:       map[string]string{"hello": "world"},
+			expectedCustomParams: map[string]string{},
+		},
+		{
+			name: "one param, one custom param",
+			config: &mysql.Config{
+				Params: map[string]string{"hello": "world", "x-foo": "bar"},
+			},
+			expectedParams:       map[string]string{"hello": "world"},
+			expectedCustomParams: map[string]string{"x-foo": "bar"},
+		},
+		{
+			name: "multiple params, multiple custom params",
+			config: &mysql.Config{
+				Params: map[string]string{
+					"hello": "world",
+					"x-foo": "bar",
+					"dead":  "beef",
+					"x-cat": "hat",
+				},
+			},
+			expectedParams:       map[string]string{"hello": "world", "dead": "beef"},
+			expectedCustomParams: map[string]string{"x-foo": "bar", "x-cat": "hat"},
+		},
+	}
+	for _, tc := range testcases {
+		t.Run(tc.name, func(t *testing.T) {
+			customParams, err := extractCustomQueryParams(tc.config)
+			if tc.config != nil {
+				assert.Equal(t, tc.expectedParams, tc.config.Params,
+					"Expected config params have custom params properly removed")
+			}
+			assert.Equal(t, tc.expectedErr, err, "Expected errors to match")
+			assert.Equal(t, tc.expectedCustomParams, customParams,
+				"Expected custom params to be properly extracted")
+		})
+	}
+}
+
+func createTmpCert(t *testing.T) string {
+	tmpCertFile, err := os.CreateTemp("", "migrate_test_cert")
+	if err != nil {
+		t.Fatal("Failed to create temp cert file:", err)
+	}
+	t.Cleanup(func() {
+		if err := os.Remove(tmpCertFile.Name()); err != nil {
+			t.Log("Failed to cleanup temp cert file:", err)
+		}
+	})
+
+	r := rand.New(rand.NewSource(0))
+	pub, priv, err := ed25519.GenerateKey(r)
+	if err != nil {
+		t.Fatal("Failed to generate ed25519 key for temp cert file:", err)
+	}
+	tmpl := x509.Certificate{
+		SerialNumber: big.NewInt(0),
+	}
+	derBytes, err := x509.CreateCertificate(r, &tmpl, &tmpl, pub, priv)
+	if err != nil {
+		t.Fatal("Failed to generate temp cert file:", err)
+	}
+	if err := pem.Encode(tmpCertFile, &pem.Block{Type: "CERTIFICATE", Bytes: derBytes}); err != nil {
+		t.Fatal("Failed to encode ")
+	}
+	if err := tmpCertFile.Close(); err != nil {
+		t.Fatal("Failed to close temp cert file:", err)
+	}
+	return tmpCertFile.Name()
+}
+
 func TestURLToMySQLConfig(t *testing.T) {
+	tmpCertFilename := createTmpCert(t)
+	tmpCertFilenameEscaped := url.PathEscape(tmpCertFilename)
+
 	testcases := []struct {
 		name        string
 		urlStr      string
@@ -147,7 +389,7 @@ func TestURLToMySQLConfig(t *testing.T) {
 		// Not supported yet: https://github.com/go-sql-driver/mysql/issues/591
 		// {name: "user/password - user with encoded :",
 		// 	urlStr:      "mysql://username%3A:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
-		// 	expectedDSN: "username::pasword@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
+		// 	expectedDSN: "username::password@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
 		{name: "user/password - user with encoded @",
 			urlStr:      "mysql://username%40:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
 			expectedDSN: "username@:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
@@ -157,22 +399,19 @@ func TestURLToMySQLConfig(t *testing.T) {
 		{name: "user/password - password with encoded @",
 			urlStr:      "mysql://username:password%40@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
 			expectedDSN: "username:password@@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
+		{name: "custom tls",
+			urlStr:      "mysql://username:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true&tls=custom&x-tls-ca=" + tmpCertFilenameEscaped,
+			expectedDSN: "username:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true&tls=custom&x-tls-ca=" + tmpCertFilenameEscaped},
 	}
 	for _, tc := range testcases {
 		t.Run(tc.name, func(t *testing.T) {
-			u, err := url.Parse(tc.urlStr)
+			config, err := urlToMySQLConfig(tc.urlStr)
 			if err != nil {
 				t.Fatal("Failed to parse url string:", tc.urlStr, "error:", err)
 			}
-			if config, err := urlToMySQLConfig(*u); err == nil {
-				dsn := config.FormatDSN()
-				if dsn != tc.expectedDSN {
-					t.Error("Got unexpected DSN:", dsn, "!=", tc.expectedDSN)
-				}
-			} else {
-				if tc.expectedDSN != "" {
-					t.Error("Got unexpected error:", err, "urlStr:", tc.urlStr)
-				}
+			dsn := config.FormatDSN()
+			if dsn != tc.expectedDSN {
+				t.Error("Got unexpected DSN:", dsn, "!=", tc.expectedDSN)
 			}
 		})
 	}

database/neo4j/README.md

@@ -0,0 +1,20 @@
+# neo4j
+The Neo4j driver (bolt) does not natively support executing multiple statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. 
+This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
+The queries **should** run in a single transaction, so partial migrations should not be a concern, but this is untested.
+
+
+`neo4j://user:password@host:port/`
+
+| URL Query  | WithInstance Config | Description |
+|------------|---------------------|-------------|
+| `x-multi-statement` | `MultiStatement` | Enable multiple statements to be ran in a single migration (See note above) |
+| `user` | Contained within `AuthConfig` | The user to sign in as |
+| `password` | Contained within `AuthConfig` | The user's password |
+| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
+| `port` | | The port to bind to. (default is 7687) |
+|  | `MigrationsLabel` | Name of the migrations node label |
+
+## Supported versions
+
+Only Neo4j v3.5+ is [supported](https://github.com/neo4j/neo4j-go-driver/issues/64#issuecomment-625133600)

database/neo4j/TUTORIAL.md

@@ -0,0 +1,97 @@
+## Create migrations
+Let's create nodes called `Users`:
+```
+migrate create -ext cypher -dir db/migrations -seq create_user_nodes
+```
+If there were no errors, we should have two files available under `db/migrations` folder:
+- 000001_create_user_nodes.down.cypher
+- 000001_create_user_nodes.up.cypher
+
+Note the `cypher` extension that we provided.
+
+In the `.up.cypher` file let's create the table:
+```
+CREATE (u1:User {name: "Peter"})
+CREATE (u2:User {name: "Paul"})
+CREATE (u3:User {name: "Mary"})
+```
+And in the `.down.sql` let's delete it:
+```
+MATCH (u:User) WHERE u.name IN ["Peter", "Paul", "Mary"] DELETE u
+```
+Ideally your migrations should be idempotent. You can read more about idempotency in [getting started](GETTING_STARTED.md#create-migrations)
+
+## Run migrations
+```
+migrate -database ${NEO4J_URL} -path db/migrations up
+```
+Let's check if the table was created properly by running `bin/cypher-shell -u neo4j -p password`, then `neo4j> MATCH (u:User)`
+The output you are supposed to see:
+```
++-----------------------------------------------------------------+
+| u                                                               |
++-----------------------------------------------------------------+
+| (:User {name: "Peter")                                          |
+| (:User {name: "Paul")                                           |
+| (:User {name: "Mary")                                           |
++-----------------------------------------------------------------+
+```
+Great! Now let's check if running reverse migration also works:
+```
+migrate -database ${NEO4J_URL} -path db/migrations down
+```
+Make sure to check if your database changed as expected in this case as well.
+
+## Database transactions
+
+To show database transactions usage, let's create another set of migrations by running:
+```
+migrate create -ext cypher -dir db/migrations -seq add_mood_to_users
+```
+Again, it should create for us two migrations files:
+- 000002_add_mood_to_users.down.cypher
+- 000002_add_mood_to_users.up.cypher
+
+In Neo4j, when we want our queries to be done in a transaction, we need to wrap it with `:BEGIN` and `:COMMIT` commands.
+Migration up:
+```
+:BEGIN
+
+MATCH (u:User)
+SET u.mood = "Cheery"
+
+:COMMIT
+```
+Migration down:
+```
+:BEGIN
+
+MATCH (u:User)
+SET u.mood = null
+
+:COMMIT
+```
+
+## Optional: Run migrations within your Go app
+Here is a very simple app running migrations for the above configuration:
+```
+import (
+	"log"
+
+	"github.com/golang-migrate/migrate/v4"
+	_ "github.com/golang-migrate/migrate/v4/database/neo4j"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+)
+
+func main() {
+	m, err := migrate.New(
+		"file://db/migrations",
+		"neo4j://neo4j:password@localhost:7687/")
+	if err != nil {
+		log.Fatal(err)
+	}
+	if err := m.Up(); err != nil {
+		log.Fatal(err)
+	}
+}
+```

database/neo4j/examples/migrations/1578421040_create_movies_constraint.down.cypher

@@ -0,0 +1 @@
+DROP CONSTRAINT ON (m:Movie) ASSERT m.Name IS UNIQUE

database/neo4j/examples/migrations/1578421040_create_movies_constraint.up.cypher

@@ -0,0 +1 @@
+CREATE CONSTRAINT ON (m:Movie) ASSERT m.Name IS UNIQUE

database/neo4j/examples/migrations/1578421725_create_movies.down.cypher

@@ -0,0 +1,2 @@
+MATCH (m:Movie)
+DELETE m

database/neo4j/examples/migrations/1578421725_create_movies.up.cypher

@@ -0,0 +1,2 @@
+CREATE (:Movie {name: "Footloose"})
+CREATE (:Movie {name: "Ghost"})

database/neo4j/examples/migrations/1578421726_multistatement_test.up.cypher

@@ -0,0 +1,3 @@
+CREATE (:Movie {name: "Hollow Man"});
+CREATE (:Movie {name: "Mystic River"});
+;;;

database/neo4j/neo4j.go

@@ -0,0 +1,303 @@
+package neo4j
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"io"
+	neturl "net/url"
+	"strconv"
+	"sync/atomic"
+
+	"github.com/golang-migrate/migrate/v4/database"
+	"github.com/golang-migrate/migrate/v4/database/multistmt"
+	"github.com/neo4j/neo4j-go-driver/neo4j"
+)
+
+func init() {
+	db := Neo4j{}
+	database.Register("neo4j", &db)
+}
+
+const DefaultMigrationsLabel = "SchemaMigration"
+
+var (
+	StatementSeparator           = []byte(";")
+	DefaultMultiStatementMaxSize = 10 * 1 << 20 // 10 MB
+)
+
+var (
+	ErrNilConfig = fmt.Errorf("no config")
+)
+
+type Config struct {
+	MigrationsLabel       string
+	MultiStatement        bool
+	MultiStatementMaxSize int
+}
+
+type Neo4j struct {
+	driver neo4j.Driver
+	lock   uint32
+
+	// Open and WithInstance need to guarantee that config is never nil
+	config *Config
+}
+
+func WithInstance(driver neo4j.Driver, config *Config) (database.Driver, error) {
+	if config == nil {
+		return nil, ErrNilConfig
+	}
+
+	nDriver := &Neo4j{
+		driver: driver,
+		config: config,
+	}
+
+	if err := nDriver.ensureVersionConstraint(); err != nil {
+		return nil, err
+	}
+
+	return nDriver, nil
+}
+
+func (n *Neo4j) Open(url string) (database.Driver, error) {
+	uri, err := neturl.Parse(url)
+	if err != nil {
+		return nil, err
+	}
+	password, _ := uri.User.Password()
+	authToken := neo4j.BasicAuth(uri.User.Username(), password, "")
+	uri.User = nil
+	uri.Scheme = "bolt"
+	msQuery := uri.Query().Get("x-multi-statement")
+
+	// Whether to turn on/off TLS encryption.
+	tlsEncrypted := uri.Query().Get("x-tls-encrypted")
+	multi := false
+	encrypted := false
+	if msQuery != "" {
+		multi, err = strconv.ParseBool(uri.Query().Get("x-multi-statement"))
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	if tlsEncrypted != "" {
+		encrypted, err = strconv.ParseBool(tlsEncrypted)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	multiStatementMaxSize := DefaultMultiStatementMaxSize
+	if s := uri.Query().Get("x-multi-statement-max-size"); s != "" {
+		multiStatementMaxSize, err = strconv.Atoi(s)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	uri.RawQuery = ""
+
+	driver, err := neo4j.NewDriver(uri.String(), authToken, func(config *neo4j.Config) {
+		config.Encrypted = encrypted
+	})
+	if err != nil {
+		return nil, err
+	}
+
+	return WithInstance(driver, &Config{
+		MigrationsLabel:       DefaultMigrationsLabel,
+		MultiStatement:        multi,
+		MultiStatementMaxSize: multiStatementMaxSize,
+	})
+}
+
+func (n *Neo4j) Close() error {
+	return n.driver.Close()
+}
+
+// local locking in order to pass tests, Neo doesn't support database locking
+func (n *Neo4j) Lock() error {
+	if !atomic.CompareAndSwapUint32(&n.lock, 0, 1) {
+		return database.ErrLocked
+	}
+
+	return nil
+}
+
+func (n *Neo4j) Unlock() error {
+	if !atomic.CompareAndSwapUint32(&n.lock, 1, 0) {
+		return database.ErrNotLocked
+	}
+	return nil
+}
+
+func (n *Neo4j) Run(migration io.Reader) (err error) {
+	session, err := n.driver.Session(neo4j.AccessModeWrite)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if cerr := session.Close(); cerr != nil {
+			err = errors.Join(err, cerr)
+		}
+	}()
+
+	if n.config.MultiStatement {
+		_, err = session.WriteTransaction(func(transaction neo4j.Transaction) (interface{}, error) {
+			var stmtRunErr error
+			if err := multistmt.Parse(migration, StatementSeparator, n.config.MultiStatementMaxSize, func(stmt []byte) bool {
+				trimStmt := bytes.TrimSpace(stmt)
+				if len(trimStmt) == 0 {
+					return true
+				}
+				trimStmt = bytes.TrimSuffix(trimStmt, StatementSeparator)
+				if len(trimStmt) == 0 {
+					return true
+				}
+
+				result, err := transaction.Run(string(trimStmt), nil)
+				if _, err := neo4j.Collect(result, err); err != nil {
+					stmtRunErr = err
+					return false
+				}
+				return true
+			}); err != nil {
+				return nil, err
+			}
+			return nil, stmtRunErr
+		})
+		return err
+	}
+
+	body, err := io.ReadAll(migration)
+	if err != nil {
+		return err
+	}
+
+	_, err = neo4j.Collect(session.Run(string(body[:]), nil))
+	return err
+}
+
+func (n *Neo4j) SetVersion(version int, dirty bool) (err error) {
+	session, err := n.driver.Session(neo4j.AccessModeWrite)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if cerr := session.Close(); cerr != nil {
+			err = errors.Join(err, cerr)
+		}
+	}()
+
+	query := fmt.Sprintf("MERGE (sm:%s {version: $version}) SET sm.dirty = $dirty, sm.ts = datetime()",
+		n.config.MigrationsLabel)
+	_, err = neo4j.Collect(session.Run(query, map[string]interface{}{"version": version, "dirty": dirty}))
+	if err != nil {
+		return err
+	}
+	return nil
+}
+
+type MigrationRecord struct {
+	Version int
+	Dirty   bool
+}
+
+func (n *Neo4j) Version() (version int, dirty bool, err error) {
+	session, err := n.driver.Session(neo4j.AccessModeRead)
+	if err != nil {
+		return database.NilVersion, false, err
+	}
+	defer func() {
+		if cerr := session.Close(); cerr != nil {
+			err = errors.Join(err, cerr)
+		}
+	}()
+
+	query := fmt.Sprintf(`MATCH (sm:%s) RETURN sm.version AS version, sm.dirty AS dirty
+ORDER BY COALESCE(sm.ts, datetime({year: 0})) DESC, sm.version DESC LIMIT 1`,
+		n.config.MigrationsLabel)
+	result, err := session.ReadTransaction(func(transaction neo4j.Transaction) (interface{}, error) {
+		result, err := transaction.Run(query, nil)
+		if err != nil {
+			return nil, err
+		}
+		if result.Next() {
+			record := result.Record()
+			mr := MigrationRecord{}
+			versionResult, ok := record.Get("version")
+			if !ok {
+				mr.Version = database.NilVersion
+			} else {
+				mr.Version = int(versionResult.(int64))
+			}
+
+			dirtyResult, ok := record.Get("dirty")
+			if ok {
+				mr.Dirty = dirtyResult.(bool)
+			}
+
+			return mr, nil
+		}
+		return nil, result.Err()
+	})
+	if err != nil {
+		return database.NilVersion, false, err
+	}
+	if result == nil {
+		return database.NilVersion, false, err
+	}
+	mr := result.(MigrationRecord)
+	return mr.Version, mr.Dirty, err
+}
+
+func (n *Neo4j) Drop() (err error) {
+	session, err := n.driver.Session(neo4j.AccessModeWrite)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if cerr := session.Close(); cerr != nil {
+			err = errors.Join(err, cerr)
+		}
+	}()
+
+	if _, err := neo4j.Collect(session.Run("MATCH (n) DETACH DELETE n", nil)); err != nil {
+		return err
+	}
+	return nil
+}
+
+func (n *Neo4j) ensureVersionConstraint() (err error) {
+	session, err := n.driver.Session(neo4j.AccessModeWrite)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if cerr := session.Close(); cerr != nil {
+			err = errors.Join(err, cerr)
+		}
+	}()
+
+	/**
+	Get constraint and check to avoid error duplicate
+	using db.labels() to support Neo4j 3 and 4.
+	Neo4J 3 doesn't support db.constraints() YIELD name
+	*/
+	res, err := neo4j.Collect(session.Run(fmt.Sprintf("CALL db.labels() YIELD label WHERE label=\"%s\" RETURN label", n.config.MigrationsLabel), nil))
+	if err != nil {
+		return err
+	}
+	if len(res) == 1 {
+		return nil
+	}
+
+	query := fmt.Sprintf("CREATE CONSTRAINT ON (a:%s) ASSERT a.version IS UNIQUE", n.config.MigrationsLabel)
+	if _, err := neo4j.Collect(session.Run(query, nil)); err != nil {
+		return err
+	}
+	return nil
+}

database/neo4j/neo4j_test.go

@@ -0,0 +1,138 @@
+package neo4j
+
+import (
+	"bytes"
+	"context"
+	"fmt"
+	"log"
+	"testing"
+
+	"github.com/dhui/dktest"
+	"github.com/neo4j/neo4j-go-driver/neo4j"
+
+	"github.com/golang-migrate/migrate/v4"
+	dt "github.com/golang-migrate/migrate/v4/database/testing"
+	"github.com/golang-migrate/migrate/v4/dktesting"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+)
+
+var (
+	opts = dktest.Options{PortRequired: true, ReadyFunc: isReady,
+		Env: map[string]string{"NEO4J_AUTH": "neo4j/migratetest", "NEO4J_ACCEPT_LICENSE_AGREEMENT": "yes"}}
+	specs = []dktesting.ContainerSpec{
+		{ImageName: "neo4j:4.0", Options: opts},
+		{ImageName: "neo4j:4.0-enterprise", Options: opts},
+		{ImageName: "neo4j:3.5", Options: opts},
+		{ImageName: "neo4j:3.5-enterprise", Options: opts},
+	}
+)
+
+func neoConnectionString(host, port string) string {
+	return fmt.Sprintf("bolt://neo4j:migratetest@%s:%s", host, port)
+}
+
+func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
+	ip, port, err := c.Port(7687)
+	if err != nil {
+		return false
+	}
+
+	driver, err := neo4j.NewDriver(
+		neoConnectionString(ip, port),
+		neo4j.BasicAuth("neo4j", "migratetest", ""),
+		func(config *neo4j.Config) {
+			config.Encrypted = false
+		})
+	if err != nil {
+		return false
+	}
+	defer func() {
+		if err := driver.Close(); err != nil {
+			log.Println("close error:", err)
+		}
+	}()
+	session, err := driver.Session(neo4j.AccessModeRead)
+	if err != nil {
+		return false
+	}
+	result, err := session.Run("RETURN 1", nil)
+	if err != nil {
+		return false
+	} else if result.Err() != nil {
+		return false
+	}
+
+	return true
+}
+
+func Test(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(7687)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		n := &Neo4j{}
+		d, err := n.Open(neoConnectionString(ip, port))
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+		dt.Test(t, d, []byte("MATCH (a) RETURN a"))
+	})
+}
+
+func TestMigrate(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(7687)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		n := &Neo4j{}
+		neoUrl := neoConnectionString(ip, port) + "/?x-multi-statement=true"
+		d, err := n.Open(neoUrl)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+		m, err := migrate.NewWithDatabaseInstance("file://./examples/migrations", "neo4j", d)
+		if err != nil {
+			t.Fatal(err)
+		}
+		dt.TestMigrate(t, m)
+	})
+}
+
+func TestMalformed(t *testing.T) {
+	dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
+		ip, port, err := c.Port(7687)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		n := &Neo4j{}
+		d, err := n.Open(neoConnectionString(ip, port))
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer func() {
+			if err := d.Close(); err != nil {
+				t.Error(err)
+			}
+		}()
+
+		migration := bytes.NewReader([]byte("CREATE (a {qid: 1) RETURN a"))
+		if err := d.Run(migration); err == nil {
+			t.Fatal("expected failure for malformed migration")
+		}
+	})
+}

database/parse_test.go

@@ -8,12 +8,15 @@ import (
 )
 
 const reservedChars = "!#$%&'()*+,/:;=?@[]"
+const reservedCharTestNamePrefix = "reserved char "
+
+const baseUsername = "username"
+
+const scheme = "database://"
 
 // TestUserUnencodedReservedURLChars documents the behavior of using unencoded reserved characters in usernames with
 // net/url Parse()
 func TestUserUnencodedReservedURLChars(t *testing.T) {
-	scheme := "database://"
-	baseUsername := "username"
 	urlSuffix := "password@localhost:12345/myDB?someParam=true"
 	urlSuffixAndSep := ":" + urlSuffix
 
@@ -46,7 +49,7 @@ func TestUserUnencodedReservedURLChars(t *testing.T) {
 			encodedURL: scheme + baseUsername + "," + urlSuffixAndSep},
 		{char: "/", parses: true, expectedUsername: "",
 			encodedURL: scheme + baseUsername + "/" + urlSuffixAndSep},
-		{char: ":", parses: true, expectedUsername: "username",
+		{char: ":", parses: true, expectedUsername: baseUsername,
 			encodedURL: scheme + baseUsername + ":%3A" + urlSuffix},
 		{char: ";", parses: true, expectedUsername: "username;",
 			encodedURL: scheme + baseUsername + ";" + urlSuffixAndSep},
@@ -63,7 +66,7 @@ func TestUserUnencodedReservedURLChars(t *testing.T) {
 	testedChars := make([]string, 0, len(reservedChars))
 	for _, tc := range testcases {
 		testedChars = append(testedChars, tc.char)
-		t.Run("reserved char "+tc.char, func(t *testing.T) {
+		t.Run(reservedCharTestNamePrefix+tc.char, func(t *testing.T) {
 			s := scheme + baseUsername + tc.char + urlSuffixAndSep
 			u, err := url.Parse(s)
 			if err == nil {
@@ -97,14 +100,12 @@ func TestUserUnencodedReservedURLChars(t *testing.T) {
 }
 
 func TestUserEncodedReservedURLChars(t *testing.T) {
-	scheme := "database://"
-	baseUsername := "username"
 	urlSuffix := "password@localhost:12345/myDB?someParam=true"
 	urlSuffixAndSep := ":" + urlSuffix
 
 	for _, c := range reservedChars {
 		c := string(c)
-		t.Run("reserved char "+c, func(t *testing.T) {
+		t.Run(reservedCharTestNamePrefix+c, func(t *testing.T) {
 			encodedChar := "%" + hex.EncodeToString([]byte(c))
 			s := scheme + baseUsername + encodedChar + urlSuffixAndSep
 			expectedUsername := baseUsername + c
@@ -125,8 +126,8 @@ func TestUserEncodedReservedURLChars(t *testing.T) {
 // TestPasswordUnencodedReservedURLChars documents the behavior of using unencoded reserved characters in passwords
 // with net/url Parse()
 func TestPasswordUnencodedReservedURLChars(t *testing.T) {
-	username := "username"
-	schemeAndUsernameAndSep := "database://" + username + ":"
+	username := baseUsername
+	schemeAndUsernameAndSep := scheme + username + ":"
 	basePassword := "password"
 	urlSuffixAndSep := "@localhost:12345/myDB?someParam=true"
 
@@ -139,8 +140,7 @@ func TestPasswordUnencodedReservedURLChars(t *testing.T) {
 	}{
 		{char: "!", parses: true, expectedUsername: username, expectedPassword: basePassword + "!",
 			encodedURL: schemeAndUsernameAndSep + basePassword + "%21" + urlSuffixAndSep},
-		{char: "#", parses: true, expectedUsername: "", expectedPassword: "",
-			encodedURL: schemeAndUsernameAndSep + basePassword + "#" + urlSuffixAndSep},
+		{char: "#", parses: false},
 		{char: "$", parses: true, expectedUsername: username, expectedPassword: basePassword + "$",
 			encodedURL: schemeAndUsernameAndSep + basePassword + "$" + urlSuffixAndSep},
 		{char: "%", parses: false},
@@ -158,16 +158,14 @@ func TestPasswordUnencodedReservedURLChars(t *testing.T) {
 			encodedURL: schemeAndUsernameAndSep + basePassword + "+" + urlSuffixAndSep},
 		{char: ",", parses: true, expectedUsername: username, expectedPassword: "password,",
 			encodedURL: schemeAndUsernameAndSep + basePassword + "," + urlSuffixAndSep},
-		{char: "/", parses: true, expectedUsername: "", expectedPassword: "",
-			encodedURL: schemeAndUsernameAndSep + basePassword + "/" + urlSuffixAndSep},
+		{char: "/", parses: false},
 		{char: ":", parses: true, expectedUsername: username, expectedPassword: "password:",
 			encodedURL: schemeAndUsernameAndSep + basePassword + "%3A" + urlSuffixAndSep},
 		{char: ";", parses: true, expectedUsername: username, expectedPassword: "password;",
 			encodedURL: schemeAndUsernameAndSep + basePassword + ";" + urlSuffixAndSep},
 		{char: "=", parses: true, expectedUsername: username, expectedPassword: "password=",
 			encodedURL: schemeAndUsernameAndSep + basePassword + "=" + urlSuffixAndSep},
-		{char: "?", parses: true, expectedUsername: "", expectedPassword: "",
-			encodedURL: schemeAndUsernameAndSep + basePassword + "?" + urlSuffixAndSep},
+		{char: "?", parses: false},
 		{char: "@", parses: true, expectedUsername: username, expectedPassword: "password@",
 			encodedURL: schemeAndUsernameAndSep + basePassword + "%40" + urlSuffixAndSep},
 		{char: "[", parses: false},
@@ -177,7 +175,7 @@ func TestPasswordUnencodedReservedURLChars(t *testing.T) {
 	testedChars := make([]string, 0, len(reservedChars))
 	for _, tc := range testcases {
 		testedChars = append(testedChars, tc.char)
-		t.Run("reserved char "+tc.char, func(t *testing.T) {
+		t.Run(reservedCharTestNamePrefix+tc.char, func(t *testing.T) {
 			s := schemeAndUsernameAndSep + basePassword + tc.char + urlSuffixAndSep
 			u, err := url.Parse(s)
 			if err == nil {
@@ -215,14 +213,14 @@ func TestPasswordUnencodedReservedURLChars(t *testing.T) {
 }
 
 func TestPasswordEncodedReservedURLChars(t *testing.T) {
-	username := "username"
-	schemeAndUsernameAndSep := "database://" + username + ":"
+	username := baseUsername
+	schemeAndUsernameAndSep := scheme + username + ":"
 	basePassword := "password"
 	urlSuffixAndSep := "@localhost:12345/myDB?someParam=true"
 
 	for _, c := range reservedChars {
 		c := string(c)
-		t.Run("reserved char "+c, func(t *testing.T) {
+		t.Run(reservedCharTestNamePrefix+c, func(t *testing.T) {
 			encodedChar := "%" + hex.EncodeToString([]byte(c))
 			s := schemeAndUsernameAndSep + basePassword + encodedChar + urlSuffixAndSep
 			expectedPassword := basePassword + c

database/pgx/README.md

@@ -0,0 +1,43 @@
+# pgx
+
+This package is for [pgx/v4](https://pkg.go.dev/github.com/jackc/pgx/v4). A backend for the newer [pgx/v5](https://pkg.go.dev/github.com/jackc/pgx/v5) is [also available](v5).
+
+`pgx://user:password@host:port/dbname?query`
+
+| URL Query  | WithInstance Config | Description |
+|------------|---------------------|-------------|
+| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
+| `x-migrations-table-quoted` | `MigrationsTableQuoted` | By default, migrate quotes the migration table for SQL injection safety reasons. This option disable quoting and naively checks that you have quoted the migration table name. e.g. `"my_schema"."schema_migrations"` |
+| `x-statement-timeout` | `StatementTimeout` | Abort any statement that takes more than the specified number of milliseconds |
+| `x-multi-statement` | `MultiStatementEnabled` | Enable multi-statement execution (default: false) |
+| `x-multi-statement-max-size` | `MultiStatementMaxSize` | Maximum size of single statement in bytes (default: 10MB) |
+| `x-lock-strategy` | `LockStrategy` | Strategy used for locking during migration (default: advisory) |
+| `x-lock-table` | `LockTable` | Name of the table which maintains the migration lock (default: schema_lock) |
+| `dbname` | `DatabaseName` | The name of the database to connect to |
+| `search_path` | | This variable specifies the order in which schemas are searched when an object is referenced by a simple name with no schema specified. |
+| `user` | | The user to sign in as |
+| `password` | | The user's password |
+| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
+| `port` | | The port to bind to. (default is 5432) |
+| `fallback_application_name` | | An application_name to fall back to if one isn't provided. |
+| `connect_timeout` | | Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. |
+| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
+| `sslkey` | | Key file location. The file must contain PEM encoded data. |
+| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
+| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
+
+
+## Upgrading from v1
+
+1. Write down the current migration version from schema_migrations
+1. `DROP TABLE schema_migrations`
+2. Wrap your existing migrations in transactions ([BEGIN/COMMIT](https://www.postgresql.org/docs/current/static/transaction-iso.html)) if you use multiple statements within one migration.
+3. Download and install the latest migrate version.
+4. Force the current migration version with `migrate force <current_version>`.
+
+## Multi-statement mode
+
+In PostgreSQL running multiple SQL statements in one `Exec` executes them inside a transaction. Sometimes this
+behavior is not desirable because some statements can be only run outside of transaction (e.g.
+`CREATE INDEX CONCURRENTLY`). If you want to use `CREATE INDEX CONCURRENTLY` without activating multi-statement mode
+you have to put such statements in a separate migration files.

database/pgx/examples/migrations/1085649617_create_users_table.down.sql

@@ -0,0 +1 @@
+DROP TABLE IF EXISTS users;

database/pgx/examples/migrations/1085649617_create_users_table.up.sql

@@ -0,0 +1,5 @@
+CREATE TABLE users (
+  user_id integer unique,
+  name    varchar(40),
+  email   varchar(40)
+);

database/pgx/examples/migrations/1185749658_add_city_to_users.down.sql

@@ -0,0 +1 @@
+ALTER TABLE users DROP COLUMN IF EXISTS city;

database/pgx/examples/migrations/1185749658_add_city_to_users.up.sql

@@ -0,0 +1,3 @@
+ALTER TABLE users ADD COLUMN city varchar(100);
+
+

database/pgx/examples/migrations/1285849751_add_index_on_user_emails.down.sql

@@ -0,0 +1 @@
+DROP INDEX IF EXISTS users_email_index;

database/pgx/examples/migrations/1285849751_add_index_on_user_emails.up.sql

@@ -0,0 +1,3 @@
+CREATE UNIQUE INDEX CONCURRENTLY users_email_index ON users (email);
+
+-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.

database/pgx/examples/migrations/1385949617_create_books_table.down.sql

@@ -0,0 +1 @@
+DROP TABLE IF EXISTS books;

database/pgx/examples/migrations/1385949617_create_books_table.up.sql

@@ -0,0 +1,5 @@
+CREATE TABLE books (
+  user_id integer,
+  name    varchar(40),
+  author  varchar(40)
+);