Compare commits

..
36 Commits
Author SHA1 Message Date
Matthias KadenbachandGitHub dfe3ec0fce Merge pull request #198 from rmulley/travis-go-1.8
TravisCI Updates
2017-07-09 11:00:29 -07:00
Matthias KadenbachandGitHub 01b25ee18b Merge branch 'v1' into travis-go-1.8 2017-07-09 10:59:35 -07:00
Matthias KadenbachandGitHub afc973c4e5 Merge pull request #216 from thingful/v1.3.1-neo4j-https
Register neo4j driver using a pseudo `neo4js` protocol scheme.
2017-05-09 10:37:06 -07:00
Sam Mulube 250543a561 Register neo4j driver using a pseudo neo4js protocol scheme.
Neo4J supports both `http` and `https` connections, however `https`
connections weren't supported previously as the `neo4j` URL scheme
currently used to identify the driver type is always substituted for
just `http`.

This PR also registers the driver under an alternate key: `neo4js` which
given the way the scheme substitution happens, should allow the driver
to correctly create an `https` connection URL. This will allow the
library to be used for cases where we don't have access to the plain
`http` endpoint.
2017-05-09 09:54:37 +01:00
Matthias KadenbachandGitHub 1dffba9fa5 Merge pull request #212 from mattes/v1-gopkg
v1: github.com/mattes/migrate -> gopkg.in/mattes/migrate.v1
2017-05-06 16:25:42 -07:00
Matthias Kadenbach 964e6bc070 rewrite in readmes 2017-05-06 16:18:32 -07:00
Matthias Kadenbach 1887c0f3e3 rewrite paths 2017-05-06 16:13:08 -07:00
Matthias Kadenbach 62dba4f8ea update travis branch 2017-04-25 12:02:57 -07:00
Matthias KadenbachandGitHub 78ca4eb007 Merge pull request #190 from alaczi/mssql_support
MSSQL support
2017-04-25 11:35:40 -07:00
Andras Laczi d21443f74b Remove go1.4 from travis as the ql package is not compatible and breaks the build 2017-04-25 09:35:03 +01:00
Andras Laczi 7f4000c0d6 Add wait-retry to mssql test 2017-04-24 13:15:17 +01:00
Andras Laczi ecd74e1ce6 Add go 1.8 to travis config
Fix getting the latest version
2017-04-24 13:15:12 +01:00
Matthias Kadenbach 9f14676773 Update Readme 2017-04-20 15:04:54 -07:00
Matthias KadenbachandGitHub 7dde434714 Update version.go 2017-04-20 11:50:49 -07:00
Robert Mulley fff063351d Pull latest patch versions for each go minor version 2017-04-13 13:01:59 -06:00
Robert Mulley 5498e70562 Bump docker-compose to latest, 1.11.2 2017-04-13 12:54:35 -06:00
Robert Mulley 0da98940d6 Add go 1.8 to test matrix 2017-04-13 12:44:27 -06:00
Matthias KadenbachandGitHub 4cfc044489 Merge pull request #193 from Arkq/master
Support for alternative postgres URI scheme
2017-04-03 10:33:35 -07:00
Arkadiusz Bokowy 83c703f09f Support for alternative postgres URI scheme
According to the PostgreSQL documentation (section 32.1.1.2), postgres
library supports two URI schemes: postgresql:// and postgres://

Reference:
https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
2017-04-03 09:15:25 +02:00
Andras Laczi b7845a779e Add MSSQL support 2017-03-30 15:32:14 +01:00
Matthias KadenbachandGitHub 09640b136e Merge pull request #189 from nathj07/issue-188-sqlite-err-cast
fixing error cast for sqlite3
2017-03-29 11:48:54 -07:00
Nathan Davies 7ca71e567d fixing error cast for sqlite3
Seems the sqlite3 packaged reverted to using non-pointer Error. This patch corrects the use of it here.
2017-03-29 14:06:38 +01:00
Matthias KadenbachandGitHub c7653285b2 Merge pull request #185 from adrianduke/master
driver/sqlite3: Fix for upstream changes
2017-03-20 10:09:33 -07:00
Adrian Duke f4979054e2 driver/sqlite3 fix for upstream change sqlite3.Error{} Error() now takes a pointer receiver 2017-03-20 16:07:01 +00:00
Matthias KadenbachandGitHub 305a63a447 Merge pull request #181 from chrisaxiom/master
Call Close() in Version()
2017-03-02 13:39:55 -08:00
Chris S 528bc5bbbb close driver 2017-03-02 12:48:31 -08:00
Matthias KadenbachandGitHub 83e27c8b88 Merge pull request #176 from Synesis-LLC/stage
Golint. Rewrite cassandra driver
2017-02-27 10:12:48 -08:00
Pavel Sapezhko e8bfb1b2b3 Merge remote-tracking branch 'upstream/master' into stage 2017-02-27 17:56:15 +03:00
Matthias KadenbachandGitHub 2cd9bfcd04 Merge pull request #173 from sekimura/govet
Fix for go vet
2017-02-21 10:35:02 -08:00
Pavel Sapezhko 21ccec2aa1 allow create migration files without db connection 2017-02-21 09:13:59 +03:00
Masa Sekimura fe6711645a Fix for go vet
Avoid getting caught by go vet for using "\n" with Println

	main.go:187: Println call ends with newline
2017-02-20 23:39:51 +00:00
Pavel Sapezhko adf1cff659 drivers: cassandra: rewrite 2017-02-17 17:59:38 +03:00
Pavel Sapezhko da12e27062 fix some lint errors 2017-02-17 13:06:26 +03:00
Matthias KadenbachandGitHub d2ad6f8ecf Update README.md 2017-02-10 02:42:46 -08:00
Matthias KadenbachandGitHub bbb58aca8d Merge pull request #146 from FreGalle/postgres-version-table-check-schema
Postgres: version table existence check in current schema
2017-02-04 10:17:41 -08:00
Frederik Galle 5aed294d25 Check version table existence for current schema 2017-02-03 12:47:24 +01:00
249 changed files with 5359 additions and 14245 deletions
-13
View File
@@ -1,13 +0,0 @@
# Project
FAQ.md
README.md
LICENSE
Makefile
.gitignore
.travis.yml
CONTRIBUTING.md
MIGRATIONS.md
docker-deploy.sh
# Golang
testing
-39
View File
@@ -1,39 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
---
**Describe the Bug**
A clear and concise description of what the bug is.
**Steps to Reproduce**
Steps to reproduce the behavior:
1. My migrations look like '...'
2. I ran migrate with the following options '....'
3. See error
**Expected Behavior**
A clear and concise description of what you expected to happen.
**Migrate Version**
e.g. v3.4.0
Obtained by running: `migrate -version`
**Loaded Source Drivers**
e.g. s3, github, go-bindata, gcs, file
Obtained by running: `migrate -help`
**Loaded Database Drivers**
e.g. spanner, stub, clickhouse, cockroachdb, crdb-postgres, postgres, postgresql, redshift, cassandra, cockroach, mysql
Obtained by running: `migrate -help`
**Go Version**
e.g. go version go1.11 linux/amd64
Obtained by running: `go version`
**Stacktrace**
Please provide if available
**Additional context**
Add any other context about the problem here.
-17
View File
@@ -1,17 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
+1 -6
View File
@@ -1,7 +1,2 @@
.DS_Store
cli/build
cli/cli
cli/migrate
.coverage
.godoc.pid
vendor/
test.db
+12 -122
View File
@@ -1,131 +1,21 @@
language: go
sudo: required
matrix:
allow_failures:
- go: master
include:
# Supported versions of Go: https://golang.org/dl/
- go: "1.10.x"
- go: "1.11.x"
- go: master
go_import_path: github.com/golang-migrate/migrate
env:
global:
- GO111MODULE=on
- MIGRATE_TEST_CONTAINER_BOOT_TIMEOUT=60
- DOCKER_USERNAME=golangmigrate
- secure: "oSOznzUrgr5h45qW4PONkREpisPAt40tnM+KFWtS/Ggu5UI2Ie0CmyYXWuBjbt7B97a4yN9Qzmn8FxJHJ7kk+ABOi3muhkxeIhr6esXbzHhX/Jhv0mj1xkzX7KoVN9oHBz3cOI/QeRyEAO68xjDHNE2kby4RTT9VBt6TQUakKVkqI5qkqLBTADepCjVC+9XhxVxUNyeWKU8ormaUfJBjoNVoDlwXekUPnJenfmfZqXxUInvBCfUyp7Pq+kurBORmg4yc6qOlRYuK67Xw+i5xpjbZouNlXPk0rq7pPy5zjhmZQ3kImoFPvNMeKViDcI6kSIJKtjdhms9/g/6MgXS9HlL5kFy8tYKbsyiHnHB1BsvaLAKXctbUZFDPstgMPADfnad2kZXPrNqIhfWKZrGRWidawCYJ1sKKwYxLMKrtA0umqgMoL90MmBOELhuGmvMV0cFJB+zo+K2YWjEiMGd8xRb5mC5aAy0ZcCehO46jGtpr217EJmMF8Ywr7cFqM2Shg5U2jev9qUpYiXwmPnJKDuoT2ZHuHmPgFIkYiWC5yeJnnmG5bed1sKBp93AFrJX+1Rx5oC4BpNegewmBZKpOSwls/D1uMAeQK3dPmQHLsT6o2VBLfeDGr+zY0R85ywwPZCv00vGol02zYoTqN7eFqr6Qhjr/qx5K1nnxJdFK3Ts="
go:
- 1.4.x
- 1.5.x
- 1.6.x
- 1.7.x
- 1.8.x
services:
- docker
cache:
directories:
- $GOPATH/pkg
before_install:
- echo "TRAVIS_GO_VERSION=${TRAVIS_GO_VERSION}"
# Download the binary to bin folder in $GOPATH
- if [[ "${TRAVIS_GO_VERSION}" == 1.10* ]]; then curl -L -s https://github.com/golang/dep/releases/download/v0.5.0/dep-linux-amd64 -o $GOPATH/bin/dep; fi
# Make the binary executable
- if [[ "${TRAVIS_GO_VERSION}" == 1.10* ]]; then chmod +x $GOPATH/bin/dep; fi
- curl -L https://github.com/docker/compose/releases/download/1.11.2/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
- sed -i -e 's/golang/golang:'"$TRAVIS_GO_VERSION"'/' docker-compose.yml
install:
- if [[ "${TRAVIS_GO_VERSION}" == 1.10* ]]; then dep ensure -vendor-only; fi
- go get github.com/mattn/goveralls
script:
- make test COVERAGE_DIR=/tmp/coverage
after_success:
- goveralls -service=travis-ci -coverprofile /tmp/coverage/combined.txt
- make list-external-deps > dependency_tree.txt && cat dependency_tree.txt
- make build-cli
- gem install --no-document fpm
- fpm -s dir -t deb -n migrate -v "$(git describe --tags 2>/dev/null | cut -c 2-)" --license MIT -m dhui@users.noreply.github.com --url https://github.com/golang-migrate/migrate --description='Database migrations' -a amd64 -p migrate.$(git describe --tags 2>/dev/null | cut -c 2-).deb --deb-no-default-config-files -f -C cli/build migrate.linux-amd64=/usr/local/bin/migrate
deploy:
- provider: releases
api_key:
secure: hWH1HLPpzpfA8pXQ93T1qKQVFSpQp0as/JLQ7D91jHuJ8p+RxVeqblDrR6HQY/95R/nyiE9GJmvUolSuw5h449LSrGxPtVWhdh6EnkxlQHlen5XeMhVjRjFV0sE9qGe8v7uAkiTfRO61ktTWHrEAvw5qpyqnNISodmZS78XIasPODQbNlzwINhWhDTHIjXGb4FpizYaL3OGCanrxfR9fQyCaqKGGBjRq3Mfq8U6Yd4mApmsE+uJxgaZV8K5zBqpkSzQRWhcVGNL5DuLsU3gfSJOo7kZeA2G71SHffH577dBoqtCZ4VFv169CoUZehLWCb+7XKJZmHXVujCURATSySLGUOPc6EoLFAn3YtsCA04mS4bZVo5FZPWVwfhjmkhtDR4f6wscKp7r1HsFHSOgm59QfETQdrn4MnZ44H2Jd39axqndn5DvK9EcZVjPHynOPnueXP2u6mTuUgh2VyyWBCDO3CNo0fGlo7VJI69IkIWNSD87K9cHZWYMClyKZkUzS+PmRAhHRYbVd+9ZjKOmnU36kUHNDG/ft1D4ogsY+rhVtXB4lgWDM5adri+EIScYdYnB1/pQexLBigcJY9uE7nQTR0U6QgVNYvun7uRNs40E0c4voSfmPdFO0FlOD2y1oQhnaXfWLbu9nMcTcs4RFGrcC7NzkUN4/WjG8s285V6w=
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
file:
- cli/build/migrate.linux-amd64.tar.gz
- cli/build/migrate.darwin-amd64.tar.gz
- cli/build/migrate.windows-amd64.exe.tar.gz
- cli/build/sha256sum.txt
- dependency_tree.txt
- provider: packagecloud
repository: migrate
username: golang-migrate
token:
secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o=
dist: ubuntu/xenial
package_glob: '*.deb'
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
- provider: packagecloud
repository: migrate
username: golang-migrate
token:
secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o=
dist: ubuntu/bionic
package_glob: '*.deb'
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
- provider: packagecloud
repository: migrate
username: golang-migrate
token:
secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o=
dist: ubuntu/cosmic
package_glob: '*.deb'
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
- provider: packagecloud
repository: migrate
username: golang-migrate
token:
secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o=
dist: debian/stretch
package_glob: '*.deb'
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
- provider: packagecloud
repository: migrate
username: golang-migrate
token:
secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o=
dist: debian/buster
package_glob: '*.deb'
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
- provider: script
script: ./docker-deploy.sh
skip_cleanup: true
on:
go: "1.11.x"
repo: golang-migrate/migrate
tags: true
script: make test
-22
View File
@@ -1,22 +0,0 @@
# Development, Testing and Contributing
1. Make sure you have a running Docker daemon
(Install for [MacOS](https://docs.docker.com/docker-for-mac/))
1. Use a version of Go that supports [modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) (e.g. Go 1.11+)
1. Fork this repo and `git clone` somewhere to `$GOPATH/src/github.com/golang-migrate/migrate`
* Ensure that [Go modules are enabled](https://golang.org/cmd/go/#hdr-Preliminary_module_support) (e.g. your repo path or the `GO111MODULE` environment variable are set correctly)
1. Confirm tests are working: `make test-short`
1. Write awesome code ...
1. `make test` to run all tests against all database versions
1. Push code and open Pull Request
Some more helpful commands:
* You can specify which database/ source tests to run:
`make test-short SOURCE='file go_bindata' DATABASE='postgres cassandra'`
* After `make test`, run `make html-coverage` which opens a shiny test coverage overview.
* `make build-cli` builds the CLI in directory `cli/build/`.
* `make list-external-deps` lists all external dependencies for each package
* `make docs && make open-docs` opens godoc in your browser, `make kill-docs` kills the godoc server.
Repeatedly call `make docs` to refresh the server.
* Set the `DOCKER_API_VERSION` environment variable to the latest supported version if you get errors regarding the docker client API version being too new.
+2 -22
View File
@@ -1,23 +1,3 @@
FROM golang:1.11-alpine3.8 AS downloader
ARG VERSION
RUN apk add --no-cache git gcc musl-dev
WORKDIR /go/src/github.com/golang-migrate/migrate
COPY . ./
ENV GO111MODULE=on
ENV DATABASES="postgres mysql redshift cassandra spanner cockroachdb clickhouse"
ENV SOURCES="file go_bindata github aws_s3 google_cloud_storage"
RUN go build -a -o build/migrate.linux-386 -ldflags="-X main.Version=${VERSION}" -tags "$DATABASES $SOURCES" ./cmd/migrate
FROM alpine:3.8
RUN apk add --no-cache ca-certificates
COPY --from=downloader /go/src/github.com/golang-migrate/migrate/build/migrate.linux-386 /migrate
FROM scratch
ADD migrate /migrate
ENTRYPOINT ["/migrate"]
CMD ["--help"]
-70
View File
@@ -1,70 +0,0 @@
# FAQ
#### How is the code base structured?
```
/ package migrate (the heart of everything)
/cli the CLI wrapper
/database database driver and sub directories have the actual driver implementations
/source source driver and sub directories have the actual driver implementations
```
#### Why is there no `source/driver.go:Last()`?
It's not needed. And unless the source has a "native" way to read a directory in reversed order,
it might be expensive to do a full directory scan in order to get the last element.
#### What is a NilMigration? NilVersion?
NilMigration defines a migration without a body. NilVersion is defined as const -1.
#### What is the difference between uint(version) and int(targetVersion)?
version refers to an existing migration version coming from a source and therefor can never be negative.
targetVersion can either be a version OR represent a NilVersion, which equals -1.
#### What's the difference between Next/Previous and Up/Down?
```
1_first_migration.up.extension next -> 2_second_migration.up.extension ...
1_first_migration.down.extension <- previous 2_second_migration.down.extension ...
```
#### Why two separate files (up and down) for a migration?
It makes all of our lives easier. No new markup/syntax to learn for users
and existing database utility tools continue to work as expected.
#### How many migrations can migrate handle?
Whatever the maximum positive signed integer value is for your platform.
For 32bit it would be 2,147,483,647 migrations. Migrate only keeps references to
the currently run and pre-fetched migrations in memory. Please note that some
source drivers need to do build a full "directory" tree first, which puts some
heat on the memory consumption.
#### Are the table tests in migrate_test.go bloated?
Yes and no. There are duplicate test cases for sure but they don't hurt here. In fact
the tests are very visual now and might help new users understand expected behaviors quickly.
Migrate from version x to y and y is the last migration? Just check out the test for
that particular case and know what's going on instantly.
#### What is Docker being used for?
Only for testing. See [testing/docker.go](testing/docker.go)
#### Why not just use docker-compose?
It doesn't give us enough runtime control for testing. We want to be able to bring up containers fast
and whenever we want, not just once at the beginning of all tests.
#### Can I maintain my driver in my own repository?
Yes, technically thats possible. We want to encourage you to contribute your driver to this respository though.
The driver's functionality is dictated by migrate's interfaces. That means there should really
just be one driver for a database/ source. We want to prevent a future where several drivers doing the exact same thing,
just implemented a bit differently, co-exist somewhere on Github. If users have to do research first to find the
"best" available driver for a database in order to get started, we would have failed as an open source community.
#### Can I mix multiple sources during a batch of migrations?
No.
#### What does "dirty" database mean?
Before a migration runs, each database sets a dirty flag. Execution stops if a migration fails and the dirty state persists,
which prevents attempts to run more migrations on top of a failed migration. You need to manually fix the error
and then "force" the expected version.
#### What happens if two programs try and update the database at the same time?
Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations at the same time
the same database at the same time. For example, the MySQL driver uses the `GET_LOCK` function, while the Postgres driver uses
the `pg_advisory_lock` function.
Generated
-828
View File
@@ -1,828 +0,0 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
digest = "1:4a799555f4a58b849ca3a3bed67c10ca484a2d3cedf4199773211d04b0109793"
name = "cloud.google.com/go"
packages = [
"civil",
"compute/metadata",
"iam",
"internal",
"internal/atomiccache",
"internal/fields",
"internal/optional",
"internal/protostruct",
"internal/trace",
"internal/version",
"longrunning",
"longrunning/autogen",
"spanner",
"spanner/admin/database/apiv1",
"storage",
]
pruneopts = "UT"
revision = "c728a003b238b26cef9ab6753a5dc424b331c3ad"
version = "v0.27.0"
[[projects]]
branch = "master"
digest = "1:6da51e5ec493ad2b44cb04129e2d0a068c8fb9bd6cb5739d199573558696bb94"
name = "github.com/Azure/go-ansiterm"
packages = [
".",
"winterm",
]
pruneopts = "UT"
revision = "d6e3b3328b783f23731bc4d058875b0371ff8109"
[[projects]]
digest = "1:f9ae348e1f793dcf9ed930ed47136a67343dbd6809c5c91391322267f4476892"
name = "github.com/Microsoft/go-winio"
packages = ["."]
pruneopts = "UT"
revision = "97e4973ce50b2ff5f09635a57e2b88a037aae829"
version = "v0.4.11"
[[projects]]
branch = "master"
digest = "1:3721a10686511b80c052323423f0de17a8c06d417dbdd3b392b1578432a33aae"
name = "github.com/Nvveen/Gotty"
packages = ["."]
pruneopts = "UT"
revision = "cd527374f1e5bff4938207604a14f2e38a9cf512"
[[projects]]
digest = "1:1d58a675bc7df5691f81b4a28a39d2d218f020b23644a6c7771f7ebaab272ac8"
name = "github.com/aws/aws-sdk-go"
packages = [
"aws",
"aws/awserr",
"aws/awsutil",
"aws/client",
"aws/client/metadata",
"aws/corehandlers",
"aws/credentials",
"aws/credentials/ec2rolecreds",
"aws/credentials/endpointcreds",
"aws/credentials/stscreds",
"aws/csm",
"aws/defaults",
"aws/ec2metadata",
"aws/endpoints",
"aws/request",
"aws/session",
"aws/signer/v4",
"internal/s3err",
"internal/sdkio",
"internal/sdkrand",
"internal/sdkuri",
"internal/shareddefaults",
"private/protocol",
"private/protocol/eventstream",
"private/protocol/eventstream/eventstreamapi",
"private/protocol/query",
"private/protocol/query/queryutil",
"private/protocol/rest",
"private/protocol/restxml",
"private/protocol/xml/xmlutil",
"service/s3",
"service/s3/s3iface",
"service/sts",
]
pruneopts = "UT"
revision = "bf8067ceb6e7f51e150c218972dccfeeed892b85"
version = "v1.15.54"
[[projects]]
branch = "master"
digest = "1:5a75ff130630f57609ffca00d6a76dc31bb50ad6a03809848e0f9e13add31987"
name = "github.com/cockroachdb/cockroach-go"
packages = ["crdb"]
pruneopts = "UT"
revision = "e0a95dfd547cc9c3ebaaba1a12c2afe4bf621ac5"
[[projects]]
branch = "master"
digest = "1:3ba3d94d45f2d5c4b3411452474d96996b7e0dce730b689920f7c307511e5655"
name = "github.com/cznic/b"
packages = ["."]
pruneopts = "UT"
revision = "35e9bbe41f07452a183c517a5fc5f3c9f45eaa0f"
[[projects]]
branch = "master"
digest = "1:4688e1d4b22367c5bd24fe65f44b3e53a443a93fd19cf34169f1c8337d93dcf7"
name = "github.com/cznic/fileutil"
packages = ["."]
pruneopts = "UT"
revision = "6a051e75936f623600b67c2b1116b6b6c0ffb936"
[[projects]]
branch = "master"
digest = "1:2a4afef2ef0e4e4ab66b4f4f6a70bf6551ae8b727184771b5547e2b90245fe58"
name = "github.com/cznic/golex"
packages = ["lex"]
pruneopts = "UT"
revision = "4ab7c5e190e49208c823ce8ec803aa39e6a4b31a"
[[projects]]
digest = "1:a9e2943ed681d4758a43d1bd9e80d7e993b66ce5adc7aa5dcc083ba48a47a4eb"
name = "github.com/cznic/internal"
packages = [
"buffer",
"file",
"slice",
]
pruneopts = "UT"
revision = "cef02a853c3a93623c42eacd574e7ea05f55531b"
version = "1.0.0"
[[projects]]
digest = "1:ec7b58207ac8eee1c554b7947b3f5124f711238a80edb7224a53bae7263c95ae"
name = "github.com/cznic/lldb"
packages = ["."]
pruneopts = "UT"
revision = "bea8611dd5c407f3c5eab9f9c68e887a27dc6f0e"
version = "v1.1.0"
[[projects]]
branch = "master"
digest = "1:42951d361293c2f7068ab0411afeb57856afba843fe681cea1e2731b47021222"
name = "github.com/cznic/mathutil"
packages = ["."]
pruneopts = "UT"
revision = "ca4c9f2c136954238c3158b92de72078c7672ecc"
[[projects]]
digest = "1:336a34e066968a8e30ee73a0d4419f37d3e8286b94f996cddf46a5e82d13b2fa"
name = "github.com/cznic/ql"
packages = [
".",
"driver",
"vendored/github.com/camlistore/go4/lock",
]
pruneopts = "UT"
revision = "47bf73cf8ed137969f12b223b7016784cdd9d42e"
version = "v1.2.0"
[[projects]]
branch = "master"
digest = "1:06877d57543857a87186f43b6195bbdf5e8f4a0f51412fdf4baef504cead7973"
name = "github.com/cznic/sortutil"
packages = ["."]
pruneopts = "UT"
revision = "4c7342852e65c2088c981288f2c5610d10b9f7f4"
[[projects]]
branch = "master"
digest = "1:8a26e11e94f1c9f03ab9fc1e816b003629d41b26ab41eefcb7444fad5b24fc4b"
name = "github.com/cznic/strutil"
packages = ["."]
pruneopts = "UT"
revision = "529a34b1c186b483642a7a230c67521d9aa4b0fb"
[[projects]]
branch = "master"
digest = "1:bb8236408f88f411f565dffcf1e648fe34c7456820b707f96ccbfb89e1cd8eeb"
name = "github.com/cznic/zappy"
packages = ["."]
pruneopts = "UT"
revision = "2533cb5b45cc6c07421468ce262899ddc9d53fb7"
[[projects]]
digest = "1:8779ca6bbfd7b38a775f89f564834d2f1031fbf337f2eb5f18f115e40ca9a290"
name = "github.com/dhui/dktest"
packages = ["."]
pruneopts = "UT"
revision = "d72c967df8c2fb5403bfc0c37425986977343f6a"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:4ddc17aeaa82cb18c5f0a25d7c253a10682f518f4b2558a82869506eec223d76"
name = "github.com/docker/distribution"
packages = [
"digestset",
"reference",
]
pruneopts = "UT"
revision = "1cb4180b1a5b9c029b2f2beaeb38f0b5cf64e12e"
[[projects]]
digest = "1:8a2c57c577e78f40e2b3cd34ba1d050d9afd1060f91552ea03704324d1f50024"
name = "github.com/docker/docker"
packages = [
"api",
"api/types",
"api/types/blkiodev",
"api/types/container",
"api/types/events",
"api/types/filters",
"api/types/image",
"api/types/mount",
"api/types/network",
"api/types/registry",
"api/types/strslice",
"api/types/swarm",
"api/types/swarm/runtime",
"api/types/time",
"api/types/versions",
"api/types/volume",
"client",
"pkg/jsonmessage",
"pkg/term",
"pkg/term/windows",
]
pruneopts = "UT"
revision = "453f2b8b40b923862ce1c08c11531ff5042770f1"
[[projects]]
digest = "1:811c86996b1ca46729bad2724d4499014c4b9effd05ef8c71b852aad90deb0ce"
name = "github.com/docker/go-connections"
packages = [
"nat",
"sockets",
"tlsconfig",
]
pruneopts = "UT"
revision = "7395e3f8aa162843a74ed6d48e79627d9792ac55"
version = "v0.4.0"
[[projects]]
digest = "1:6f82cacd0af5921e99bf3f46748705239b36489464f4529a1589bc895764fb18"
name = "github.com/docker/go-units"
packages = ["."]
pruneopts = "UT"
revision = "47565b4f722fb6ceae66b95f853feed578a4a51c"
version = "v0.3.3"
[[projects]]
branch = "master"
digest = "1:67d0b50be0549e610017cb91e0b0b745ec0cad7c613bc8e18ff2d1c1fc8825a7"
name = "github.com/edsrzf/mmap-go"
packages = ["."]
pruneopts = "UT"
revision = "0bce6a6887123b67a60366d2c9fe2dfb74289d2e"
[[projects]]
digest = "1:812c304ad1cbb214459f7a198cc246a9c8a114a6a92063cef7b31c4ec42a5d6c"
name = "github.com/fsouza/fake-gcs-server"
packages = [
"fakestorage",
"internal/backend",
]
pruneopts = "UT"
revision = "9f9efdc50d6ad4314a4ff85646341f2a9b75ae5f"
version = "v1.3.0"
[[projects]]
digest = "1:15e27372d379b45b18ac917b9dafc45c45485239490ece18cca97a12f9591146"
name = "github.com/go-ini/ini"
packages = ["."]
pruneopts = "UT"
revision = "9c8236e659b76e87bf02044d06fde8683008ff3e"
version = "v1.39.0"
[[projects]]
digest = "1:ec6f9bf5e274c833c911923c9193867f3f18788c461f76f05f62bb1510e0ae65"
name = "github.com/go-sql-driver/mysql"
packages = ["."]
pruneopts = "UT"
revision = "72cd26f257d44c1114970e19afddcd812016007e"
version = "v1.4.1"
[[projects]]
digest = "1:586ea76dbd0374d6fb649a91d70d652b7fe0ccffb8910a77468e7702e7901f3d"
name = "github.com/go-stack/stack"
packages = ["."]
pruneopts = "UT"
revision = "2fee6af1a9795aafbe0253a0cfbdf668e1fb8a9a"
version = "v1.8.0"
[[projects]]
branch = "master"
digest = "1:1a5f28249f1a5b118994359a1c7b8c53ee1ff47aa7fe65884fbd20cd237001f4"
name = "github.com/gocql/gocql"
packages = [
".",
"internal/lru",
"internal/murmur",
"internal/streams",
]
pruneopts = "UT"
revision = "70385f88b28b43805bd83d212169ab2d38810b15"
[[projects]]
digest = "1:bbadccf3d3317ea03c0dac0b45b673b4b397c8f91a1d2eff550a3c51c4ad770e"
name = "github.com/gogo/protobuf"
packages = ["proto"]
pruneopts = "UT"
revision = "636bf0302bc95575d69441b25a2603156ffdddf1"
version = "v1.1.1"
[[projects]]
digest = "1:ec0bde6dc8cf3cb9254f79603bd8489c366a0a8584949cac5a73fa10b9f47c21"
name = "github.com/golang/protobuf"
packages = [
"proto",
"protoc-gen-go/descriptor",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/empty",
"ptypes/struct",
"ptypes/timestamp",
]
pruneopts = "UT"
revision = "aa810b61a9c79d51363740d207bb46cf8e620ed5"
version = "v1.2.0"
[[projects]]
branch = "master"
digest = "1:4a0c6bb4805508a6287675fac876be2ac1182539ca8a32468d8128882e9d5009"
name = "github.com/golang/snappy"
packages = ["."]
pruneopts = "UT"
revision = "2e65f85255dbc3072edf28d6b5b8efc472979f5a"
[[projects]]
digest = "1:f3fb942d9a52aff79191a8318317147ab76ca609bced931b8be9ab1825569993"
name = "github.com/google/go-github"
packages = ["github"]
pruneopts = "UT"
revision = "747f93dd9b489bd56f7442b3c42bfea12800a75b"
version = "v17.0.0"
[[projects]]
digest = "1:a63cff6b5d8b95638bfe300385d93b2a6d9d687734b863da8e09dc834510a690"
name = "github.com/google/go-querystring"
packages = ["query"]
pruneopts = "UT"
revision = "44c6ddd0a2342c386950e880b658017258da92fc"
version = "v1.0.0"
[[projects]]
digest = "1:e145e9710a10bc114a6d3e2738aadf8de146adaa031854ffdf7bbfe15da85e63"
name = "github.com/googleapis/gax-go"
packages = ["."]
pruneopts = "UT"
revision = "317e0006254c44a0ac427cc52a0e083ff0b9622f"
version = "v2.0.0"
[[projects]]
digest = "1:c79fb010be38a59d657c48c6ba1d003a8aa651fa56b579d959d74573b7dff8e1"
name = "github.com/gorilla/context"
packages = ["."]
pruneopts = "UT"
revision = "08b5f424b9271eedf6f9f0ce86cb9396ed337a42"
version = "v1.1.1"
[[projects]]
digest = "1:e73f5b0152105f18bc131fba127d9949305c8693f8a762588a82a48f61756f5f"
name = "github.com/gorilla/mux"
packages = ["."]
pruneopts = "UT"
revision = "e3702bed27f0d39777b0b37b664b6280e8ef8fbf"
version = "v1.6.2"
[[projects]]
branch = "master"
digest = "1:364b908b9b27b97ab838f2f6f1b1f46281fa29b978a037d72a9b1d4f6d940190"
name = "github.com/hailocab/go-hostpool"
packages = ["."]
pruneopts = "UT"
revision = "e80d13ce29ede4452c43dea11e79b9bc8a15b478"
[[projects]]
digest = "1:0ade334594e69404d80d9d323445d2297ff8161637f9b2d347cc6973d2d6f05b"
name = "github.com/hashicorp/errwrap"
packages = ["."]
pruneopts = "UT"
revision = "8a6fb523712970c966eefc6b39ed2c5e74880354"
version = "v1.0.0"
[[projects]]
digest = "1:f668349b83f7d779567c880550534addeca7ebadfdcf44b0b9c39be61864b4b7"
name = "github.com/hashicorp/go-multierror"
packages = ["."]
pruneopts = "UT"
revision = "886a7fbe3eb1c874d46f623bfa70af45f425b3d1"
version = "v1.0.0"
[[projects]]
digest = "1:8dfbec76b4834d96c37ba4c4969fc47c9a15e67c5a7eecfa031fbe67dab50429"
name = "github.com/jackc/pgx"
packages = [
".",
"chunkreader",
"internal/sanitize",
"pgio",
"pgproto3",
"pgtype",
]
pruneopts = "UT"
revision = "89f1e6ac7276b61d885db5e5aed6fcbedd1c7e31"
version = "v3.2.0"
[[projects]]
digest = "1:e22af8c7518e1eab6f2eab2b7d7558927f816262586cd6ed9f349c97a6c285c4"
name = "github.com/jmespath/go-jmespath"
packages = ["."]
pruneopts = "UT"
revision = "0b12d6b5"
[[projects]]
digest = "1:0a69a1c0db3591fcefb47f115b224592c8dfa4368b7ba9fae509d5e16cdc95c8"
name = "github.com/konsorten/go-windows-terminal-sequences"
packages = ["."]
pruneopts = "UT"
revision = "5c8c8bd35d3832f5d134ae1e1e375b69a4d25242"
version = "v1.0.1"
[[projects]]
digest = "1:f02dbf2124e5dfb5305a6c3c8957a41d54a89d2a3ef60f030769f86e7851d5f5"
name = "github.com/kshvakov/clickhouse"
packages = [
".",
"lib/binary",
"lib/column",
"lib/data",
"lib/protocol",
"lib/types",
"lib/writebuffer",
]
pruneopts = "UT"
revision = "aeedb7b9d0584f393905447021238b5c6e9c5154"
version = "v1.3.4"
[[projects]]
digest = "1:8ef506fc2bb9ced9b151dafa592d4046063d744c646c1bbe801982ce87e4bc24"
name = "github.com/lib/pq"
packages = [
".",
"oid",
]
pruneopts = "UT"
revision = "4ded0e9383f75c197b3a2aaa6d590ac52df6fd79"
version = "v1.0.0"
[[projects]]
digest = "1:3cafc6a5a1b8269605d9df4c6956d43d8011fc57f266ca6b9d04da6c09dee548"
name = "github.com/mattn/go-sqlite3"
packages = ["."]
pruneopts = "UT"
revision = "25ecb14adfc7543176f7d85291ec7dba82c6f7e4"
version = "v1.9.0"
[[projects]]
digest = "1:820eeb72f78947f9b2e87bd99a57b5cc791c8fb47676a5c3824d408441ab1546"
name = "github.com/mongodb/mongo-go-driver"
packages = [
"bson",
"bson/bsoncodec",
"bson/bsonrw",
"bson/bsontype",
"bson/primitive",
"event",
"internal",
"mongo",
"mongo/options",
"mongo/readconcern",
"mongo/readpref",
"mongo/writeconcern",
"tag",
"version",
"x/bsonx",
"x/bsonx/bsoncore",
"x/mongo/driver",
"x/mongo/driver/auth",
"x/mongo/driver/auth/internal/gssapi",
"x/mongo/driver/session",
"x/mongo/driver/topology",
"x/mongo/driver/uuid",
"x/network/address",
"x/network/command",
"x/network/compressor",
"x/network/connection",
"x/network/connstring",
"x/network/description",
"x/network/result",
"x/network/wiremessage",
]
pruneopts = "UT"
revision = "29905d4bda472574c8b499e2f93b5b2747d8fbd7"
version = "v0.1.0"
[[projects]]
digest = "1:ee4d4af67d93cc7644157882329023ce9a7bcfce956a079069a9405521c7cc8d"
name = "github.com/opencontainers/go-digest"
packages = ["."]
pruneopts = "UT"
revision = "279bed98673dd5bef374d3b6e4b09e2af76183bf"
version = "v1.0.0-rc1"
[[projects]]
digest = "1:11db38d694c130c800d0aefb502fb02519e514dc53d9804ce51d1ad25ec27db6"
name = "github.com/opencontainers/image-spec"
packages = [
"specs-go",
"specs-go/v1",
]
pruneopts = "UT"
revision = "d60099175f88c47cd379c4738d158884749ed235"
version = "v1.0.1"
[[projects]]
digest = "1:40e195917a951a8bf867cd05de2a46aaf1806c50cf92eebf4c16f78cd196f747"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = "UT"
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
digest = "1:87c2e02fb01c27060ccc5ba7c5a407cc91147726f8f40b70cceeedbc52b1f3a8"
name = "github.com/sirupsen/logrus"
packages = ["."]
pruneopts = "UT"
revision = "e1e72e9de974bd926e5c56f83753fba2df402ce5"
version = "v1.3.0"
[[projects]]
branch = "master"
digest = "1:40fdfd6ab85ca32b6935853bbba35935dcb1d796c8135efd85947566c76e662e"
name = "github.com/xdg/scram"
packages = ["."]
pruneopts = "UT"
revision = "7eeb5667e42c09cb51bf7b7c28aea8c56767da90"
[[projects]]
branch = "master"
digest = "1:f5c1d04bc09c644c592b45b9f0bad4030521b1a7d11c7dadbb272d9439fa6e8e"
name = "github.com/xdg/stringprep"
packages = ["."]
pruneopts = "UT"
revision = "73f8eece6fdcd902c185bf651de50f3828bed5ed"
[[projects]]
digest = "1:8e7fcf54cc7c0c3aba7801812ae70343585b3cb663f6cb11ad50c0c28552f359"
name = "go.opencensus.io"
packages = [
".",
"internal",
"internal/tagencoding",
"plugin/ocgrpc",
"plugin/ochttp",
"plugin/ochttp/propagation/b3",
"stats",
"stats/internal",
"stats/view",
"tag",
"trace",
"trace/internal",
"trace/propagation",
"trace/tracestate",
]
pruneopts = "UT"
revision = "79993219becaa7e29e3b60cb67f5b8e82dee11d6"
version = "v0.17.0"
[[projects]]
branch = "master"
digest = "1:9ff2b9f8d4e47013a213b9ea0d850ed787c51f5eab2038842d82397ad85a19c1"
name = "golang.org/x/crypto"
packages = [
"pbkdf2",
"ssh/terminal",
]
pruneopts = "UT"
revision = "8d7daa0c54b357f3071e11eaef7efc4e19a417e2"
[[projects]]
branch = "master"
digest = "1:56975096c005d57a2c9c2faefbec772c44c29c084af35facbfca85961469ff6d"
name = "golang.org/x/net"
packages = [
"context",
"context/ctxhttp",
"http/httpguts",
"http2",
"http2/hpack",
"idna",
"internal/socks",
"internal/timeseries",
"proxy",
"trace",
]
pruneopts = "UT"
revision = "49bb7cea24b1df9410e1712aa6433dae904ff66a"
[[projects]]
branch = "master"
digest = "1:39273b3e6c96fed52bd8f7793fc463c40b1a27a7caa51302b81d45306b9646b4"
name = "golang.org/x/oauth2"
packages = [
".",
"google",
"internal",
"jws",
"jwt",
]
pruneopts = "UT"
revision = "c57b0facaced709681d9f90397429b9430a74754"
[[projects]]
branch = "master"
digest = "1:75515eedc0dc2cb0b40372008b616fa2841d831c63eedd403285ff286c593295"
name = "golang.org/x/sync"
packages = ["semaphore"]
pruneopts = "UT"
revision = "37e7f081c4d4c64e13b10787722085407fe5d15f"
[[projects]]
branch = "master"
digest = "1:f5aa274a0377f85735edc7fedfb0811d3cbc20af91633797cb359e29c3272271"
name = "golang.org/x/sys"
packages = [
"unix",
"windows",
]
pruneopts = "UT"
revision = "fa43e7bc11baaae89f3f902b2b4d832b68234844"
[[projects]]
digest = "1:a2ab62866c75542dd18d2b069fec854577a20211d7c0ea6ae746072a1dccdd18"
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"language",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable",
]
pruneopts = "UT"
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:fc393bc931c39e70d322b5158a90ef38e1a1617afdffb4efaaa4567f5c4f0410"
name = "golang.org/x/tools"
packages = [
"godoc/vfs",
"godoc/vfs/mapfs",
]
pruneopts = "UT"
revision = "8919434dde1e35f7b9ef589ecb80ec0833f021bc"
[[projects]]
branch = "master"
digest = "1:c8a31b4403741a053bf08594b91189ec63c96b041bdfddef06a272f42549f2cf"
name = "google.golang.org/api"
packages = [
"gensupport",
"googleapi",
"googleapi/internal/uritemplates",
"googleapi/transport",
"internal",
"iterator",
"option",
"storage/v1",
"transport",
"transport/grpc",
"transport/http",
"transport/http/internal/propagation",
]
pruneopts = "UT"
revision = "625cd1887957946515db468ce519bb71fa31fc7f"
[[projects]]
digest = "1:d2cfb607095cee410f25a3c04eed77f76c4721d1bb0cbc894f5d8624b33394c8"
name = "google.golang.org/appengine"
packages = [
".",
"cloudsql",
"internal",
"internal/app_identity",
"internal/base",
"internal/datastore",
"internal/log",
"internal/modules",
"internal/remote_api",
"internal/socket",
"internal/urlfetch",
"socket",
"urlfetch",
]
pruneopts = "UT"
revision = "ae0ab99deb4dc413a2b4bd6c8bdd0eb67f1e4d06"
version = "v1.2.0"
[[projects]]
branch = "master"
digest = "1:315f8fe35ad73727d15b20fc3a533e7ab9ea8f0c3ff6441753d04086e5ca5003"
name = "google.golang.org/genproto"
packages = [
"googleapis/api/annotations",
"googleapis/iam/v1",
"googleapis/longrunning",
"googleapis/rpc/code",
"googleapis/rpc/errdetails",
"googleapis/rpc/status",
"googleapis/spanner/admin/database/v1",
"googleapis/spanner/v1",
]
pruneopts = "UT"
revision = "af9cb2a35e7f169ec875002c1829c9b315cddc04"
[[projects]]
digest = "1:4ad047d772a7d4841687df9534a767a9e243885ed6d7d2ced6af5995a5dc44b3"
name = "google.golang.org/grpc"
packages = [
".",
"balancer",
"balancer/base",
"balancer/roundrobin",
"codes",
"connectivity",
"credentials",
"credentials/oauth",
"encoding",
"encoding/proto",
"grpclog",
"internal",
"internal/backoff",
"internal/channelz",
"internal/envconfig",
"internal/grpcrand",
"internal/transport",
"keepalive",
"metadata",
"naming",
"peer",
"resolver",
"resolver/dns",
"resolver/passthrough",
"stats",
"status",
"tap",
]
pruneopts = "UT"
revision = "8dea3dc473e90c8179e519d91302d0597c0ca1d1"
version = "v1.15.0"
[[projects]]
digest = "1:2d1fbdc6777e5408cabeb02bf336305e724b925ff4546ded0fa8715a7267922a"
name = "gopkg.in/inf.v0"
packages = ["."]
pruneopts = "UT"
revision = "d2d2541c53f18d2a059457998ce2876cc8e67cbf"
version = "v0.9.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"cloud.google.com/go/spanner",
"cloud.google.com/go/spanner/admin/database/apiv1",
"cloud.google.com/go/storage",
"github.com/aws/aws-sdk-go/aws",
"github.com/aws/aws-sdk-go/aws/session",
"github.com/aws/aws-sdk-go/service/s3",
"github.com/aws/aws-sdk-go/service/s3/s3iface",
"github.com/cockroachdb/cockroach-go/crdb",
"github.com/cznic/ql/driver",
"github.com/dhui/dktest",
"github.com/docker/docker/api/types",
"github.com/docker/docker/api/types/container",
"github.com/docker/docker/api/types/network",
"github.com/docker/docker/client",
"github.com/fsouza/fake-gcs-server/fakestorage",
"github.com/go-sql-driver/mysql",
"github.com/gocql/gocql",
"github.com/google/go-github/github",
"github.com/hashicorp/go-multierror",
"github.com/kshvakov/clickhouse",
"github.com/lib/pq",
"github.com/mattn/go-sqlite3",
"github.com/mongodb/mongo-go-driver/bson",
"github.com/mongodb/mongo-go-driver/mongo",
"github.com/mongodb/mongo-go-driver/x/network/connstring",
"golang.org/x/net/context",
"golang.org/x/tools/godoc/vfs",
"golang.org/x/tools/godoc/vfs/mapfs",
"google.golang.org/api/iterator",
"google.golang.org/genproto/googleapis/spanner/admin/database/v1",
]
solver-name = "gps-cdcl"
solver-version = 1
-110
View File
@@ -1,110 +0,0 @@
# Gopkg.toml example
#
# Refer to https://golang.github.io/dep/docs/Gopkg.toml.html
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
# name = "github.com/user/project"
# version = "1.0.0"
#
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"
#
# [prune]
# non-go = false
# go-tests = true
# unused-packages = true
[[constraint]]
name = "github.com/aws/aws-sdk-go"
version = "1.15.34"
[[constraint]]
branch = "master"
name = "github.com/cockroachdb/cockroach-go"
[[constraint]]
name = "github.com/cznic/ql"
version = "1.2.0"
[[constraint]]
name = "github.com/docker/docker"
revision = "453f2b8b40b923862ce1c08c11531ff5042770f1"
[[constraint]]
name = "github.com/fsouza/fake-gcs-server"
version = "1.2.0"
[[constraint]]
name = "github.com/go-sql-driver/mysql"
version = "v1.4.0"
[[constraint]]
branch = "master"
name = "github.com/gocql/gocql"
[[constraint]]
name = "github.com/google/go-github"
version = "17.0.0"
[[constraint]]
name = "github.com/kshvakov/clickhouse"
version = "1.3.4"
[[constraint]]
name = "github.com/lib/pq"
version = "1.0.0"
[[constraint]]
name = "github.com/mattn/go-sqlite3"
version = "1.9.0"
[[constraint]]
branch = "master"
name = "golang.org/x/net"
[[constraint]]
branch = "master"
name = "google.golang.org/api"
[[constraint]]
branch = "master"
name = "google.golang.org/genproto"
[[constraint]]
name = "github.com/mongodb/mongo-go-driver"
version = "0.1.0"
[[override]]
name = "cloud.google.com/go"
version = "0.27.0"
[[override]]
branch = "master"
name = "golang.org/x/tools"
[[override]]
branch = "master"
name = "github.com/docker/distribution"
[prune]
go-tests = true
unused-packages = true
[[constraint]]
name = "github.com/dhui/dktest"
version = "0.3.0"
[[constraint]]
name = "github.com/hashicorp/go-multierror"
version = "1.0.0"
+2 -9
View File
@@ -1,13 +1,6 @@
The MIT License (MIT)
Original Work
Copyright (c) 2016 Matthias Kadenbach
https://github.com/mattes/migrate
Modified Work
Copyright (c) 2018 Dale Hui
https://github.com/golang-migrate/migrate
Copyright (c) 2014 Matthias Kadenbach
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
@@ -25,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
THE SOFTWARE.
-81
View File
@@ -1,81 +0,0 @@
# Migrations
## Migration Filename Format
A single logical migration is represented as two separate migration files, one
to migrate "up" to the specified version from the previous version, and a second
to migrate back "down" to the previous version. These migrations can be provided
by any one of the supported [migration sources](./README.md#migration-sources).
The ordering and direction of the migration files is determined by the filenames
used for them. `migrate` expects the filenames of migrations to have the format:
{version}_{title}.up.{extension}
{version}_{title}.down.{extension}
The `title` of each migration is unused, and is only for readability. Similarly,
the `extension` of the migration files is not checked by the library, and should
be an appropriate format for the database in use (`.sql` for SQL variants, for
instance).
Versions of migrations may be represented as any 64 bit unsigned integer.
All migrations are applied upward in order of increasing version number, and
downward by decreasing version number.
Common versioning schemes include incrementing integers:
1_initialize_schema.down.sql
1_initialize_schema.up.sql
2_add_table.down.sql
2_add_table.up.sql
...
Or timestamps at an appropriate resolution:
1500360784_initialize_schema.down.sql
1500360784_initialize_schema.up.sql
1500445949_add_table.down.sql
1500445949_add_table.up.sql
...
But any scheme resulting in distinct, incrementing integers as versions is valid.
It is suggested that the version number of corresponding `up` and `down` migration
files be equivalent for clarity, but they are allowed to differ so long as the
relative ordering of the migrations is preserved.
The migration files are permitted to be empty, so in the event that a migration
is a no-op or is irreversible, it is recommended to still include both migration
files, and either leaving them empty or adding a comment as appropriate.
## Migration Content Format
The format of the migration files themselves varies between database systems.
Different databases have different semantics around schema changes and when and
how they are allowed to occur
(for instance, [if schema changes can occur within a transaction](https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis)).
As such, the `migrate` library has little to no checking around the format of
migration sources. The migration files are generally processed directly by the
drivers as raw operations.
## Reversibility of Migrations
Best practice for writing schema migration is that all migrations should be
reversible. It should in theory be possible for run migrations down and back up
through any and all versions with the state being fully cleaned and recreated
by doing so.
By adhering to this recommended practice, development and deployment of new code
is cleaner and easier (cleaning database state for a new feature should be as
easy as migrating down to a prior version, and back up to the latest).
As opposed to some other migration libraries, `migrate` represents up and down
migrations as separate files. This prevents any non-standard file syntax from
being introduced which may result in unintended behavior or errors, depending
on what database is processing the file.
While it is technically possible for an up or down migration to exist on its own
without an equivalently versioned counterpart, it is strongly recommended to
always include a down migration which cleans up the state of the corresponding
up migration.
+20 -108
View File
@@ -1,118 +1,30 @@
SOURCE ?= file go_bindata github aws_s3 google_cloud_storage godoc_vfs
DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb clickhouse mongodb
VERSION ?= $(shell git describe --tags 2>/dev/null | cut -c 2-)
TEST_FLAGS ?=
REPO_OWNER ?= $(shell cd .. && basename "$$(pwd)")
COVERAGE_DIR ?= .coverage
build-cli: clean
-mkdir ./cli/build
cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o ../../cli/build/migrate.linux-amd64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
cd ./cmd/migrate && CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -a -o ../../cli/build/migrate.darwin-amd64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
cd ./cmd/migrate && CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -a -o ../../cli/build/migrate.windows-amd64.exe -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' .
cd ./cli/build && find . -name 'migrate*' | xargs -I{} tar czf {}.tar.gz {}
cd ./cli/build && shasum -a 256 * > sha256sum.txt
cat ./cli/build/sha256sum.txt
TESTFLAGS?=
IMAGE=mattes/migrate
DCR=docker-compose run --rm
GOTEST=go test $(TESTFLAGS) `go list ./... | grep -v "/vendor/"`
.PHONY: clean test build release docker-build docker-push run
all: release
clean:
-rm -r ./cli/build
rm -f migrate
fmt:
@gofmt -s -w `go list -f {{.Dir}} ./... | grep -v "/vendor/"`
test-short:
make test-with-flags --ignore-errors TEST_FLAGS='-short'
test: fmt
$(DCR) go-test
go-test: fmt
@$(GOTEST)
test:
@-rm -r $(COVERAGE_DIR)
@mkdir $(COVERAGE_DIR)
make test-with-flags TEST_FLAGS='-v -race -covermode atomic -coverprofile $$(COVERAGE_DIR)/_$$(RAND).txt -bench=. -benchmem -timeout 20m'
@echo 'mode: atomic' > $(COVERAGE_DIR)/combined.txt
@cat $(COVERAGE_DIR)/_*.txt | grep -v 'mode: atomic' >> $(COVERAGE_DIR)/combined.txt
build:
$(DCR) go-build
release: test build docker-build docker-push
test-with-flags:
@echo SOURCE: $(SOURCE)
@echo DATABASE: $(DATABASE)
@go test $(TEST_FLAGS) .
@go test $(TEST_FLAGS) ./cli/...
@go test $(TEST_FLAGS) ./database
@go test $(TEST_FLAGS) ./testing/...
@echo -n '$(SOURCE)' | tr -s ' ' '\n' | xargs -I{} go test $(TEST_FLAGS) ./source/{}
@go test $(TEST_FLAGS) ./source/testing/...
@go test $(TEST_FLAGS) ./source/stub/...
@echo -n '$(DATABASE)' | tr -s ' ' '\n' | xargs -I{} go test $(TEST_FLAGS) ./database/{}
@go test $(TEST_FLAGS) ./database/testing/...
@go test $(TEST_FLAGS) ./database/stub/...
kill-orphaned-docker-containers:
docker rm -f $(shell docker ps -aq --filter label=migrate_test)
html-coverage:
go tool cover -html=$(COVERAGE_DIR)/combined.txt
list-external-deps:
$(call external_deps,'.')
$(call external_deps,'./cli/...')
$(call external_deps,'./testing/...')
$(foreach v, $(SOURCE), $(call external_deps,'./source/$(v)/...'))
$(call external_deps,'./source/testing/...')
$(call external_deps,'./source/stub/...')
$(foreach v, $(DATABASE), $(call external_deps,'./database/$(v)/...'))
$(call external_deps,'./database/testing/...')
$(call external_deps,'./database/stub/...')
restore-import-paths:
find . -name '*.go' -type f -execdir sed -i '' s%\"github.com/$(REPO_OWNER)/migrate%\"github.com/mattes/migrate%g '{}' \;
rewrite-import-paths:
find . -name '*.go' -type f -execdir sed -i '' s%\"github.com/mattes/migrate%\"github.com/$(REPO_OWNER)/migrate%g '{}' \;
# example: fswatch -0 --exclude .godoc.pid --event Updated . | xargs -0 -n1 -I{} make docs
docs:
-make kill-docs
nohup godoc -play -http=127.0.0.1:6064 </dev/null >/dev/null 2>&1 & echo $$! > .godoc.pid
cat .godoc.pid
kill-docs:
@cat .godoc.pid
kill -9 $$(cat .godoc.pid)
rm .godoc.pid
open-docs:
open http://localhost:6064/pkg/github.com/$(REPO_OWNER)/migrate
# example: make release V=0.0.0
release:
git tag v$(V)
@read -p "Press enter to confirm and push to origin ..." && git push origin v$(V)
define external_deps
@echo '-- $(1)'; go list -f '{{join .Deps "\n"}}' $(1) | grep -v github.com/$(REPO_OWNER)/migrate | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}'
endef
.PHONY: build-cli clean test-short test test-with-flags html-coverage \
restore-import-paths rewrite-import-paths list-external-deps release \
docs kill-docs open-docs kill-orphaned-docker-containers
SHELL = /bin/bash
RAND = $(shell echo $$RANDOM)
docker-build:
docker build --rm -t $(IMAGE) .
docker-push:
docker push $(IMAGE)
+96 -134
View File
@@ -1,170 +1,132 @@
[![Build Status](https://img.shields.io/travis/com/golang-migrate/migrate/master.svg)](https://travis-ci.com/golang-migrate/migrate)
[![GoDoc](https://godoc.org/github.com/golang-migrate/migrate?status.svg)](https://godoc.org/github.com/golang-migrate/migrate)
[![Coverage Status](https://img.shields.io/coveralls/github/golang-migrate/migrate/master.svg)](https://coveralls.io/github/golang-migrate/migrate?branch=master)
[![packagecloud.io](https://img.shields.io/badge/deb-packagecloud.io-844fec.svg)](https://packagecloud.io/golang-migrate/migrate?filter=debs)
[![Docker Pulls](https://img.shields.io/docker/pulls/migrate/migrate.svg)](https://hub.docker.com/r/migrate/migrate/)
![Supported Go Versions](https://img.shields.io/badge/Go-1.10%2C%201.11-lightgrey.svg)
[![GitHub Release](https://img.shields.io/github/release/golang-migrate/migrate.svg)](https://github.com/golang-migrate/migrate/releases)
# You are looking at a deprecated version.
# Use the [latest version](https://github.com/mattes/migrate) instead.
---
# migrate
__Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__
[![Build Status](https://travis-ci.org/mattes/migrate.svg?branch=v1)](https://travis-ci.org/mattes/migrate)
* Migrate reads migrations from [sources](#migration-sources)
and applies them in correct order to a [database](#databases).
* Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof.
(Keeps the drivers lightweight, too.)
* Database drivers don't assume things or try to correct user input. When in doubt, fail.
A migration helper written in Go. Use it in your existing Golang code
or run commands via the CLI.
Forked from [mattes/migrate](https://github.com/mattes/migrate)
```
GoCode import gopkg.in/mattes/migrate.v1/migrate
CLI go get -u gopkg.in/mattes/migrate.v1
```
## Databases
__Features__
Database drivers run migrations. [Add a new database?](database/driver.go)
* Super easy to implement [Driver interface](http://godoc.org/gopkg.in/mattes/migrate.v1/driver#Driver).
* Gracefully quit running migrations on ``^C``.
* No magic search paths routines, no hard-coded config files.
* CLI is build on top of the ``migrate package``.
* [PostgreSQL](database/postgres)
* [Redshift](database/redshift)
* [Ql](database/ql)
* [Cassandra](database/cassandra)
* [SQLite](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165))
* [MySQL/ MariaDB](database/mysql)
* [Neo4j](database/neo4j) ([todo #167](https://github.com/mattes/migrate/issues/167))
* [MongoDB](database/mongodb)
* [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170))
* [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171))
* [Google Cloud Spanner](database/spanner)
* [CockroachDB](database/cockroachdb)
* [ClickHouse](database/clickhouse)
### Database URLs
## Available Drivers
Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: `dbdriver://username:password@host:port/dbname?option1=true&option2=false`
* [PostgreSQL](driver/postgres)
* [Cassandra](driver/cassandra)
* [SQLite](driver/sqlite3)
* [MySQL](driver/mysql) ([experimental](https://github.com/mattes/migrate/issues/1#issuecomment-58728186))
* [Neo4j](driver/neo4j)
* [Ql](driver/ql)
* [MongoDB](driver/mongodb)
* [CrateDB](driver/crate)
* [MSSQL](driver/mssql)
Any [reserved URL characters](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters) need to be escaped. Note, the `%` character also [needs to be escaped](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_the_percent_character)
Need another driver? Just implement the [Driver interface](http://godoc.org/gopkg.in/mattes/migrate.v1/driver#Driver) and open a PR.
Explicitly, the following characters need to be escaped:
`!`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `/`, `:`, `;`, `=`, `?`, `@`, `[`, `]`
It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:
## Usage from Terminal
```bash
$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$
# install
go get gopkg.in/mattes/migrate.v1
# create new migration file in path
migrate -url driver://url -path ./migrations create migration_file_xyz
# apply all available migrations
migrate -url driver://url -path ./migrations up
# roll back all migrations
migrate -url driver://url -path ./migrations down
# roll back the most recently applied migration, then run it again.
migrate -url driver://url -path ./migrations redo
# run down and then up command
migrate -url driver://url -path ./migrations reset
# show the current migration version
migrate -url driver://url -path ./migrations version
# apply the next n migrations
migrate -url driver://url -path ./migrations migrate +1
migrate -url driver://url -path ./migrations migrate +2
migrate -url driver://url -path ./migrations migrate +n
# roll back the previous n migrations
migrate -url driver://url -path ./migrations migrate -1
migrate -url driver://url -path ./migrations migrate -2
migrate -url driver://url -path ./migrations migrate -n
# go to specific migration
migrate -url driver://url -path ./migrations goto 1
migrate -url driver://url -path ./migrations goto 10
migrate -url driver://url -path ./migrations goto v
```
## Migration Sources
Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go)
## Usage in Go
* [Filesystem](source/file) - read from fileystem
* [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata))
* [Github](source/github) - read from remote Github repositories
* [AWS S3](source/aws_s3) - read from Amazon Web Services S3
* [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage
## CLI usage
* Simple wrapper around this library.
* Handles ctrl+c (SIGINT) gracefully.
* No config search paths, no config files, no magic ENV var injections.
__[CLI Documentation](cli)__
### Basic usage:
```
$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2
```
### Docker usage
```
$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
-path=/migrations/ -database postgres://localhost:5432/database up 2
```
## Use in your Go project
* API is stable and frozen for this release (v3 & v4).
* Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies.
* To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`.
* Bring your own logger.
* Uses `io.Reader` streams internally for low memory overhead.
* Thread-safe and no goroutine leaks.
__[Go Documentation](https://godoc.org/github.com/golang-migrate/migrate)__
See GoDoc here: http://godoc.org/gopkg.in/mattes/migrate.v1/migrate
```go
import (
"github.com/golang-migrate/migrate/v4"
_ "github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/github"
)
import "gopkg.in/mattes/migrate.v1/migrate"
func main() {
m, err := migrate.New(
"github://mattes:personal-access-token@mattes/migrate_test",
"postgres://localhost:5432/database?sslmode=enable")
m.Steps(2)
// Import any required drivers so that they are registered and available
import _ "gopkg.in/mattes/migrate.v1/driver/mysql"
// use synchronous versions of migration functions ...
allErrors, ok := migrate.UpSync("driver://url", "./path")
if !ok {
fmt.Println("Oh no ...")
// do sth with allErrors slice
}
```
Want to use an existing database client?
```go
import (
"database/sql"
_ "github.com/lib/pq"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/file"
)
func main() {
db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
driver, err := postgres.WithInstance(db, &postgres.Config{})
m, err := migrate.NewWithDatabaseInstance(
"file:///migrations",
"postgres", driver)
m.Steps(2)
}
// use the asynchronous version of migration functions ...
pipe := migrate.NewPipe()
go migrate.Up(pipe, "driver://url", "./path")
// pipe is basically just a channel
// write your own channel listener. see writePipe() in main.go as an example.
```
## Migration files
Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration)
The format of migration files looks like this:
```
1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql
1481574547_initial_plan_to_do_sth.up.sql # up migration instructions
1481574547_initial_plan_to_do_sth.down.sql # down migration instructions
1482438365_xxx.up.sql
1482438365_xxx.down.sql
...
```
[Best practices: How to write migrations.](MIGRATIONS.md)
## Supported Major Versions
Version | Go Modules | Min Go Version | Import | Notes
--------|------------|----------------|--------|------
**master** | :white_check_mark: | 1.10.3 | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first |
**v4** | :white_check_mark: | 1.10.3 | `import "github.com/golang-migrate/migrate/v4"` | |
**v3** | :x: | 1.10 | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | Only security fixes are backported. Only use if your project is not using Go modules. |
## Development and Contributing
Yes, please! [`Makefile`](Makefile) is your friend,
read the [development guide](CONTRIBUTING.md).
Also have a look at the [FAQ](FAQ.md).
Why two files? This way you could still do sth like
``psql -f ./db/migrations/1481574547_initial_plan_to_do_sth.up.sql`` and there is no
need for any custom markup language to divide up and down migrations. Please note
that the filename extension depends on the driver.
## Alternatives
* https://bitbucket.org/liamstask/goose
* https://github.com/tanel/dbmigrate
* https://github.com/BurntSushi/migration
* https://github.com/DavidHuie/gomigrate
* https://github.com/rubenv/sql-migrate
---
Looking for alternatives? [https://awesome-go.com/#database](https://awesome-go.com/#database).
-125
View File
@@ -1,125 +0,0 @@
# migrate CLI
## Installation
#### Download pre-build binary (Windows, MacOS, or Linux)
[Release Downloads](https://github.com/golang-migrate/migrate/releases)
```
$ curl -L https://github.com/golang-migrate/migrate/releases/download/$version/migrate.$platform-amd64.tar.gz | tar xvz
```
#### MacOS
```
$ brew install golang-migrate
```
#### Linux (*.deb package)
```
$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add -
$ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
$ apt-get update
$ apt-get install -y migrate
```
#### With Go toolchain
##### Unversioned
```
$ go get -tags 'postgres' -u github.com/golang-migrate/migrate/cmd/migrate
```
##### Versioned
```
$ go get -u -d github.com/golang-migrate/migrate/cmd/migrate
$ cd $GOPATH/src/github.com/golang-migrate/migrate/cmd/migrate
$ git checkout $TAG # e.g. v4.1.0
$ go build -tags 'postgres' -ldflags="-X main.Version=$(git describe --tags)" -o $GOPATH/bin/migrate github.com/golang-migrate/migrate/cmd/migrate
```
##### Notes:
1. Requires a version of Go that [supports modules](https://golang.org/cmd/go/#hdr-Preliminary_module_support). e.g. Go 1.11+
1. This example builds the cli which will only work with postgres. In order
to build the cli for use with other databases, replace the `postgres` build tag
with the appropriate database tag(s) for the databases desired. The tags
correspond to the names of the sub-packages underneath the
[`database`](../database) package.
1. Similarly to the database build tags, if you need to support other sources, use the appropriate build tag(s).
1. Support for build constraints will be removed in the future: https://github.com/golang-migrate/migrate/issues/60
## Usage
```
$ migrate -help
Usage: migrate OPTIONS COMMAND [arg...]
migrate [ -version | -help ]
Options:
-source Location of the migrations (driver://url)
-path Shorthand for -source=file://path
-database Run migrations against this database (driver://url)
-prefetch N Number of migrations to load in advance before executing (default 10)
-lock-timeout N Allow N seconds to acquire database lock (default 15)
-verbose Print verbose logging
-version Print version
-help Print usage
Commands:
create [-ext E] [-dir D] [-seq] [-digits N] [-format] NAME
Create a set of timestamped up/down migrations titled NAME, in directory D with extension E.
Use -seq option to generate sequential up/down migrations with N digits.
Use -format option to specify a Go time format string.
goto V Migrate to version V
up [N] Apply all or N up migrations
down [N] Apply all or N down migrations
drop Drop everyting inside database
force V Set version V but don't run migration (ignores dirty state)
version Print current migration version
```
So let's say you want to run the first two migrations
```
$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2
```
If your migrations are hosted on github
```
$ migrate -source github://mattes:personal-access-token@mattes/migrate_test \
-database postgres://localhost:5432/database down 2
```
The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received.
Send SIGKILL for immediate halt.
## Reading CLI arguments from somewhere else
##### ENV variables
```
$ migrate -database "$MY_MIGRATE_DATABASE"
```
##### JSON files
Check out https://stedolan.github.io/jq/
```
$ migrate -database "$(cat config.json | jq '.database')"
```
##### YAML files
````
$ migrate -database "$(cat config/database.yml | ruby -ryaml -e "print YAML.load(STDIN.read)['database']")"
$ migrate -database "$(cat config/database.yml | python -c 'import yaml,sys;print yaml.safe_load(sys.stdin)["database"]')"
```
-12
View File
@@ -1,12 +0,0 @@
FROM ubuntu:xenial
RUN apt-get update && \
apt-get install -y curl apt-transport-https
RUN curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add - && \
echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ xenial main" > /etc/apt/sources.list.d/migrate.list && \
apt-get update && \
apt-get install -y migrate
RUN migrate -version
-8
View File
@@ -1,8 +0,0 @@
package main
import "github.com/golang-migrate/migrate/v4/internal/cli"
// Deprecated, please use cmd/migrate
func main() {
cli.Main(Version)
}
-4
View File
@@ -1,4 +0,0 @@
package main
// Version is set in Makefile with build flags
var Version = "dev"
-7
View File
@@ -1,7 +0,0 @@
package main
import "github.com/golang-migrate/migrate/v4/internal/cli"
func main() {
cli.Main(Version)
}
-4
View File
@@ -1,4 +0,0 @@
package main
// Version is set in Makefile with build flags
var Version = "dev"
-38
View File
@@ -1,38 +0,0 @@
# Cassandra
* Drop command will not work on Cassandra 2.X because it rely on
system_schema table which comes with 3.X
* Other commands should work properly but are **not tested**
* The Cassandra driver (gocql) does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
* This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
* The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
## Usage
`cassandra://host:port/keyspace?param1=value&param2=value2`
| URL Query | Default value | Description |
|------------|-------------|-----------|
| `x-migrations-table` | schema_migrations | Name of the migrations table |
| `x-multi-statement` | false | Enable multiple statements to be ran in a single migration (See note above) |
| `port` | 9042 | The port to bind to |
| `consistency` | ALL | Migration consistency
| `protocol` | | Cassandra protocol version (3 or 4)
| `timeout` | 1 minute | Migration timeout
| `username` | nil | Username to use when authenticating. |
| `password` | nil | Password to use when authenticating. |
| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
| `sslkey` | | Key file location. The file must contain PEM encoded data. |
| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
`timeout` is parsed using [time.ParseDuration(s string)](https://golang.org/pkg/time/#ParseDuration)
## Upgrading from v1
1. Write down the current migration version from schema_migrations
2. `DROP TABLE schema_migrations`
4. Download and install the latest migrate version.
5. Force the current migration version with `migrate force <current_version>`.
-274
View File
@@ -1,274 +0,0 @@
package cassandra
import (
"errors"
"fmt"
"io"
"io/ioutil"
nurl "net/url"
"strconv"
"strings"
"time"
"github.com/gocql/gocql"
"github.com/golang-migrate/migrate/v4/database"
)
func init() {
db := new(Cassandra)
database.Register("cassandra", db)
}
var DefaultMigrationsTable = "schema_migrations"
var (
ErrNilConfig = errors.New("no config")
ErrNoKeyspace = errors.New("no keyspace provided")
ErrDatabaseDirty = errors.New("database is dirty")
ErrClosedSession = errors.New("session is closed")
)
type Config struct {
MigrationsTable string
KeyspaceName string
MultiStatementEnabled bool
}
type Cassandra struct {
session *gocql.Session
isLocked bool
// Open and WithInstance need to guarantee that config is never nil
config *Config
}
func WithInstance(session *gocql.Session, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
} else if len(config.KeyspaceName) == 0 {
return nil, ErrNoKeyspace
}
if session.Closed() {
return nil, ErrClosedSession
}
if len(config.MigrationsTable) == 0 {
config.MigrationsTable = DefaultMigrationsTable
}
c := &Cassandra{
session: session,
config: config,
}
if err := c.ensureVersionTable(); err != nil {
return nil, err
}
return c, nil
}
func (c *Cassandra) Open(url string) (database.Driver, error) {
u, err := nurl.Parse(url)
if err != nil {
return nil, err
}
// Check for missing mandatory attributes
if len(u.Path) == 0 {
return nil, ErrNoKeyspace
}
cluster := gocql.NewCluster(u.Host)
cluster.Keyspace = strings.TrimPrefix(u.Path, "/")
cluster.Consistency = gocql.All
cluster.Timeout = 1 * time.Minute
if len(u.Query().Get("username")) > 0 && len(u.Query().Get("password")) > 0 {
authenticator := gocql.PasswordAuthenticator{
Username: u.Query().Get("username"),
Password: u.Query().Get("password"),
}
cluster.Authenticator = authenticator
}
// Retrieve query string configuration
if len(u.Query().Get("consistency")) > 0 {
var consistency gocql.Consistency
consistency, err = parseConsistency(u.Query().Get("consistency"))
if err != nil {
return nil, err
}
cluster.Consistency = consistency
}
if len(u.Query().Get("protocol")) > 0 {
var protoversion int
protoversion, err = strconv.Atoi(u.Query().Get("protocol"))
if err != nil {
return nil, err
}
cluster.ProtoVersion = protoversion
}
if len(u.Query().Get("timeout")) > 0 {
var timeout time.Duration
timeout, err = time.ParseDuration(u.Query().Get("timeout"))
if err != nil {
return nil, err
}
cluster.Timeout = timeout
}
if len(u.Query().Get("sslmode")) > 0 && len(u.Query().Get("sslrootcert")) > 0 && len(u.Query().Get("sslcert")) > 0 && len(u.Query().Get("sslkey")) > 0 {
if u.Query().Get("sslmode") != "disable" {
cluster.SslOpts = &gocql.SslOptions{
CaPath: u.Query().Get("sslrootcert"),
CertPath: u.Query().Get("sslcert"),
KeyPath: u.Query().Get("sslkey"),
}
if u.Query().Get("sslmode") == "verify-full" {
cluster.SslOpts.EnableHostVerification = true
}
}
}
session, err := cluster.CreateSession()
if err != nil {
return nil, err
}
return WithInstance(session, &Config{
KeyspaceName: strings.TrimPrefix(u.Path, "/"),
MigrationsTable: u.Query().Get("x-migrations-table"),
MultiStatementEnabled: u.Query().Get("x-multi-statement") == "true",
})
}
func (c *Cassandra) Close() error {
c.session.Close()
return nil
}
func (c *Cassandra) Lock() error {
if c.isLocked {
return database.ErrLocked
}
c.isLocked = true
return nil
}
func (c *Cassandra) Unlock() error {
c.isLocked = false
return nil
}
func (c *Cassandra) Run(migration io.Reader) error {
migr, err := ioutil.ReadAll(migration)
if err != nil {
return err
}
// run migration
query := string(migr[:])
if c.config.MultiStatementEnabled {
// split query by semi-colon
queries := strings.Split(query, ";")
for _, q := range queries {
tq := strings.TrimSpace(q)
if tq == "" {
continue
}
if err := c.session.Query(tq).Exec(); err != nil {
// TODO: cast to Cassandra error and get line number
return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
}
}
return nil
}
if err := c.session.Query(query).Exec(); err != nil {
// TODO: cast to Cassandra error and get line number
return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
}
return nil
}
func (c *Cassandra) SetVersion(version int, dirty bool) error {
query := `TRUNCATE "` + c.config.MigrationsTable + `"`
if err := c.session.Query(query).Exec(); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if version >= 0 {
query = `INSERT INTO "` + c.config.MigrationsTable + `" (version, dirty) VALUES (?, ?)`
if err := c.session.Query(query, version, dirty).Exec(); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
return nil
}
// Return current keyspace version
func (c *Cassandra) Version() (version int, dirty bool, err error) {
query := `SELECT version, dirty FROM "` + c.config.MigrationsTable + `" LIMIT 1`
err = c.session.Query(query).Scan(&version, &dirty)
switch {
case err == gocql.ErrNotFound:
return database.NilVersion, false, nil
case err != nil:
if _, ok := err.(*gocql.Error); ok {
return database.NilVersion, false, nil
}
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
default:
return version, dirty, nil
}
}
func (c *Cassandra) Drop() error {
// select all tables in current schema
query := fmt.Sprintf(`SELECT table_name from system_schema.tables WHERE keyspace_name='%s'`, c.config.KeyspaceName)
iter := c.session.Query(query).Iter()
var tableName string
for iter.Scan(&tableName) {
err := c.session.Query(fmt.Sprintf(`DROP TABLE %s`, tableName)).Exec()
if err != nil {
return err
}
}
// Re-create the version table
return c.ensureVersionTable()
}
// Ensure version table exists
func (c *Cassandra) ensureVersionTable() error {
err := c.session.Query(fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (version bigint, dirty boolean, PRIMARY KEY(version))", c.config.MigrationsTable)).Exec()
if err != nil {
return err
}
if _, _, err = c.Version(); err != nil {
return err
}
return nil
}
// ParseConsistency wraps gocql.ParseConsistency
// to return an error instead of a panicking.
func parseConsistency(consistencyStr string) (consistency gocql.Consistency, err error) {
defer func() {
if r := recover(); r != nil {
var ok bool
err, ok = r.(error)
if !ok {
err = fmt.Errorf("Failed to parse consistency \"%s\": %v", consistencyStr, r)
}
}
}()
consistency = gocql.ParseConsistency(consistencyStr)
return consistency, nil
}
-74
View File
@@ -1,74 +0,0 @@
package cassandra
import (
"context"
"fmt"
"strconv"
"testing"
)
import (
"github.com/dhui/dktest"
"github.com/gocql/gocql"
)
import (
dt "github.com/golang-migrate/migrate/v4/database/testing"
"github.com/golang-migrate/migrate/v4/dktesting"
)
var (
opts = dktest.Options{PortRequired: true, ReadyFunc: isReady}
// Supported versions: http://cassandra.apache.org/download/
// Although Cassandra 2.x is supported by the Apache Foundation,
// the migrate db driver only supports Cassandra 3.x since it uses
// the system_schema keyspace.
specs = []dktesting.ContainerSpec{
{ImageName: "cassandra:3.0", Options: opts},
{ImageName: "cassandra:3.11", Options: opts},
}
)
func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
// Cassandra exposes 5 ports (7000, 7001, 7199, 9042 & 9160)
// We only need the port bound to 9042
ip, portStr, err := c.Port(9042)
if err != nil {
return false
}
port, err := strconv.Atoi(portStr)
if err != nil {
return false
}
cluster := gocql.NewCluster(ip)
cluster.Port = port
cluster.Consistency = gocql.All
p, err := cluster.CreateSession()
if err != nil {
return false
}
defer p.Close()
// Create keyspace for tests
if err = p.Query("CREATE KEYSPACE testks WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor':1}").Exec(); err != nil {
return false
}
return true
}
func Test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.Port(9042)
if err != nil {
t.Fatal("Unable to get mapped port:", err)
}
addr := fmt.Sprintf("cassandra://%v:%v/testks", ip, port)
p := &Cassandra{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
dt.Test(t, d, []byte("SELECT table_name from system_schema.tables"))
})
}
-19
View File
@@ -1,19 +0,0 @@
# ClickHouse
`clickhouse://host:port?username=user&password=qwerty&database=clicks&x-multi-statement=true`
| URL Query | Description |
|------------|-------------|
| `x-migrations-table`| Name of the migrations table |
| `database` | The name of the database to connect to |
| `username` | The user to sign in as |
| `password` | The user's password |
| `host` | The host to connect to. |
| `port` | The port to bind to. |
| `x-multi-statement` | false | Enable multiple statements to be ran in a single migration (See note below) |
## Notes
* The Clickhouse driver does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
* This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
* The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
-215
View File
@@ -1,215 +0,0 @@
package clickhouse
import (
"database/sql"
"fmt"
"io"
"io/ioutil"
"net/url"
"strings"
"time"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
)
var DefaultMigrationsTable = "schema_migrations"
var ErrNilConfig = fmt.Errorf("no config")
type Config struct {
DatabaseName string
MigrationsTable string
MultiStatementEnabled bool
}
func init() {
database.Register("clickhouse", &ClickHouse{})
}
func WithInstance(conn *sql.DB, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
}
if err := conn.Ping(); err != nil {
return nil, err
}
ch := &ClickHouse{
conn: conn,
config: config,
}
if err := ch.init(); err != nil {
return nil, err
}
return ch, nil
}
type ClickHouse struct {
conn *sql.DB
config *Config
}
func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
purl, err := url.Parse(dsn)
if err != nil {
return nil, err
}
q := migrate.FilterCustomQuery(purl)
q.Scheme = "tcp"
conn, err := sql.Open("clickhouse", q.String())
if err != nil {
return nil, err
}
ch = &ClickHouse{
conn: conn,
config: &Config{
MigrationsTable: purl.Query().Get("x-migrations-table"),
DatabaseName: purl.Query().Get("database"),
MultiStatementEnabled: purl.Query().Get("x-multi-statement") == "true",
},
}
if err := ch.init(); err != nil {
return nil, err
}
return ch, nil
}
func (ch *ClickHouse) init() error {
if len(ch.config.DatabaseName) == 0 {
if err := ch.conn.QueryRow("SELECT currentDatabase()").Scan(&ch.config.DatabaseName); err != nil {
return err
}
}
if len(ch.config.MigrationsTable) == 0 {
ch.config.MigrationsTable = DefaultMigrationsTable
}
return ch.ensureVersionTable()
}
func (ch *ClickHouse) Run(r io.Reader) error {
migration, err := ioutil.ReadAll(r)
if err != nil {
return err
}
if ch.config.MultiStatementEnabled {
// split query by semi-colon
queries := strings.Split(string(migration), ";")
for _, q := range queries {
tq := strings.TrimSpace(q)
if tq == "" {
continue
}
if _, err := ch.conn.Exec(string(q)); err != nil {
return database.Error{OrigErr: err, Err: "migration failed", Query: []byte(q)}
}
}
return nil
}
if _, err := ch.conn.Exec(string(migration)); err != nil {
return database.Error{OrigErr: err, Err: "migration failed", Query: migration}
}
return nil
}
func (ch *ClickHouse) Version() (int, bool, error) {
var (
version int
dirty uint8
query = "SELECT version, dirty FROM `" + ch.config.MigrationsTable + "` ORDER BY sequence DESC LIMIT 1"
)
if err := ch.conn.QueryRow(query).Scan(&version, &dirty); err != nil {
if err == sql.ErrNoRows {
return database.NilVersion, false, nil
}
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
}
return version, dirty == 1, nil
}
func (ch *ClickHouse) SetVersion(version int, dirty bool) error {
var (
bool = func(v bool) uint8 {
if v {
return 1
}
return 0
}
tx, err = ch.conn.Begin()
)
if err != nil {
return err
}
query := "INSERT INTO " + ch.config.MigrationsTable + " (version, dirty, sequence) VALUES (?, ?, ?)"
if _, err := tx.Exec(query, version, bool(dirty), time.Now().UnixNano()); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return tx.Commit()
}
func (ch *ClickHouse) ensureVersionTable() error {
var (
table string
query = "SHOW TABLES FROM " + ch.config.DatabaseName + " LIKE '" + ch.config.MigrationsTable + "'"
)
// check if migration table exists
if err := ch.conn.QueryRow(query).Scan(&table); err != nil {
if err != sql.ErrNoRows {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
} else {
return nil
}
// if not, create the empty migration table
query = `
CREATE TABLE ` + ch.config.MigrationsTable + ` (
version Int64,
dirty UInt8,
sequence UInt64
) Engine=TinyLog
`
if _, err := ch.conn.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
func (ch *ClickHouse) Drop() error {
var (
query = "SHOW TABLES FROM " + ch.config.DatabaseName
tables, err = ch.conn.Query(query)
)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
defer tables.Close()
for tables.Next() {
var table string
if err := tables.Scan(&table); err != nil {
return err
}
query = "DROP TABLE IF EXISTS " + ch.config.DatabaseName + "." + table
if _, err := ch.conn.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
return ch.ensureVersionTable()
}
func (ch *ClickHouse) Lock() error { return nil }
func (ch *ClickHouse) Unlock() error { return nil }
func (ch *ClickHouse) Close() error { return ch.conn.Close() }
@@ -1 +0,0 @@
DROP TABLE IF EXISTS test_1;
@@ -1,3 +0,0 @@
CREATE TABLE test_1 (
Date Date
) Engine=Memory;
@@ -1 +0,0 @@
DROP TABLE IF EXISTS test_2;
@@ -1,3 +0,0 @@
CREATE TABLE test_2 (
Date Date
) Engine=Memory;
-19
View File
@@ -1,19 +0,0 @@
# cockroachdb
`cockroachdb://user:password@host:port/dbname?query` (`cockroach://`, and `crdb-postgres://` work, too)
| URL Query | WithInstance Config | Description |
|------------|---------------------|-------------|
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
| `x-lock-table` | `LockTable` | Name of the table which maintains the migration lock |
| `x-force-lock` | `ForceLock` | Force lock acquisition to fix faulty migrations which may not have released the schema lock (Boolean, default is `false`) |
| `dbname` | `DatabaseName` | The name of the database to connect to |
| `user` | | The user to sign in as |
| `password` | | The user's password |
| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
| `port` | | The port to bind to. (default is 5432) |
| `connect_timeout` | | Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. |
| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
| `sslkey` | | Key file location. The file must contain PEM encoded data. |
| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
-342
View File
@@ -1,342 +0,0 @@
package cockroachdb
import (
"context"
"database/sql"
"fmt"
"io"
"io/ioutil"
nurl "net/url"
"regexp"
"strconv"
)
import (
"github.com/cockroachdb/cockroach-go/crdb"
"github.com/lib/pq"
)
import (
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
)
func init() {
db := CockroachDb{}
database.Register("cockroach", &db)
database.Register("cockroachdb", &db)
database.Register("crdb-postgres", &db)
}
var DefaultMigrationsTable = "schema_migrations"
var DefaultLockTable = "schema_lock"
var (
ErrNilConfig = fmt.Errorf("no config")
ErrNoDatabaseName = fmt.Errorf("no database name")
)
type Config struct {
MigrationsTable string
LockTable string
ForceLock bool
DatabaseName string
}
type CockroachDb struct {
db *sql.DB
isLocked bool
// Open and WithInstance need to guarantee that config is never nil
config *Config
}
func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
}
if err := instance.Ping(); err != nil {
return nil, err
}
query := `SELECT current_database()`
var databaseName string
if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
}
if len(databaseName) == 0 {
return nil, ErrNoDatabaseName
}
config.DatabaseName = databaseName
if len(config.MigrationsTable) == 0 {
config.MigrationsTable = DefaultMigrationsTable
}
if len(config.LockTable) == 0 {
config.LockTable = DefaultLockTable
}
px := &CockroachDb{
db: instance,
config: config,
}
if err := px.ensureVersionTable(); err != nil {
return nil, err
}
if err := px.ensureLockTable(); err != nil {
return nil, err
}
return px, nil
}
func (c *CockroachDb) Open(url string) (database.Driver, error) {
purl, err := nurl.Parse(url)
if err != nil {
return nil, err
}
// As Cockroach uses the postgres protocol, and 'postgres' is already a registered database, we need to replace the
// connect prefix, with the actual protocol, so that the library can differentiate between the implementations
re := regexp.MustCompile("^(cockroach(db)?|crdb-postgres)")
connectString := re.ReplaceAllString(migrate.FilterCustomQuery(purl).String(), "postgres")
db, err := sql.Open("postgres", connectString)
if err != nil {
return nil, err
}
migrationsTable := purl.Query().Get("x-migrations-table")
if len(migrationsTable) == 0 {
migrationsTable = DefaultMigrationsTable
}
lockTable := purl.Query().Get("x-lock-table")
if len(lockTable) == 0 {
lockTable = DefaultLockTable
}
forceLockQuery := purl.Query().Get("x-force-lock")
forceLock, err := strconv.ParseBool(forceLockQuery)
if err != nil {
forceLock = false
}
px, err := WithInstance(db, &Config{
DatabaseName: purl.Path,
MigrationsTable: migrationsTable,
LockTable: lockTable,
ForceLock: forceLock,
})
if err != nil {
return nil, err
}
return px, nil
}
func (c *CockroachDb) Close() error {
return c.db.Close()
}
// Locking is done manually with a separate lock table. Implementing advisory locks in CRDB is being discussed
// See: https://github.com/cockroachdb/cockroach/issues/13546
func (c *CockroachDb) Lock() error {
err := crdb.ExecuteTx(context.Background(), c.db, nil, func(tx *sql.Tx) error {
aid, err := database.GenerateAdvisoryLockId(c.config.DatabaseName)
if err != nil {
return err
}
query := "SELECT * FROM " + c.config.LockTable + " WHERE lock_id = $1"
rows, err := tx.Query(query, aid)
if err != nil {
return database.Error{OrigErr: err, Err: "failed to fetch migration lock", Query: []byte(query)}
}
defer rows.Close()
// If row exists at all, lock is present
locked := rows.Next()
if locked && !c.config.ForceLock {
return database.ErrLocked
}
query = "INSERT INTO " + c.config.LockTable + " (lock_id) VALUES ($1)"
if _, err := tx.Exec(query, aid); err != nil {
return database.Error{OrigErr: err, Err: "failed to set migration lock", Query: []byte(query)}
}
return nil
})
if err != nil {
return err
} else {
c.isLocked = true
return nil
}
}
// Locking is done manually with a separate lock table. Implementing advisory locks in CRDB is being discussed
// See: https://github.com/cockroachdb/cockroach/issues/13546
func (c *CockroachDb) Unlock() error {
aid, err := database.GenerateAdvisoryLockId(c.config.DatabaseName)
if err != nil {
return err
}
// In the event of an implementation (non-migration) error, it is possible for the lock to not be released. Until
// a better locking mechanism is added, a manual purging of the lock table may be required in such circumstances
query := "DELETE FROM " + c.config.LockTable + " WHERE lock_id = $1"
if _, err := c.db.Exec(query, aid); err != nil {
if e, ok := err.(*pq.Error); ok {
// 42P01 is "UndefinedTableError" in CockroachDB
// https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/pgwire/pgerror/codes.go
if e.Code == "42P01" {
// On drops, the lock table is fully removed; This is fine, and is a valid "unlocked" state for the schema
c.isLocked = false
return nil
}
}
return database.Error{OrigErr: err, Err: "failed to release migration lock", Query: []byte(query)}
}
c.isLocked = false
return nil
}
func (c *CockroachDb) Run(migration io.Reader) error {
migr, err := ioutil.ReadAll(migration)
if err != nil {
return err
}
// run migration
query := string(migr[:])
if _, err := c.db.Exec(query); err != nil {
return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
}
return nil
}
func (c *CockroachDb) SetVersion(version int, dirty bool) error {
return crdb.ExecuteTx(context.Background(), c.db, nil, func(tx *sql.Tx) error {
if _, err := tx.Exec(`DELETE FROM "` + c.config.MigrationsTable + `"`); err != nil {
return err
}
if version >= 0 {
if _, err := tx.Exec(`INSERT INTO "`+c.config.MigrationsTable+`" (version, dirty) VALUES ($1, $2)`, version, dirty); err != nil {
return err
}
}
return nil
})
}
func (c *CockroachDb) Version() (version int, dirty bool, err error) {
query := `SELECT version, dirty FROM "` + c.config.MigrationsTable + `" LIMIT 1`
err = c.db.QueryRow(query).Scan(&version, &dirty)
switch {
case err == sql.ErrNoRows:
return database.NilVersion, false, nil
case err != nil:
if e, ok := err.(*pq.Error); ok {
// 42P01 is "UndefinedTableError" in CockroachDB
// https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/pgwire/pgerror/codes.go
if e.Code == "42P01" {
return database.NilVersion, false, nil
}
}
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
default:
return version, dirty, nil
}
}
func (c *CockroachDb) Drop() error {
// select all tables in current schema
query := `SELECT table_name FROM information_schema.tables WHERE table_schema=(SELECT current_schema())`
tables, err := c.db.Query(query)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
defer tables.Close()
// delete one table after another
tableNames := make([]string, 0)
for tables.Next() {
var tableName string
if err := tables.Scan(&tableName); err != nil {
return err
}
if len(tableName) > 0 {
tableNames = append(tableNames, tableName)
}
}
if len(tableNames) > 0 {
// delete one by one ...
for _, t := range tableNames {
query = `DROP TABLE IF EXISTS ` + t + ` CASCADE`
if _, err := c.db.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := c.ensureVersionTable(); err != nil {
return err
}
}
return nil
}
func (c *CockroachDb) ensureVersionTable() error {
// check if migration table exists
var count int
query := `SELECT COUNT(1) FROM information_schema.tables WHERE table_name = $1 AND table_schema = (SELECT current_schema()) LIMIT 1`
if err := c.db.QueryRow(query, c.config.MigrationsTable).Scan(&count); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if count == 1 {
return nil
}
// if not, create the empty migration table
query = `CREATE TABLE "` + c.config.MigrationsTable + `" (version INT NOT NULL PRIMARY KEY, dirty BOOL NOT NULL)`
if _, err := c.db.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
func (c *CockroachDb) ensureLockTable() error {
// check if lock table exists
var count int
query := `SELECT COUNT(1) FROM information_schema.tables WHERE table_name = $1 AND table_schema = (SELECT current_schema()) LIMIT 1`
if err := c.db.QueryRow(query, c.config.LockTable).Scan(&count); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if count == 1 {
return nil
}
// if not, create the empty lock table
query = `CREATE TABLE "` + c.config.LockTable + `" (lock_id INT NOT NULL PRIMARY KEY)`
if _, err := c.db.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
-141
View File
@@ -1,141 +0,0 @@
package cockroachdb
// error codes https://github.com/lib/pq/blob/master/error.go
import (
"context"
"database/sql"
"fmt"
"strings"
"testing"
)
import (
"github.com/dhui/dktest"
_ "github.com/lib/pq"
)
import (
dt "github.com/golang-migrate/migrate/v4/database/testing"
"github.com/golang-migrate/migrate/v4/dktesting"
)
const defaultPort = 26257
var (
opts = dktest.Options{Cmd: []string{"start", "--insecure"}, PortRequired: true, ReadyFunc: isReady}
// Released versions: https://www.cockroachlabs.com/docs/releases/
specs = []dktesting.ContainerSpec{
{ImageName: "cockroachdb/cockroach:v1.0.7", Options: opts},
{ImageName: "cockroachdb/cockroach:v1.1.9", Options: opts},
{ImageName: "cockroachdb/cockroach:v2.0.7", Options: opts},
{ImageName: "cockroachdb/cockroach:v2.1.3", Options: opts},
}
)
func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
ip, port, err := c.Port(defaultPort)
if err != nil {
fmt.Println("port error:", err)
return false
}
db, err := sql.Open("postgres", fmt.Sprintf("postgres://root@%v:%v?sslmode=disable", ip, port))
if err != nil {
fmt.Println("open error:", err)
return false
}
if err := db.PingContext(ctx); err != nil {
fmt.Println("ping error:", err)
return false
}
db.Close()
return true
}
func createDB(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.Port(defaultPort)
if err != nil {
t.Fatal(err)
}
db, err := sql.Open("postgres", fmt.Sprintf("postgres://root@%v:%v?sslmode=disable", ip, port))
if err != nil {
t.Fatal(err)
}
if err = db.Ping(); err != nil {
t.Fatal(err)
}
defer db.Close()
if _, err = db.Exec("CREATE DATABASE migrate"); err != nil {
t.Fatal(err)
}
}
func Test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
ip, port, err := ci.Port(26257)
if err != nil {
t.Fatal(err)
}
addr := fmt.Sprintf("cockroach://root@%v:%v/migrate?sslmode=disable", ip, port)
c := &CockroachDb{}
d, err := c.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
dt.Test(t, d, []byte("SELECT 1"))
})
}
func TestMultiStatement(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
ip, port, err := ci.Port(26257)
if err != nil {
t.Fatal(err)
}
addr := fmt.Sprintf("cockroach://root@%v:%v/migrate?sslmode=disable", ip, port)
c := &CockroachDb{}
d, err := c.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
if err := d.Run(strings.NewReader("CREATE TABLE foo (foo text); CREATE TABLE bar (bar text);")); err != nil {
t.Fatalf("expected err to be nil, got %v", err)
}
// make sure second table exists
var exists bool
if err := d.(*CockroachDb).db.QueryRow("SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'bar' AND table_schema = (SELECT current_schema()))").Scan(&exists); err != nil {
t.Fatal(err)
}
if !exists {
t.Fatalf("expected table bar to exist")
}
})
}
func TestFilterCustomQuery(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, ci dktest.ContainerInfo) {
createDB(t, ci)
ip, port, err := ci.Port(26257)
if err != nil {
t.Fatal(err)
}
addr := fmt.Sprintf("cockroach://root@%v:%v/migrate?sslmode=disable&x-custom=foobar", ip, port)
c := &CockroachDb{}
_, err = c.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
})
}
@@ -1 +0,0 @@
DROP TABLE IF EXISTS users;
@@ -1,5 +0,0 @@
CREATE TABLE users (
user_id INT UNIQUE,
name STRING(40),
email STRING(40)
);
@@ -1 +0,0 @@
ALTER TABLE users DROP COLUMN IF EXISTS city;
@@ -1 +0,0 @@
ALTER TABLE users ADD COLUMN city TEXT;
@@ -1 +0,0 @@
DROP INDEX IF EXISTS users_email_index;
@@ -1,3 +0,0 @@
CREATE UNIQUE INDEX IF NOT EXISTS users_email_index ON users (email);
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
DROP TABLE IF EXISTS books;
@@ -1,5 +0,0 @@
CREATE TABLE books (
user_id INT,
name STRING(40),
author STRING(40)
);
@@ -1 +0,0 @@
DROP TABLE IF EXISTS movies;
@@ -1,5 +0,0 @@
CREATE TABLE movies (
user_id INT,
name STRING(40),
director STRING(40)
);
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
View File
-124
View File
@@ -1,124 +0,0 @@
// Package database provides the Database interface.
// All database drivers must implement this interface, register themselves,
// optionally provide a `WithInstance` function and pass the tests
// in package database/testing.
package database
import (
"fmt"
"io"
nurl "net/url"
"sync"
)
var (
ErrLocked = fmt.Errorf("can't acquire lock")
)
const NilVersion int = -1
var driversMu sync.RWMutex
var drivers = make(map[string]Driver)
// Driver is the interface every database driver must implement.
//
// How to implement a database driver?
// 1. Implement this interface.
// 2. Optionally, add a function named `WithInstance`.
// This function should accept an existing DB instance and a Config{} struct
// and return a driver instance.
// 3. Add a test that calls database/testing.go:Test()
// 4. Add own tests for Open(), WithInstance() (when provided) and Close().
// All other functions are tested by tests in database/testing.
// Saves you some time and makes sure all database drivers behave the same way.
// 5. Call Register in init().
// 6. Create a migrate/cli/build_<driver-name>.go file
// 7. Add driver name in 'DATABASE' variable in Makefile
//
// Guidelines:
// * Don't try to correct user input. Don't assume things.
// When in doubt, return an error and explain the situation to the user.
// * All configuration input must come from the URL string in func Open()
// or the Config{} struct in WithInstance. Don't os.Getenv().
type Driver interface {
// Open returns a new driver instance configured with parameters
// coming from the URL string. Migrate will call this function
// only once per instance.
Open(url string) (Driver, error)
// Close closes the underlying database instance managed by the driver.
// Migrate will call this function only once per instance.
Close() error
// Lock should acquire a database lock so that only one migration process
// can run at a time. Migrate will call this function before Run is called.
// If the implementation can't provide this functionality, return nil.
// Return database.ErrLocked if database is already locked.
Lock() error
// Unlock should release the lock. Migrate will call this function after
// all migrations have been run.
Unlock() error
// Run applies a migration to the database. migration is garantueed to be not nil.
Run(migration io.Reader) error
// SetVersion saves version and dirty state.
// Migrate will call this function before and after each call to Run.
// version must be >= -1. -1 means NilVersion.
SetVersion(version int, dirty bool) error
// Version returns the currently active version and if the database is dirty.
// When no migration has been applied, it must return version -1.
// Dirty means, a previous migration failed and user interaction is required.
Version() (version int, dirty bool, err error)
// Drop deletes everything in the database.
Drop() error
}
// Open returns a new driver instance.
func Open(url string) (Driver, error) {
u, err := nurl.Parse(url)
if err != nil {
return nil, fmt.Errorf("Unable to parse URL. Did you escape all reserved URL characters? "+
"See: https://github.com/golang-migrate/migrate#database-urls Error: %v", err)
}
if u.Scheme == "" {
return nil, fmt.Errorf("database driver: invalid URL scheme")
}
driversMu.RLock()
d, ok := drivers[u.Scheme]
driversMu.RUnlock()
if !ok {
return nil, fmt.Errorf("database driver: unknown driver %v (forgotten import?)", u.Scheme)
}
return d.Open(url)
}
// Register globally registers a driver.
func Register(name string, driver Driver) {
driversMu.Lock()
defer driversMu.Unlock()
if driver == nil {
panic("Register driver is nil")
}
if _, dup := drivers[name]; dup {
panic("Register called twice for driver " + name)
}
drivers[name] = driver
}
// List lists the registered drivers
func List() []string {
driversMu.RLock()
defer driversMu.RUnlock()
names := make([]string, 0, len(drivers))
for n := range drivers {
names = append(names, n)
}
return names
}
-8
View File
@@ -1,8 +0,0 @@
package database
func ExampleDriver() {
// see database/stub for an example
// database/stub/stub.go has the driver implementation
// database/stub/stub_test.go runs database/testing/test.go:Test
}
-27
View File
@@ -1,27 +0,0 @@
package database
import (
"fmt"
)
// Error should be used for errors involving queries ran against the database
type Error struct {
// Optional: the line number
Line uint
// Query is a query excerpt
Query []byte
// Err is a useful/helping error message for humans
Err string
// OrigErr is the underlying error
OrigErr error
}
func (e Error) Error() string {
if len(e.Err) == 0 {
return fmt.Sprintf("%v in line %v: %s", e.OrigErr, e.Line, e.Query)
}
return fmt.Sprintf("%v in line %v: %s (details: %v)", e.Err, e.Line, e.Query, e.OrigErr)
}
-20
View File
@@ -1,20 +0,0 @@
# MongoDB
* Driver work with mongo through [db.runCommands](https://docs.mongodb.com/manual/reference/command/)
* Migrations support json format. It contains array of commands for `db.runCommand`. Every command is executed in separate request to database
* All keys have to be in quotes `"`
* [Examples](./examples)
# Usage
`mongodb://user:password@host:port/dbname?query`
| URL Query | WithInstance Config | Description |
|------------|---------------------|-------------|
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
| `x-transaction-mode` | `TransactionMode` | If set to `true` wrap commands in [transaction](https://docs.mongodb.com/manual/core/transactions). Available only for replica set. Driver is using [strconv.ParseBool](https://golang.org/pkg/strconv/#ParseBool) for parsing|
| `dbname` | `DatabaseName` | The name of the database to connect to |
| `user` | | The user to sign in as. Can be omitted |
| `password` | | The user's password. Can be omitted |
| `host` | | The host to connect to |
| `port` | | The port to bind to |
@@ -1,5 +0,0 @@
[
{
"dropUser": "deminem"
}
]
@@ -1,12 +0,0 @@
[
{
"createUser": "deminem",
"pwd": "gogo",
"roles": [
{
"role": "readWrite",
"db": "testMigration"
}
]
}
]
@@ -1,10 +0,0 @@
[
{
"dropIndexes": "mycollection",
"index": "username_sort_by_asc_created"
},
{
"dropIndexes": "mycollection",
"index": "unique_email"
}
]
@@ -1,21 +0,0 @@
[{
"createIndexes": "mycollection",
"indexes": [
{
"key": {
"username": 1,
"created": -1
},
"name": "username_sort_by_asc_created",
"background": true
},
{
"key": {
"email": 1
},
"name": "unique_email",
"unique": true,
"background": true
}
]
}]
-199
View File
@@ -1,199 +0,0 @@
package mongodb
import (
"context"
"fmt"
"io"
"io/ioutil"
"net/url"
"strconv"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
"github.com/mongodb/mongo-go-driver/bson"
"github.com/mongodb/mongo-go-driver/mongo"
"github.com/mongodb/mongo-go-driver/x/network/connstring"
)
func init() {
database.Register("mongodb", &Mongo{})
}
var DefaultMigrationsCollection = "schema_migrations"
var (
ErrNoDatabaseName = fmt.Errorf("no database name")
ErrNilConfig = fmt.Errorf("no config")
)
type Mongo struct {
client *mongo.Client
db *mongo.Database
config *Config
}
type Config struct {
DatabaseName string
MigrationsCollection string
TransactionMode bool
}
type versionInfo struct {
Version int `bson:"version"`
Dirty bool `bson:"dirty"`
}
func WithInstance(instance *mongo.Client, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
}
if len(config.DatabaseName) == 0 {
return nil, ErrNoDatabaseName
}
if len(config.MigrationsCollection) == 0 {
config.MigrationsCollection = DefaultMigrationsCollection
}
mc := &Mongo{
client: instance,
db: instance.Database(config.DatabaseName),
config: config,
}
return mc, nil
}
func (m *Mongo) Open(dsn string) (database.Driver, error) {
//connsting is experimental package, but it used for parse connection string in mongo.Connect function
uri, err := connstring.Parse(dsn)
if err != nil {
return nil, err
}
if len(uri.Database) == 0 {
return nil, ErrNoDatabaseName
}
purl, err := url.Parse(dsn)
if err != nil {
return nil, err
}
migrationsCollection := purl.Query().Get("x-migrations-collection")
if len(migrationsCollection) == 0 {
migrationsCollection = DefaultMigrationsCollection
}
transactionMode, _ := strconv.ParseBool(purl.Query().Get("x-transaction-mode"))
q := migrate.FilterCustomQuery(purl)
q.Scheme = "mongodb"
client, err := mongo.Connect(context.TODO(), q.String())
if err != nil {
return nil, err
}
if err = client.Ping(context.TODO(), nil); err != nil {
return nil, err
}
mc, err := WithInstance(client, &Config{
DatabaseName: uri.Database,
MigrationsCollection: migrationsCollection,
TransactionMode: transactionMode,
})
if err != nil {
return nil, err
}
return mc, nil
}
func (m *Mongo) SetVersion(version int, dirty bool) error {
migrationsCollection := m.db.Collection(m.config.MigrationsCollection)
if err := migrationsCollection.Drop(context.TODO()); err != nil {
return &database.Error{OrigErr: err, Err: "drop migrations collection failed"}
}
_, err := migrationsCollection.InsertOne(context.TODO(), bson.M{"version": version, "dirty": dirty})
if err != nil {
return &database.Error{OrigErr: err, Err: "save version failed"}
}
return nil
}
func (m *Mongo) Version() (version int, dirty bool, err error) {
var versionInfo versionInfo
err = m.db.Collection(m.config.MigrationsCollection).FindOne(context.TODO(), nil).Decode(&versionInfo)
switch {
case err == mongo.ErrNoDocuments:
return database.NilVersion, false, nil
case err != nil:
return 0, false, &database.Error{OrigErr: err, Err: "failed to get migration version"}
default:
return versionInfo.Version, versionInfo.Dirty, nil
}
}
func (m *Mongo) Run(migration io.Reader) error {
migr, err := ioutil.ReadAll(migration)
if err != nil {
return err
}
var cmds []bson.D
err = bson.UnmarshalExtJSON(migr, true, &cmds)
if err != nil {
return fmt.Errorf("unmarshaling json error: %s", err)
}
if m.config.TransactionMode {
if err := m.executeCommandsWithTransaction(context.TODO(), cmds); err != nil {
return err
}
} else {
if err := m.executeCommands(context.TODO(), cmds); err != nil {
return err
}
}
return nil
}
func (m *Mongo) executeCommandsWithTransaction(ctx context.Context, cmds []bson.D) error {
err := m.db.Client().UseSession(ctx, func(sessionContext mongo.SessionContext) error {
if err := sessionContext.StartTransaction(); err != nil {
return &database.Error{OrigErr: err, Err: "failed to start transaction"}
}
if err := m.executeCommands(sessionContext, cmds); err != nil {
//When command execution is failed, it's aborting transaction
//If you tried to call abortTransaction, it`s return error that transaction already aborted
return err
}
if err := sessionContext.CommitTransaction(sessionContext); err != nil {
return &database.Error{OrigErr: err, Err: "failed to commit transaction"}
}
return nil
})
if err != nil {
return err
}
return nil
}
func (m *Mongo) executeCommands(ctx context.Context, cmds []bson.D) error {
for _, cmd := range cmds {
err := m.db.RunCommand(ctx, cmd).Err()
if err != nil {
return &database.Error{OrigErr: err, Err: fmt.Sprintf("failed to execute command:%v", cmd)}
}
}
return nil
}
func (m *Mongo) Close() error {
return m.client.Disconnect(context.TODO())
}
func (m *Mongo) Drop() error {
return m.db.Drop(context.TODO())
}
func (m *Mongo) Lock() error {
return nil
}
func (m *Mongo) Unlock() error {
return nil
}
-293
View File
@@ -1,293 +0,0 @@
package mongodb
import (
"bytes"
"context"
"fmt"
"io"
"os"
"strconv"
"testing"
"time"
)
import (
"github.com/dhui/dktest"
"github.com/mongodb/mongo-go-driver/bson"
"github.com/mongodb/mongo-go-driver/mongo"
)
import (
dt "github.com/golang-migrate/migrate/v4/database/testing"
"github.com/golang-migrate/migrate/v4/dktesting"
)
var (
opts = dktest.Options{PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.mongodb.com/support-policy
specs = []dktesting.ContainerSpec{
{ImageName: "mongo:3.4", Options: opts},
{ImageName: "mongo:3.6", Options: opts},
{ImageName: "mongo:4.0", Options: opts},
}
)
func mongoConnectionString(host, port string) string {
// there is connect option for excluding serverConnection algorithm
// it's let avoid errors with mongo replica set connection in docker container
return fmt.Sprintf("mongodb://%s:%s/testMigration?connect=single", host, port)
}
func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
ip, port, err := c.FirstPort()
if err != nil {
return false
}
client, err := mongo.Connect(ctx, mongoConnectionString(ip, port))
if err != nil {
return false
}
defer client.Disconnect(ctx)
if err = client.Ping(ctx, nil); err != nil {
switch err {
case io.EOF:
return false
default:
fmt.Println(err)
}
return false
}
return true
}
func Test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := mongoConnectionString(ip, port)
p := &Mongo{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
dt.TestNilVersion(t, d)
//TestLockAndUnlock(t, d) driver doesn't support lock on database level
dt.TestRun(t, d, bytes.NewReader([]byte(`[{"insert":"hello","documents":[{"wild":"world"}]}]`)))
dt.TestSetVersion(t, d)
dt.TestDrop(t, d)
})
}
func TestWithAuth(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := mongoConnectionString(ip, port)
p := &Mongo{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
createUserCMD := []byte(`[{"createUser":"deminem","pwd":"gogo","roles":[{"role":"readWrite","db":"testMigration"}]}]`)
err = d.Run(bytes.NewReader(createUserCMD))
if err != nil {
t.Fatalf("%v", err)
}
testcases := []struct {
name string
connectUri string
isErrorExpected bool
}{
{"right auth data", "mongodb://deminem:gogo@%s:%v/testMigration", false},
{"wrong auth data", "mongodb://wrong:auth@%s:%v/testMigration", true},
}
insertCMD := []byte(`[{"insert":"hello","documents":[{"wild":"world"}]}]`)
for _, tcase := range testcases {
//With wrong authenticate `Open` func doesn't return auth error
//Because at the moment golang mongo driver doesn't support auth during connection
//For getting auth error we should execute database command
t.Run(tcase.name, func(t *testing.T) {
mc := &Mongo{}
d, err := mc.Open(fmt.Sprintf(tcase.connectUri, ip, port))
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
err = d.Run(bytes.NewReader(insertCMD))
switch {
case tcase.isErrorExpected && err == nil:
t.Fatalf("no error when expected")
case !tcase.isErrorExpected && err != nil:
t.Fatalf("unexpected error: %v", err)
}
})
}
})
}
func TestTransaction(t *testing.T) {
transactionSpecs := []dktesting.ContainerSpec{
{ImageName: "mongo:4", Options: dktest.Options{PortRequired: true, ReadyFunc: isReady,
Cmd: []string{"mongod", "--bind_ip_all", "--replSet", "rs0"}}},
}
dktesting.ParallelTest(t, transactionSpecs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
client, err := mongo.Connect(context.TODO(), mongoConnectionString(ip, port))
if err != nil {
t.Fatalf("%v", err)
}
err = client.Ping(context.TODO(), nil)
if err != nil {
t.Fatalf("%v", err)
}
//rs.initiate()
err = client.Database("admin").RunCommand(context.TODO(), bson.D{{"replSetInitiate", bson.D{}}}).Err()
if err != nil {
t.Fatalf("%v", err)
}
err = waitForReplicaInit(client)
if err != nil {
t.Fatalf("%v", err)
}
d, err := WithInstance(client, &Config{
DatabaseName: "testMigration",
})
defer d.Close()
//We have to create collection
//transactions don't support operations with creating new dbs, collections
//Unique index need for checking transaction aborting
insertCMD := []byte(`[
{"create":"hello"},
{"createIndexes": "hello",
"indexes": [{
"key": {
"wild": 1
},
"name": "unique_wild",
"unique": true,
"background": true
}]
}]`)
err = d.Run(bytes.NewReader(insertCMD))
if err != nil {
t.Fatalf("%v", err)
}
testcases := []struct {
name string
cmds []byte
documentsCount int64
isErrorExpected bool
}{
{
name: "success transaction",
cmds: []byte(`[{"insert":"hello","documents":[
{"wild":"world"},
{"wild":"west"},
{"wild":"natural"}
]
}]`),
documentsCount: 3,
isErrorExpected: false,
},
{
name: "failure transaction",
//transaction have to be failure - duplicate unique key wild:west
//none of the documents should be added
cmds: []byte(`[{"insert":"hello","documents":[{"wild":"flower"}]},
{"insert":"hello","documents":[
{"wild":"cat"},
{"wild":"west"}
]
}]`),
documentsCount: 3,
isErrorExpected: true,
},
}
for _, tcase := range testcases {
t.Run(tcase.name, func(t *testing.T) {
client, err := mongo.Connect(context.TODO(), mongoConnectionString(ip, port))
if err != nil {
t.Fatalf("%v", err)
}
err = client.Ping(context.TODO(), nil)
if err != nil {
t.Fatalf("%v", err)
}
d, err := WithInstance(client, &Config{
DatabaseName: "testMigration",
TransactionMode: true,
})
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
runErr := d.Run(bytes.NewReader(tcase.cmds))
if runErr != nil {
if !tcase.isErrorExpected {
t.Fatalf("%v", runErr)
}
}
documentsCount, err := client.Database("testMigration").Collection("hello").Count(context.TODO(), nil)
if err != nil {
t.Fatalf("%v", err)
}
if tcase.documentsCount != documentsCount {
t.Fatalf("expected %d and actual %d documents count not equal. run migration error:%s", tcase.documentsCount, documentsCount, runErr)
}
})
}
})
}
type isMaster struct {
IsMaster bool `bson:"ismaster"`
}
func waitForReplicaInit(client *mongo.Client) error {
ticker := time.NewTicker(time.Second * 1)
defer ticker.Stop()
timeout, err := strconv.Atoi(os.Getenv("MIGRATE_TEST_MONGO_REPLICA_SET_INIT_TIMEOUT"))
if err != nil {
timeout = 30
}
timeoutTimer := time.NewTimer(time.Duration(timeout) * time.Second)
defer timeoutTimer.Stop()
for {
select {
case <-ticker.C:
var status isMaster
//Check that node is primary because
//during replica set initialization, the first node first becomes a secondary and then becomes the primary
//should consider that initialization is completed only after the node has become the primary
result := client.Database("admin").RunCommand(context.TODO(), bson.D{{"isMaster", 1}})
r, err := result.DecodeBytes()
if err != nil {
return err
}
err = bson.Unmarshal(r, &status)
if err != nil {
return err
}
if status.IsMaster {
return nil
}
case <-timeoutTimer.C:
return fmt.Errorf("replica init timeout")
}
}
}
-54
View File
@@ -1,54 +0,0 @@
# MySQL
`mysql://user:password@tcp(host:port)/dbname?query`
| URL Query | WithInstance Config | Description |
|------------|---------------------|-------------|
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
| `dbname` | `DatabaseName` | The name of the database to connect to |
| `user` | | The user to sign in as |
| `password` | | The user's password |
| `host` | | The host to connect to. |
| `port` | | The port to bind to. |
| `tls` | | TLS / SSL encrypted connection parameter; see [go-sql-driver](https://github.com/go-sql-driver/mysql#tls). Use any name (e.g. `migrate`) if you want to use a custom TLS config (`x-tls-` queries). |
| `x-tls-ca` | | The location of the CA (certificate authority) file. |
| `x-tls-cert` | | The location of the client certicicate file. Must be used with `x-tls-key`. |
| `x-tls-key` | | The location of the private key file. Must be used with `x-tls-cert`. |
| `x-tls-insecure-skip-verify` | | Whether or not to use SSL (true\|false) |
## Use with existing client
If you use the MySQL driver with existing database client, you must create the client with parameter `multiStatements=true`:
```go
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"github.com/golang-migrate/migrate"
"github.com/golang-migrate/migrate/database/mysql"
_ "github.com/golang-migrate/migrate/source/file"
)
func main() {
db, _ := sql.Open("mysql", "user:password@tcp(host:port)/dbname?multiStatements=true")
driver, _ := mysql.WithInstance(db, &mysql.Config{})
m, _ := migrate.NewWithDatabaseInstance(
"file:///migrations",
"mysql",
driver,
)
m.Steps(2)
}
```
## Upgrading from v1
1. Write down the current migration version from schema_migrations
1. `DROP TABLE schema_migrations`
2. Wrap your existing migrations in transactions ([BEGIN/COMMIT](https://dev.mysql.com/doc/refman/5.7/en/commit.html)) if you use multiple statements within one migration.
3. Download and install the latest migrate version.
4. Force the current migration version with `migrate force <current_version>`.
-386
View File
@@ -1,386 +0,0 @@
// +build go1.9
package mysql
import (
"context"
"crypto/tls"
"crypto/x509"
"database/sql"
"fmt"
"io"
"io/ioutil"
nurl "net/url"
"strconv"
"strings"
)
import (
"github.com/go-sql-driver/mysql"
)
import (
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
)
func init() {
database.Register("mysql", &Mysql{})
}
var DefaultMigrationsTable = "schema_migrations"
var (
ErrDatabaseDirty = fmt.Errorf("database is dirty")
ErrNilConfig = fmt.Errorf("no config")
ErrNoDatabaseName = fmt.Errorf("no database name")
ErrAppendPEM = fmt.Errorf("failed to append PEM")
ErrTLSCertKeyConfig = fmt.Errorf("To use TLS client authentication, both x-tls-cert and x-tls-key must not be empty")
)
type Config struct {
MigrationsTable string
DatabaseName string
}
type Mysql struct {
// mysql RELEASE_LOCK must be called from the same conn, so
// just do everything over a single conn anyway.
conn *sql.Conn
db *sql.DB
isLocked bool
config *Config
}
// instance must have `multiStatements` set to true
func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
}
if err := instance.Ping(); err != nil {
return nil, err
}
query := `SELECT DATABASE()`
var databaseName sql.NullString
if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
}
if len(databaseName.String) == 0 {
return nil, ErrNoDatabaseName
}
config.DatabaseName = databaseName.String
if len(config.MigrationsTable) == 0 {
config.MigrationsTable = DefaultMigrationsTable
}
conn, err := instance.Conn(context.Background())
if err != nil {
return nil, err
}
mx := &Mysql{
conn: conn,
db: instance,
config: config,
}
if err := mx.ensureVersionTable(); err != nil {
return nil, err
}
return mx, nil
}
// urlToMySQLConfig takes a net/url URL and returns a go-sql-driver/mysql Config.
// Manually sets username and password to avoid net/url from url-encoding the reserved URL characters
func urlToMySQLConfig(u nurl.URL) (*mysql.Config, error) {
origUserInfo := u.User
u.User = nil
c, err := mysql.ParseDSN(strings.TrimPrefix(u.String(), "mysql://"))
if err != nil {
return nil, err
}
if origUserInfo != nil {
c.User = origUserInfo.Username()
if p, ok := origUserInfo.Password(); ok {
c.Passwd = p
}
}
return c, nil
}
func (m *Mysql) Open(url string) (database.Driver, error) {
purl, err := nurl.Parse(url)
if err != nil {
return nil, err
}
q := purl.Query()
q.Set("multiStatements", "true")
purl.RawQuery = q.Encode()
migrationsTable := purl.Query().Get("x-migrations-table")
if len(migrationsTable) == 0 {
migrationsTable = DefaultMigrationsTable
}
// use custom TLS?
ctls := purl.Query().Get("tls")
if len(ctls) > 0 {
if _, isBool := readBool(ctls); !isBool && strings.ToLower(ctls) != "skip-verify" {
rootCertPool := x509.NewCertPool()
pem, err := ioutil.ReadFile(purl.Query().Get("x-tls-ca"))
if err != nil {
return nil, err
}
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
return nil, ErrAppendPEM
}
clientCert := make([]tls.Certificate, 0, 1)
if ccert, ckey := purl.Query().Get("x-tls-cert"), purl.Query().Get("x-tls-key"); ccert != "" || ckey != "" {
if ccert == "" || ckey == "" {
return nil, ErrTLSCertKeyConfig
}
certs, err := tls.LoadX509KeyPair(ccert, ckey)
if err != nil {
return nil, err
}
clientCert = append(clientCert, certs)
}
insecureSkipVerify := false
if len(purl.Query().Get("x-tls-insecure-skip-verify")) > 0 {
x, err := strconv.ParseBool(purl.Query().Get("x-tls-insecure-skip-verify"))
if err != nil {
return nil, err
}
insecureSkipVerify = x
}
mysql.RegisterTLSConfig(ctls, &tls.Config{
RootCAs: rootCertPool,
Certificates: clientCert,
InsecureSkipVerify: insecureSkipVerify,
})
}
}
c, err := urlToMySQLConfig(*migrate.FilterCustomQuery(purl))
if err != nil {
return nil, err
}
db, err := sql.Open("mysql", c.FormatDSN())
if err != nil {
return nil, err
}
mx, err := WithInstance(db, &Config{
DatabaseName: purl.Path,
MigrationsTable: migrationsTable,
})
if err != nil {
return nil, err
}
return mx, nil
}
func (m *Mysql) Close() error {
connErr := m.conn.Close()
dbErr := m.db.Close()
if connErr != nil || dbErr != nil {
return fmt.Errorf("conn: %v, db: %v", connErr, dbErr)
}
return nil
}
func (m *Mysql) Lock() error {
if m.isLocked {
return database.ErrLocked
}
aid, err := database.GenerateAdvisoryLockId(
fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable))
if err != nil {
return err
}
query := "SELECT GET_LOCK(?, 10)"
var success bool
if err := m.conn.QueryRowContext(context.Background(), query, aid).Scan(&success); err != nil {
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
}
if success {
m.isLocked = true
return nil
}
return database.ErrLocked
}
func (m *Mysql) Unlock() error {
if !m.isLocked {
return nil
}
aid, err := database.GenerateAdvisoryLockId(
fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable))
if err != nil {
return err
}
query := `SELECT RELEASE_LOCK(?)`
if _, err := m.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
// NOTE: RELEASE_LOCK could return NULL or (or 0 if the code is changed),
// in which case isLocked should be true until the timeout expires -- synchronizing
// these states is likely not worth trying to do; reconsider the necessity of isLocked.
m.isLocked = false
return nil
}
func (m *Mysql) Run(migration io.Reader) error {
migr, err := ioutil.ReadAll(migration)
if err != nil {
return err
}
query := string(migr[:])
if _, err := m.conn.ExecContext(context.Background(), query); err != nil {
return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
}
return nil
}
func (m *Mysql) SetVersion(version int, dirty bool) error {
tx, err := m.conn.BeginTx(context.Background(), &sql.TxOptions{})
if err != nil {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
query := "TRUNCATE `" + m.config.MigrationsTable + "`"
if _, err := tx.ExecContext(context.Background(), query); err != nil {
tx.Rollback()
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if version >= 0 {
query := "INSERT INTO `" + m.config.MigrationsTable + "` (version, dirty) VALUES (?, ?)"
if _, err := tx.ExecContext(context.Background(), query, version, dirty); err != nil {
tx.Rollback()
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := tx.Commit(); err != nil {
return &database.Error{OrigErr: err, Err: "transaction commit failed"}
}
return nil
}
func (m *Mysql) Version() (version int, dirty bool, err error) {
query := "SELECT version, dirty FROM `" + m.config.MigrationsTable + "` LIMIT 1"
err = m.conn.QueryRowContext(context.Background(), query).Scan(&version, &dirty)
switch {
case err == sql.ErrNoRows:
return database.NilVersion, false, nil
case err != nil:
if e, ok := err.(*mysql.MySQLError); ok {
if e.Number == 0 {
return database.NilVersion, false, nil
}
}
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
default:
return version, dirty, nil
}
}
func (m *Mysql) Drop() error {
// select all tables
query := `SHOW TABLES LIKE '%'`
tables, err := m.conn.QueryContext(context.Background(), query)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
defer tables.Close()
// delete one table after another
tableNames := make([]string, 0)
for tables.Next() {
var tableName string
if err := tables.Scan(&tableName); err != nil {
return err
}
if len(tableName) > 0 {
tableNames = append(tableNames, tableName)
}
}
if len(tableNames) > 0 {
// delete one by one ...
for _, t := range tableNames {
query = "DROP TABLE IF EXISTS `" + t + "` CASCADE"
if _, err := m.conn.ExecContext(context.Background(), query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := m.ensureVersionTable(); err != nil {
return err
}
}
return nil
}
func (m *Mysql) ensureVersionTable() error {
// check if migration table exists
var result string
query := `SHOW TABLES LIKE "` + m.config.MigrationsTable + `"`
if err := m.conn.QueryRowContext(context.Background(), query).Scan(&result); err != nil {
if err != sql.ErrNoRows {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
} else {
return nil
}
// if not, create the empty migration table
query = "CREATE TABLE `" + m.config.MigrationsTable + "` (version bigint not null primary key, dirty boolean not null)"
if _, err := m.conn.ExecContext(context.Background(), query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
// Returns the bool value of the input.
// The 2nd return value indicates if the input was a valid bool value
// See https://github.com/go-sql-driver/mysql/blob/a059889267dc7170331388008528b3b44479bffb/utils.go#L71
func readBool(input string) (value bool, valid bool) {
switch input {
case "1", "true", "TRUE", "True":
return true, true
case "0", "false", "FALSE", "False":
return false, true
}
// Not a valid bool value
return
}
-179
View File
@@ -1,179 +0,0 @@
package mysql
import (
"context"
"database/sql"
sqldriver "database/sql/driver"
"fmt"
"net/url"
"testing"
)
import (
"github.com/dhui/dktest"
"github.com/go-sql-driver/mysql"
)
import (
dt "github.com/golang-migrate/migrate/v4/database/testing"
"github.com/golang-migrate/migrate/v4/dktesting"
)
const defaultPort = 3306
var (
opts = dktest.Options{
Env: map[string]string{"MYSQL_ROOT_PASSWORD": "root", "MYSQL_DATABASE": "public"},
PortRequired: true, ReadyFunc: isReady,
}
// Supported versions: https://www.mysql.com/support/supportedplatforms/database.html
specs = []dktesting.ContainerSpec{
{ImageName: "mysql:5.5", Options: opts},
{ImageName: "mysql:5.6", Options: opts},
{ImageName: "mysql:5.7", Options: opts},
{ImageName: "mysql:8", Options: opts},
}
)
func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
ip, port, err := c.Port(defaultPort)
if err != nil {
return false
}
db, err := sql.Open("mysql", fmt.Sprintf("root:root@tcp(%v:%v)/public", ip, port))
if err != nil {
return false
}
defer db.Close()
if err = db.PingContext(ctx); err != nil {
switch err {
case sqldriver.ErrBadConn, mysql.ErrInvalidConn:
return false
default:
fmt.Println(err)
}
return false
}
return true
}
func Test(t *testing.T) {
// mysql.SetLogger(mysql.Logger(log.New(ioutil.Discard, "", log.Ltime)))
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.Port(defaultPort)
if err != nil {
t.Fatal(err)
}
addr := fmt.Sprintf("mysql://root:root@tcp(%v:%v)/public", ip, port)
p := &Mysql{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
dt.Test(t, d, []byte("SELECT 1"))
// check ensureVersionTable
if err := d.(*Mysql).ensureVersionTable(); err != nil {
t.Fatal(err)
}
// check again
if err := d.(*Mysql).ensureVersionTable(); err != nil {
t.Fatal(err)
}
})
}
func TestLockWorks(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.Port(defaultPort)
if err != nil {
t.Fatal(err)
}
addr := fmt.Sprintf("mysql://root:root@tcp(%v:%v)/public", ip, port)
p := &Mysql{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
dt.Test(t, d, []byte("SELECT 1"))
ms := d.(*Mysql)
err = ms.Lock()
if err != nil {
t.Fatal(err)
}
err = ms.Unlock()
if err != nil {
t.Fatal(err)
}
// make sure the 2nd lock works (RELEASE_LOCK is very finicky)
err = ms.Lock()
if err != nil {
t.Fatal(err)
}
err = ms.Unlock()
if err != nil {
t.Fatal(err)
}
})
}
func TestURLToMySQLConfig(t *testing.T) {
testcases := []struct {
name string
urlStr string
expectedDSN string // empty string signifies that an error is expected
}{
{name: "no user/password", urlStr: "mysql://tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "only user", urlStr: "mysql://username@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "only user - with encoded :",
urlStr: "mysql://username%3A@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username:@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "only user - with encoded @",
urlStr: "mysql://username%40@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username@@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "user/password", urlStr: "mysql://username:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
// Not supported yet: https://github.com/go-sql-driver/mysql/issues/591
// {name: "user/password - user with encoded :",
// urlStr: "mysql://username%3A:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
// expectedDSN: "username::pasword@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "user/password - user with encoded @",
urlStr: "mysql://username%40:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username@:password@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "user/password - password with encoded :",
urlStr: "mysql://username:password%3A@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username:password:@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
{name: "user/password - password with encoded @",
urlStr: "mysql://username:password%40@tcp(127.0.0.1:3306)/myDB?multiStatements=true",
expectedDSN: "username:password@@tcp(127.0.0.1:3306)/myDB?multiStatements=true"},
}
for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
u, err := url.Parse(tc.urlStr)
if err != nil {
t.Fatal("Failed to parse url string:", tc.urlStr, "error:", err)
}
if config, err := urlToMySQLConfig(*u); err == nil {
dsn := config.FormatDSN()
if dsn != tc.expectedDSN {
t.Error("Got unexpected DSN:", dsn, "!=", tc.expectedDSN)
}
} else {
if tc.expectedDSN != "" {
t.Error("Got unexpected error:", err, "urlStr:", tc.urlStr)
}
}
})
}
}
View File
-244
View File
@@ -1,244 +0,0 @@
package database_test
import (
"encoding/hex"
"net/url"
"strings"
"testing"
)
const reservedChars = "!#$%&'()*+,/:;=?@[]"
// TestUserUnencodedReservedURLChars documents the behavior of using unencoded reserved characters in usernames with
// net/url Parse()
func TestUserUnencodedReservedURLChars(t *testing.T) {
scheme := "database://"
baseUsername := "username"
urlSuffix := "password@localhost:12345/myDB?someParam=true"
urlSuffixAndSep := ":" + urlSuffix
testcases := []struct {
char string
parses bool
expectedUsername string // empty string means that the username failed to parse
encodedURL string
}{
{char: "!", parses: true, expectedUsername: baseUsername + "!",
encodedURL: scheme + baseUsername + "%21" + urlSuffixAndSep},
{char: "#", parses: true, expectedUsername: "",
encodedURL: scheme + baseUsername + "#" + urlSuffixAndSep},
{char: "$", parses: true, expectedUsername: baseUsername + "$",
encodedURL: scheme + baseUsername + "$" + urlSuffixAndSep},
{char: "%", parses: false},
{char: "&", parses: true, expectedUsername: baseUsername + "&",
encodedURL: scheme + baseUsername + "&" + urlSuffixAndSep},
{char: "'", parses: true, expectedUsername: "username'",
encodedURL: scheme + baseUsername + "%27" + urlSuffixAndSep},
{char: "(", parses: true, expectedUsername: "username(",
encodedURL: scheme + baseUsername + "%28" + urlSuffixAndSep},
{char: ")", parses: true, expectedUsername: "username)",
encodedURL: scheme + baseUsername + "%29" + urlSuffixAndSep},
{char: "*", parses: true, expectedUsername: "username*",
encodedURL: scheme + baseUsername + "%2A" + urlSuffixAndSep},
{char: "+", parses: true, expectedUsername: "username+",
encodedURL: scheme + baseUsername + "+" + urlSuffixAndSep},
{char: ",", parses: true, expectedUsername: "username,",
encodedURL: scheme + baseUsername + "," + urlSuffixAndSep},
{char: "/", parses: true, expectedUsername: "",
encodedURL: scheme + baseUsername + "/" + urlSuffixAndSep},
{char: ":", parses: true, expectedUsername: "username",
encodedURL: scheme + baseUsername + ":%3A" + urlSuffix},
{char: ";", parses: true, expectedUsername: "username;",
encodedURL: scheme + baseUsername + ";" + urlSuffixAndSep},
{char: "=", parses: true, expectedUsername: "username=",
encodedURL: scheme + baseUsername + "=" + urlSuffixAndSep},
{char: "?", parses: true, expectedUsername: "",
encodedURL: scheme + baseUsername + "?" + urlSuffixAndSep},
{char: "@", parses: true, expectedUsername: "username@",
encodedURL: scheme + baseUsername + "%40" + urlSuffixAndSep},
{char: "[", parses: false},
{char: "]", parses: false},
}
testedChars := make([]string, 0, len(reservedChars))
for _, tc := range testcases {
testedChars = append(testedChars, tc.char)
t.Run("reserved char "+tc.char, func(t *testing.T) {
s := scheme + baseUsername + tc.char + urlSuffixAndSep
u, err := url.Parse(s)
if err == nil {
if !tc.parses {
t.Error("Unexpectedly parsed reserved character. url:", s)
return
}
var username string
if u.User != nil {
username = u.User.Username()
}
if username != tc.expectedUsername {
t.Error("Got unexpected username:", username, "!=", tc.expectedUsername)
}
if s := u.String(); s != tc.encodedURL {
t.Error("Got unexpected encoded URL:", s, "!=", tc.encodedURL)
}
} else {
if tc.parses {
t.Error("Failed to parse reserved character. url:", s)
}
}
})
}
t.Run("All reserved chars tested", func(t *testing.T) {
if s := strings.Join(testedChars, ""); s != reservedChars {
t.Error("Not all reserved URL characters were tested:", s, "!=", reservedChars)
}
})
}
func TestUserEncodedReservedURLChars(t *testing.T) {
scheme := "database://"
baseUsername := "username"
urlSuffix := "password@localhost:12345/myDB?someParam=true"
urlSuffixAndSep := ":" + urlSuffix
for _, c := range reservedChars {
c := string(c)
t.Run("reserved char "+c, func(t *testing.T) {
encodedChar := "%" + hex.EncodeToString([]byte(c))
s := scheme + baseUsername + encodedChar + urlSuffixAndSep
expectedUsername := baseUsername + c
u, err := url.Parse(s)
if err != nil {
t.Fatal("Failed to parse url with encoded reserved character. url:", s)
}
if u.User == nil {
t.Fatal("Failed to parse userinfo with encoded reserve character. url:", s)
}
if username := u.User.Username(); username != expectedUsername {
t.Fatal("Got unexpected username:", username, "!=", expectedUsername)
}
})
}
}
// TestPasswordUnencodedReservedURLChars documents the behavior of using unencoded reserved characters in passwords
// with net/url Parse()
func TestPasswordUnencodedReservedURLChars(t *testing.T) {
username := "username"
schemeAndUsernameAndSep := "database://" + username + ":"
basePassword := "password"
urlSuffixAndSep := "@localhost:12345/myDB?someParam=true"
testcases := []struct {
char string
parses bool
expectedUsername string // empty string means that the username failed to parse
expectedPassword string // empty string means that the password failed to parse
encodedURL string
}{
{char: "!", parses: true, expectedUsername: username, expectedPassword: basePassword + "!",
encodedURL: schemeAndUsernameAndSep + basePassword + "%21" + urlSuffixAndSep},
{char: "#", parses: true, expectedUsername: "", expectedPassword: "",
encodedURL: schemeAndUsernameAndSep + basePassword + "#" + urlSuffixAndSep},
{char: "$", parses: true, expectedUsername: username, expectedPassword: basePassword + "$",
encodedURL: schemeAndUsernameAndSep + basePassword + "$" + urlSuffixAndSep},
{char: "%", parses: false},
{char: "&", parses: true, expectedUsername: username, expectedPassword: basePassword + "&",
encodedURL: schemeAndUsernameAndSep + basePassword + "&" + urlSuffixAndSep},
{char: "'", parses: true, expectedUsername: username, expectedPassword: "password'",
encodedURL: schemeAndUsernameAndSep + basePassword + "%27" + urlSuffixAndSep},
{char: "(", parses: true, expectedUsername: username, expectedPassword: "password(",
encodedURL: schemeAndUsernameAndSep + basePassword + "%28" + urlSuffixAndSep},
{char: ")", parses: true, expectedUsername: username, expectedPassword: "password)",
encodedURL: schemeAndUsernameAndSep + basePassword + "%29" + urlSuffixAndSep},
{char: "*", parses: true, expectedUsername: username, expectedPassword: "password*",
encodedURL: schemeAndUsernameAndSep + basePassword + "%2A" + urlSuffixAndSep},
{char: "+", parses: true, expectedUsername: username, expectedPassword: "password+",
encodedURL: schemeAndUsernameAndSep + basePassword + "+" + urlSuffixAndSep},
{char: ",", parses: true, expectedUsername: username, expectedPassword: "password,",
encodedURL: schemeAndUsernameAndSep + basePassword + "," + urlSuffixAndSep},
{char: "/", parses: true, expectedUsername: "", expectedPassword: "",
encodedURL: schemeAndUsernameAndSep + basePassword + "/" + urlSuffixAndSep},
{char: ":", parses: true, expectedUsername: username, expectedPassword: "password:",
encodedURL: schemeAndUsernameAndSep + basePassword + "%3A" + urlSuffixAndSep},
{char: ";", parses: true, expectedUsername: username, expectedPassword: "password;",
encodedURL: schemeAndUsernameAndSep + basePassword + ";" + urlSuffixAndSep},
{char: "=", parses: true, expectedUsername: username, expectedPassword: "password=",
encodedURL: schemeAndUsernameAndSep + basePassword + "=" + urlSuffixAndSep},
{char: "?", parses: true, expectedUsername: "", expectedPassword: "",
encodedURL: schemeAndUsernameAndSep + basePassword + "?" + urlSuffixAndSep},
{char: "@", parses: true, expectedUsername: username, expectedPassword: "password@",
encodedURL: schemeAndUsernameAndSep + basePassword + "%40" + urlSuffixAndSep},
{char: "[", parses: false},
{char: "]", parses: false},
}
testedChars := make([]string, 0, len(reservedChars))
for _, tc := range testcases {
testedChars = append(testedChars, tc.char)
t.Run("reserved char "+tc.char, func(t *testing.T) {
s := schemeAndUsernameAndSep + basePassword + tc.char + urlSuffixAndSep
u, err := url.Parse(s)
if err == nil {
if !tc.parses {
t.Error("Unexpectedly parsed reserved character. url:", s)
return
}
var username, password string
if u.User != nil {
username = u.User.Username()
password, _ = u.User.Password()
}
if username != tc.expectedUsername {
t.Error("Got unexpected username:", username, "!=", tc.expectedUsername)
}
if password != tc.expectedPassword {
t.Error("Got unexpected password:", password, "!=", tc.expectedPassword)
}
if s := u.String(); s != tc.encodedURL {
t.Error("Got unexpected encoded URL:", s, "!=", tc.encodedURL)
}
} else {
if tc.parses {
t.Error("Failed to parse reserved character. url:", s)
}
}
})
}
t.Run("All reserved chars tested", func(t *testing.T) {
if s := strings.Join(testedChars, ""); s != reservedChars {
t.Error("Not all reserved URL characters were tested:", s, "!=", reservedChars)
}
})
}
func TestPasswordEncodedReservedURLChars(t *testing.T) {
username := "username"
schemeAndUsernameAndSep := "database://" + username + ":"
basePassword := "password"
urlSuffixAndSep := "@localhost:12345/myDB?someParam=true"
for _, c := range reservedChars {
c := string(c)
t.Run("reserved char "+c, func(t *testing.T) {
encodedChar := "%" + hex.EncodeToString([]byte(c))
s := schemeAndUsernameAndSep + basePassword + encodedChar + urlSuffixAndSep
expectedPassword := basePassword + c
u, err := url.Parse(s)
if err != nil {
t.Fatal("Failed to parse url with encoded reserved character. url:", s)
}
if u.User == nil {
t.Fatal("Failed to parse userinfo with encoded reserve character. url:", s)
}
if n := u.User.Username(); n != username {
t.Fatal("Got unexpected username:", n, "!=", username)
}
if p, _ := u.User.Password(); p != expectedPassword {
t.Fatal("Got unexpected password:", p, "!=", expectedPassword)
}
})
}
}
-28
View File
@@ -1,28 +0,0 @@
# postgres
`postgres://user:password@host:port/dbname?query` (`postgresql://` works, too)
| URL Query | WithInstance Config | Description |
|------------|---------------------|-------------|
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
| `dbname` | `DatabaseName` | The name of the database to connect to |
| `search_path` | | This variable specifies the order in which schemas are searched when an object is referenced by a simple name with no schema specified. |
| `user` | | The user to sign in as |
| `password` | | The user's password |
| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
| `port` | | The port to bind to. (default is 5432) |
| `fallback_application_name` | | An application_name to fall back to if one isn't provided. |
| `connect_timeout` | | Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. |
| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
| `sslkey` | | Key file location. The file must contain PEM encoded data. |
| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
## Upgrading from v1
1. Write down the current migration version from schema_migrations
1. `DROP TABLE schema_migrations`
2. Wrap your existing migrations in transactions ([BEGIN/COMMIT](https://www.postgresql.org/docs/current/static/transaction-iso.html)) if you use multiple statements within one migration.
3. Download and install the latest migrate version.
4. Force the current migration version with `migrate force <current_version>`.
@@ -1 +0,0 @@
DROP TABLE IF EXISTS users;
@@ -1,5 +0,0 @@
CREATE TABLE users (
user_id integer unique,
name varchar(40),
email varchar(40)
);
@@ -1 +0,0 @@
ALTER TABLE users DROP COLUMN IF EXISTS city;
@@ -1,3 +0,0 @@
ALTER TABLE users ADD COLUMN city varchar(100);
@@ -1 +0,0 @@
DROP INDEX IF EXISTS users_email_index;
@@ -1,3 +0,0 @@
CREATE UNIQUE INDEX CONCURRENTLY users_email_index ON users (email);
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
DROP TABLE IF EXISTS books;
@@ -1,5 +0,0 @@
CREATE TABLE books (
user_id integer,
name varchar(40),
author varchar(40)
);
@@ -1 +0,0 @@
DROP TABLE IF EXISTS movies;
@@ -1,5 +0,0 @@
CREATE TABLE movies (
user_id integer,
name varchar(40),
director varchar(40)
);
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
-357
View File
@@ -1,357 +0,0 @@
// +build go1.9
package postgres
import (
"context"
"database/sql"
"fmt"
"io"
"io/ioutil"
nurl "net/url"
"strconv"
"strings"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
multierror "github.com/hashicorp/go-multierror"
"github.com/lib/pq"
)
func init() {
db := Postgres{}
database.Register("postgres", &db)
database.Register("postgresql", &db)
}
var DefaultMigrationsTable = "schema_migrations"
var (
ErrNilConfig = fmt.Errorf("no config")
ErrNoDatabaseName = fmt.Errorf("no database name")
ErrNoSchema = fmt.Errorf("no schema")
ErrDatabaseDirty = fmt.Errorf("database is dirty")
)
type Config struct {
MigrationsTable string
DatabaseName string
SchemaName string
}
type Postgres struct {
// Locking and unlocking need to use the same connection
conn *sql.Conn
db *sql.DB
isLocked bool
// Open and WithInstance need to garantuee that config is never nil
config *Config
}
func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
}
if err := instance.Ping(); err != nil {
return nil, err
}
query := `SELECT CURRENT_DATABASE()`
var databaseName string
if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
}
if len(databaseName) == 0 {
return nil, ErrNoDatabaseName
}
config.DatabaseName = databaseName
query = `SELECT CURRENT_SCHEMA()`
var schemaName string
if err := instance.QueryRow(query).Scan(&schemaName); err != nil {
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
}
if len(schemaName) == 0 {
return nil, ErrNoSchema
}
config.SchemaName = schemaName
if len(config.MigrationsTable) == 0 {
config.MigrationsTable = DefaultMigrationsTable
}
conn, err := instance.Conn(context.Background())
if err != nil {
return nil, err
}
px := &Postgres{
conn: conn,
db: instance,
config: config,
}
if err := px.ensureVersionTable(); err != nil {
return nil, err
}
return px, nil
}
func (p *Postgres) Open(url string) (database.Driver, error) {
purl, err := nurl.Parse(url)
if err != nil {
return nil, err
}
db, err := sql.Open("postgres", migrate.FilterCustomQuery(purl).String())
if err != nil {
return nil, err
}
migrationsTable := purl.Query().Get("x-migrations-table")
if len(migrationsTable) == 0 {
migrationsTable = DefaultMigrationsTable
}
px, err := WithInstance(db, &Config{
DatabaseName: purl.Path,
MigrationsTable: migrationsTable,
})
if err != nil {
return nil, err
}
return px, nil
}
func (p *Postgres) Close() error {
connErr := p.conn.Close()
dbErr := p.db.Close()
if connErr != nil || dbErr != nil {
return fmt.Errorf("conn: %v, db: %v", connErr, dbErr)
}
return nil
}
// https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
func (p *Postgres) Lock() error {
if p.isLocked {
return database.ErrLocked
}
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.SchemaName)
if err != nil {
return err
}
// This will either obtain the lock immediately and return true,
// or return false if the lock cannot be acquired immediately.
query := `SELECT pg_advisory_lock($1)`
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
}
p.isLocked = true
return nil
}
func (p *Postgres) Unlock() error {
if !p.isLocked {
return nil
}
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.SchemaName)
if err != nil {
return err
}
query := `SELECT pg_advisory_unlock($1)`
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
p.isLocked = false
return nil
}
func (p *Postgres) Run(migration io.Reader) error {
migr, err := ioutil.ReadAll(migration)
if err != nil {
return err
}
// run migration
query := string(migr[:])
if _, err := p.conn.ExecContext(context.Background(), query); err != nil {
if pgErr, ok := err.(*pq.Error); ok {
var line uint
var col uint
var lineColOK bool
if pgErr.Position != "" {
if pos, err := strconv.ParseUint(pgErr.Position, 10, 64); err == nil {
line, col, lineColOK = computeLineFromPos(query, int(pos))
}
}
message := fmt.Sprintf("migration failed: %s", pgErr.Message)
if lineColOK {
message = fmt.Sprintf("%s (column %d)", message, col)
}
if pgErr.Detail != "" {
message = fmt.Sprintf("%s, %s", message, pgErr.Detail)
}
return database.Error{OrigErr: err, Err: message, Query: migr, Line: line}
}
return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
}
return nil
}
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
// replace crlf with lf
s = strings.Replace(s, "\r\n", "\n", -1)
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
runes := []rune(s)
if pos > len(runes) {
return 0, 0, false
}
sel := runes[:pos]
line = uint(runesCount(sel, newLine) + 1)
col = uint(pos - 1 - runesLastIndex(sel, newLine))
return line, col, true
}
const newLine = '\n'
func runesCount(input []rune, target rune) int {
var count int
for _, r := range input {
if r == target {
count++
}
}
return count
}
func runesLastIndex(input []rune, target rune) int {
for i := len(input) - 1; i >= 0; i-- {
if input[i] == target {
return i
}
}
return -1
}
func (p *Postgres) SetVersion(version int, dirty bool) error {
tx, err := p.conn.BeginTx(context.Background(), &sql.TxOptions{})
if err != nil {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
query := `TRUNCATE "` + p.config.MigrationsTable + `"`
if _, err := tx.Exec(query); err != nil {
tx.Rollback()
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if version >= 0 {
query = `INSERT INTO "` + p.config.MigrationsTable + `" (version, dirty) VALUES ($1, $2)`
if _, err := tx.Exec(query, version, dirty); err != nil {
tx.Rollback()
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := tx.Commit(); err != nil {
return &database.Error{OrigErr: err, Err: "transaction commit failed"}
}
return nil
}
func (p *Postgres) Version() (version int, dirty bool, err error) {
query := `SELECT version, dirty FROM "` + p.config.MigrationsTable + `" LIMIT 1`
err = p.conn.QueryRowContext(context.Background(), query).Scan(&version, &dirty)
switch {
case err == sql.ErrNoRows:
return database.NilVersion, false, nil
case err != nil:
if e, ok := err.(*pq.Error); ok {
if e.Code.Name() == "undefined_table" {
return database.NilVersion, false, nil
}
}
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
default:
return version, dirty, nil
}
}
func (p *Postgres) Drop() error {
// select all tables in current schema
query := `SELECT table_name FROM information_schema.tables WHERE table_schema=(SELECT current_schema()) AND table_type='BASE TABLE'`
tables, err := p.conn.QueryContext(context.Background(), query)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
defer tables.Close()
// delete one table after another
tableNames := make([]string, 0)
for tables.Next() {
var tableName string
if err := tables.Scan(&tableName); err != nil {
return err
}
if len(tableName) > 0 {
tableNames = append(tableNames, tableName)
}
}
if len(tableNames) > 0 {
// delete one by one ...
for _, t := range tableNames {
query = `DROP TABLE IF EXISTS ` + t + ` CASCADE`
if _, err := p.conn.ExecContext(context.Background(), query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := p.ensureVersionTable(); err != nil {
return err
}
}
return nil
}
func (p *Postgres) ensureVersionTable() (err error) {
if err = p.Lock(); err != nil {
return err
}
defer func() {
if e := p.Unlock(); e != nil {
if err == nil {
err = e
} else {
err = multierror.Append(err, e)
}
}
}()
query := `CREATE TABLE IF NOT EXISTS "` + p.config.MigrationsTable + `" (version bigint not null primary key, dirty boolean not null)`
if _, err = p.conn.ExecContext(context.Background(), query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
return nil
}
-427
View File
@@ -1,427 +0,0 @@
package postgres
// error codes https://github.com/lib/pq/blob/master/error.go
import (
"context"
"database/sql"
sqldriver "database/sql/driver"
"fmt"
"io"
"strconv"
"strings"
"sync"
"testing"
"github.com/dhui/dktest"
dt "github.com/golang-migrate/migrate/v4/database/testing"
"github.com/golang-migrate/migrate/v4/dktesting"
)
var (
opts = dktest.Options{PortRequired: true, ReadyFunc: isReady}
// Supported versions: https://www.postgresql.org/support/versioning/
specs = []dktesting.ContainerSpec{
{ImageName: "postgres:9.4", Options: opts},
{ImageName: "postgres:9.5", Options: opts},
{ImageName: "postgres:9.6", Options: opts},
{ImageName: "postgres:10", Options: opts},
{ImageName: "postgres:11", Options: opts},
}
)
func pgConnectionString(host, port string) string {
return fmt.Sprintf("postgres://postgres@%s:%s/postgres?sslmode=disable", host, port)
}
func isReady(ctx context.Context, c dktest.ContainerInfo) bool {
ip, port, err := c.FirstPort()
if err != nil {
return false
}
db, err := sql.Open("postgres", pgConnectionString(ip, port))
if err != nil {
return false
}
defer db.Close()
if err = db.PingContext(ctx); err != nil {
switch err {
case sqldriver.ErrBadConn, io.EOF:
return false
default:
fmt.Println(err)
}
return false
}
return true
}
func Test(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
dt.Test(t, d, []byte("SELECT 1"))
})
}
func TestMultiStatement(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
if err := d.Run(strings.NewReader("CREATE TABLE foo (foo text); CREATE TABLE bar (bar text);")); err != nil {
t.Fatalf("expected err to be nil, got %v", err)
}
// make sure second table exists
var exists bool
if err := d.(*Postgres).conn.QueryRowContext(context.Background(), "SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'bar' AND table_schema = (SELECT current_schema()))").Scan(&exists); err != nil {
t.Fatal(err)
}
if !exists {
t.Fatalf("expected table bar to exist")
}
})
}
func TestErrorParsing(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
wantErr := `migration failed: syntax error at or near "TABLEE" (column 37) in line 1: CREATE TABLE foo ` +
`(foo text); CREATE TABLEE bar (bar text); (details: pq: syntax error at or near "TABLEE")`
if err := d.Run(strings.NewReader("CREATE TABLE foo (foo text); CREATE TABLEE bar (bar text);")); err == nil {
t.Fatal("expected err but got nil")
} else if err.Error() != wantErr {
t.Fatalf("expected '%s' but got '%s'", wantErr, err.Error())
}
})
}
func TestFilterCustomQuery(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := fmt.Sprintf("postgres://postgres@%v:%v/postgres?sslmode=disable&x-custom=foobar", ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
})
}
func TestWithSchema(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
// create foobar schema
if err := d.Run(strings.NewReader("CREATE SCHEMA foobar AUTHORIZATION postgres")); err != nil {
t.Fatal(err)
}
if err := d.SetVersion(1, false); err != nil {
t.Fatal(err)
}
// re-connect using that schema
d2, err := p.Open(fmt.Sprintf("postgres://postgres@%v:%v/postgres?sslmode=disable&search_path=foobar", ip, port))
if err != nil {
t.Fatalf("%v", err)
}
defer d2.Close()
version, _, err := d2.Version()
if err != nil {
t.Fatal(err)
}
if version != -1 {
t.Fatal("expected NilVersion")
}
// now update version and compare
if err := d2.SetVersion(2, false); err != nil {
t.Fatal(err)
}
version, _, err = d2.Version()
if err != nil {
t.Fatal(err)
}
if version != 2 {
t.Fatal("expected version 2")
}
// meanwhile, the public schema still has the other version
version, _, err = d.Version()
if err != nil {
t.Fatal(err)
}
if version != 1 {
t.Fatal("expected version 2")
}
})
}
func TestParallelSchema(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
defer d.Close()
// create foo and bar schemas
if err := d.Run(strings.NewReader("CREATE SCHEMA foo AUTHORIZATION postgres")); err != nil {
t.Fatal(err)
}
if err := d.Run(strings.NewReader("CREATE SCHEMA bar AUTHORIZATION postgres")); err != nil {
t.Fatal(err)
}
// re-connect using that schemas
dfoo, err := p.Open(fmt.Sprintf("postgres://postgres@%v:%v/postgres?sslmode=disable&search_path=foo", ip, port))
if err != nil {
t.Fatalf("%v", err)
}
defer dfoo.Close()
dbar, err := p.Open(fmt.Sprintf("postgres://postgres@%v:%v/postgres?sslmode=disable&search_path=bar", ip, port))
if err != nil {
t.Fatalf("%v", err)
}
defer dbar.Close()
if err := dfoo.Lock(); err != nil {
t.Fatal(err)
}
if err := dbar.Lock(); err != nil {
t.Fatal(err)
}
if err := dbar.Unlock(); err != nil {
t.Fatal(err)
}
if err := dfoo.Unlock(); err != nil {
t.Fatal(err)
}
})
}
func TestWithInstance(t *testing.T) {
}
func TestPostgres_Lock(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
addr := pgConnectionString(ip, port)
p := &Postgres{}
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
dt.Test(t, d, []byte("SELECT 1"))
ps := d.(*Postgres)
err = ps.Lock()
if err != nil {
t.Fatal(err)
}
err = ps.Unlock()
if err != nil {
t.Fatal(err)
}
err = ps.Lock()
if err != nil {
t.Fatal(err)
}
err = ps.Unlock()
if err != nil {
t.Fatal(err)
}
})
}
func TestWithInstance_Concurrent(t *testing.T) {
dktesting.ParallelTest(t, specs, func(t *testing.T, c dktest.ContainerInfo) {
ip, port, err := c.FirstPort()
if err != nil {
t.Fatal(err)
}
// The number of concurrent processes running WithInstance
const concurrency = 30
// We can instantiate a single database handle because it is
// actually a connection pool, and so, each of the below go
// routines will have a high probability of using a separate
// connection, which is something we want to exercise.
db, err := sql.Open("postgres", pgConnectionString(ip, port))
if err != nil {
t.Fatal(err)
}
defer db.Close()
db.SetMaxIdleConns(concurrency)
db.SetMaxOpenConns(concurrency)
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(concurrency)
for i := 0; i < concurrency; i++ {
go func(i int) {
defer wg.Done()
_, err := WithInstance(db, &Config{})
if err != nil {
t.Errorf("process %d error: %s", i, err)
}
}(i)
}
})
}
func Test_computeLineFromPos(t *testing.T) {
testcases := []struct {
pos int
wantLine uint
wantCol uint
input string
wantOk bool
}{
{
15, 2, 6, "SELECT *\nFROM foo", true, // foo table does not exists
},
{
16, 3, 6, "SELECT *\n\nFROM foo", true, // foo table does not exists, empty line
},
{
25, 3, 7, "SELECT *\nFROM foo\nWHERE x", true, // x column error
},
{
27, 5, 7, "SELECT *\n\nFROM foo\n\nWHERE x", true, // x column error, empty lines
},
{
10, 2, 1, "SELECT *\nFROMM foo", true, // FROMM typo
},
{
11, 3, 1, "SELECT *\n\nFROMM foo", true, // FROMM typo, empty line
},
{
17, 2, 8, "SELECT *\nFROM foo", true, // last character
},
{
18, 0, 0, "SELECT *\nFROM foo", false, // invalid position
},
}
for i, tc := range testcases {
t.Run("tc"+strconv.Itoa(i), func(t *testing.T) {
run := func(crlf bool, nonASCII bool) {
var name string
if crlf {
name = "crlf"
} else {
name = "lf"
}
if nonASCII {
name += "-nonascii"
} else {
name += "-ascii"
}
t.Run(name, func(t *testing.T) {
input := tc.input
if crlf {
input = strings.Replace(input, "\n", "\r\n", -1)
}
if nonASCII {
input = strings.Replace(input, "FROM", "FRÖM", -1)
}
gotLine, gotCol, gotOK := computeLineFromPos(input, tc.pos)
if tc.wantOk {
t.Logf("pos %d, want %d:%d, %#v", tc.pos, tc.wantLine, tc.wantCol, input)
}
if gotOK != tc.wantOk {
t.Fatalf("expected ok %v but got %v", tc.wantOk, gotOK)
}
if gotLine != tc.wantLine {
t.Fatalf("expected line %d but got %d", tc.wantLine, gotLine)
}
if gotCol != tc.wantCol {
t.Fatalf("expected col %d but got %d", tc.wantCol, gotCol)
}
})
}
run(false, false)
run(true, false)
run(false, true)
run(true, true)
})
}
}
View File
@@ -1 +0,0 @@
DROP TABLE IF EXISTS pets;
@@ -1,3 +0,0 @@
CREATE TABLE pets (
name string
);
@@ -1 +0,0 @@
DROP TABLE IF EXISTS pets;
@@ -1 +0,0 @@
ALTER TABLE pets ADD predator bool;;
-212
View File
@@ -1,212 +0,0 @@
package ql
import (
"database/sql"
"fmt"
"io"
"io/ioutil"
"strings"
nurl "net/url"
_ "github.com/cznic/ql/driver"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database"
)
func init() {
database.Register("ql", &Ql{})
}
var DefaultMigrationsTable = "schema_migrations"
var (
ErrDatabaseDirty = fmt.Errorf("database is dirty")
ErrNilConfig = fmt.Errorf("no config")
ErrNoDatabaseName = fmt.Errorf("no database name")
ErrAppendPEM = fmt.Errorf("failed to append PEM")
)
type Config struct {
MigrationsTable string
DatabaseName string
}
type Ql struct {
db *sql.DB
isLocked bool
config *Config
}
func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
if config == nil {
return nil, ErrNilConfig
}
if err := instance.Ping(); err != nil {
return nil, err
}
if len(config.MigrationsTable) == 0 {
config.MigrationsTable = DefaultMigrationsTable
}
mx := &Ql{
db: instance,
config: config,
}
if err := mx.ensureVersionTable(); err != nil {
return nil, err
}
return mx, nil
}
func (m *Ql) ensureVersionTable() error {
tx, err := m.db.Begin()
if err != nil {
return err
}
if _, err := tx.Exec(fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS %s (version uint64,dirty bool);
CREATE UNIQUE INDEX IF NOT EXISTS version_unique ON %s (version);
`, m.config.MigrationsTable, m.config.MigrationsTable)); err != nil {
if err := tx.Rollback(); err != nil {
return err
}
return err
}
if err := tx.Commit(); err != nil {
return err
}
return nil
}
func (m *Ql) Open(url string) (database.Driver, error) {
purl, err := nurl.Parse(url)
if err != nil {
return nil, err
}
dbfile := strings.Replace(migrate.FilterCustomQuery(purl).String(), "ql://", "", 1)
db, err := sql.Open("ql", dbfile)
if err != nil {
return nil, err
}
migrationsTable := purl.Query().Get("x-migrations-table")
if len(migrationsTable) == 0 {
migrationsTable = DefaultMigrationsTable
}
mx, err := WithInstance(db, &Config{
DatabaseName: purl.Path,
MigrationsTable: migrationsTable,
})
if err != nil {
return nil, err
}
return mx, nil
}
func (m *Ql) Close() error {
return m.db.Close()
}
func (m *Ql) Drop() error {
query := `SELECT Name FROM __Table`
tables, err := m.db.Query(query)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
defer tables.Close()
tableNames := make([]string, 0)
for tables.Next() {
var tableName string
if err := tables.Scan(&tableName); err != nil {
return err
}
if len(tableName) > 0 {
if strings.HasPrefix(tableName, "__") == false {
tableNames = append(tableNames, tableName)
}
}
}
if len(tableNames) > 0 {
for _, t := range tableNames {
query := "DROP TABLE " + t
err = m.executeQuery(query)
if err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := m.ensureVersionTable(); err != nil {
return err
}
}
return nil
}
func (m *Ql) Lock() error {
if m.isLocked {
return database.ErrLocked
}
m.isLocked = true
return nil
}
func (m *Ql) Unlock() error {
if !m.isLocked {
return nil
}
m.isLocked = false
return nil
}
func (m *Ql) Run(migration io.Reader) error {
migr, err := ioutil.ReadAll(migration)
if err != nil {
return err
}
query := string(migr[:])
return m.executeQuery(query)
}
func (m *Ql) executeQuery(query string) error {
tx, err := m.db.Begin()
if err != nil {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
if _, err := tx.Exec(query); err != nil {
tx.Rollback()
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if err := tx.Commit(); err != nil {
return &database.Error{OrigErr: err, Err: "transaction commit failed"}
}
return nil
}
func (m *Ql) SetVersion(version int, dirty bool) error {
tx, err := m.db.Begin()
if err != nil {
return &database.Error{OrigErr: err, Err: "transaction start failed"}
}
query := "TRUNCATE TABLE " + m.config.MigrationsTable
if _, err := tx.Exec(query); err != nil {
return &database.Error{OrigErr: err, Query: []byte(query)}
}
if version >= 0 {
query := fmt.Sprintf(`INSERT INTO %s (version, dirty) VALUES (%d, %t)`, m.config.MigrationsTable, version, dirty)
if _, err := tx.Exec(query); err != nil {
tx.Rollback()
return &database.Error{OrigErr: err, Query: []byte(query)}
}
}
if err := tx.Commit(); err != nil {
return &database.Error{OrigErr: err, Err: "transaction commit failed"}
}
return nil
}
func (m *Ql) Version() (version int, dirty bool, err error) {
query := "SELECT version, dirty FROM " + m.config.MigrationsTable + " LIMIT 1"
err = m.db.QueryRow(query).Scan(&version, &dirty)
if err != nil {
return database.NilVersion, false, nil
}
return version, dirty, nil
}
-62
View File
@@ -1,62 +0,0 @@
package ql
import (
"database/sql"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"testing"
_ "github.com/cznic/ql/driver"
"github.com/golang-migrate/migrate/v4"
dt "github.com/golang-migrate/migrate/v4/database/testing"
_ "github.com/golang-migrate/migrate/v4/source/file"
)
func Test(t *testing.T) {
dir, err := ioutil.TempDir("", "ql-driver-test")
if err != nil {
return
}
defer func() {
os.RemoveAll(dir)
}()
fmt.Printf("DB path : %s\n", filepath.Join(dir, "ql.db"))
p := &Ql{}
addr := fmt.Sprintf("ql://%s", filepath.Join(dir, "ql.db"))
d, err := p.Open(addr)
if err != nil {
t.Fatalf("%v", err)
}
db, err := sql.Open("ql", filepath.Join(dir, "ql.db"))
if err != nil {
return
}
defer func() {
if err := db.Close(); err != nil {
return
}
}()
dt.Test(t, d, []byte("CREATE TABLE t (Qty int, Name string);"))
driver, err := WithInstance(db, &Config{})
if err != nil {
t.Fatalf("%v", err)
}
if err := d.Drop(); err != nil {
t.Fatal(err)
}
m, err := migrate.NewWithDatabaseInstance(
"file://./migration",
"ql", driver)
if err != nil {
t.Fatalf("%v", err)
}
fmt.Println("UP")
err = m.Up()
if err != nil {
t.Fatalf("%v", err)
}
}
-21
View File
@@ -1,21 +0,0 @@
# Redshift
`redshift://user:password@host:port/dbname?query`
| URL Query | WithInstance Config | Description |
|------------|---------------------|-------------|
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
| `dbname` | `DatabaseName` | The name of the database to connect to |
| `search_path` | | This variable specifies the order in which schemas are searched when an object is referenced by a simple name with no schema specified. |
| `user` | | The user to sign in as |
| `password` | | The user's password |
| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
| `port` | | The port to bind to. (default is 5439) |
| `fallback_application_name` | | An application_name to fall back to if one isn't provided. |
| `connect_timeout` | | Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. |
| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
| `sslkey` | | Key file location. The file must contain PEM encoded data. |
| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
Redshift is PostgreSQL compatible but has some specific features (or lack thereof) that require slightly different behavior.
@@ -1 +0,0 @@
DROP TABLE IF EXISTS users;
@@ -1,5 +0,0 @@
CREATE TABLE users (
user_id integer unique,
name varchar(40),
email varchar(40)
);
@@ -1 +0,0 @@
ALTER TABLE users DROP COLUMN IF EXISTS city;
@@ -1,3 +0,0 @@
ALTER TABLE users ADD COLUMN city varchar(100);
@@ -1 +0,0 @@
DROP INDEX IF EXISTS users_email_index;
@@ -1,3 +0,0 @@
CREATE UNIQUE INDEX CONCURRENTLY users_email_index ON users (email);
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
DROP TABLE IF EXISTS books;
@@ -1,5 +0,0 @@
CREATE TABLE books (
user_id integer,
name varchar(40),
author varchar(40)
);
@@ -1 +0,0 @@
DROP TABLE IF EXISTS movies;
@@ -1,5 +0,0 @@
CREATE TABLE movies (
user_id integer,
name varchar(40),
director varchar(40)
);
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.
@@ -1 +0,0 @@
-- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sed interdum velit, tristique iaculis justo. Pellentesque ut porttitor dolor. Donec sit amet pharetra elit. Cras vel ligula ex. Phasellus posuere.

Some files were not shown because too many files have changed in this diff Show More