Compare commits

...
278 Commits
Author SHA1 Message Date
hanzo-devandClaude Opus 4.8 dd529b4e1c ci: move to hanzoai/platform (make ci); drop GitHub Actions
Builds/runs/scans on the Hanzo PaaS (Dockerfile/Nixpacks build + scheduled
'make ci' task) instead of GHA. make ci = build + test + govulncheck + gitleaks
+ CycloneDX SBOM. Platform build env has cross-org (hanzoai+luxfi) git creds, so
private modules resolve without the GHA token workaround. Removes .github/workflows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 00:35:52 +00:00
hanzo-devandClaude Opus 4.8 5333f203e0 ci: gitleaks allowlist for upstream SQLite/crypto-VFS test fixtures
The 34 generic-api-key hits are example keys (SHA-256/MD5 of empty string) in
the adiantum/XTS VFS tests + the SQLite C amalgamation — not real secrets.
Hanzo additions (hanzovfs/, cmd/) remain scanned.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:55:47 +00:00
hanzo-devandClaude Opus 4.8 5db4440521 fix(ci): license-free gitleaks binary + graceful cross-org govulncheck
gitleaks-action@v2 requires a paid org license; use the OSS gitleaks binary
instead. govulncheck needs cross-org (luxfi) private-module access — wire
GO_MODULE_TOKEN (falls back to GITHUB_TOKEN) and continue-on-error until the
org provisions it, so secret-scan + SBOM gate while vuln-scan is best-effort.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:51:46 +00:00
ca962ab55b chore(ci): add supply-chain security workflow (SOC 2 CC7.x/CC8.x) (#1)
Adds .github/workflows/security.yml running on push and pull_request:
gitleaks secret scanning, anchore CycloneDX SBOM generation (uploaded as
a build artifact), and (for Go repos) govulncheck vulnerability scanning
with private-module resolution for github.com/hanzoai and github.com/luxfi.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 16:41:26 -07:00
hanzo-devandClaude Opus 4.8 fab434685b feat(hanzovfs): end-to-end post-quantum native SQLite VFS (durable)
Replaces the in-memory prototype with the real thing: per-database DEK seals
every 4 KiB page with ChaCha20-Poly1305, persisted to a backend dir; the DEK
itself is sealed with luxfi/age PQ-hybrid (ML-KEM-768 + X25519). The full path
tenant page → encrypted block → object is end-to-end post-quantum — recording
ciphertext yields nothing without the ML-KEM identity.

- Config{Dir, Recipients, Identities}; Register(name, cfg); GenerateIdentity().
- Test proves: DEK is age-sealed (never raw), pages carry no plaintext SQLite,
  right identity → data durable across reopen, WRONG identity → open rejected.
- Durable PQ bench (arm64): 720K insert/s, 59.6K point-read/s — 2.6x writes /
  5x reads vs the FUSE path, and post-quantum.

No FUSE, no cgo, no kludge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 20:48:33 +00:00
hanzo-devandClaude Opus 4.8 aeb79a1e8f feat: add cmd/hanzovfs-bench (runnable native-VFS benchmark)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 20:35:07 +00:00
hanzo-devandClaude Opus 4.8 764539773a feat: canonical Hanzo SQLite — rename module + native encrypted hanzovfs
Fork of ncruces/go-sqlite3 (MIT, retained). Renames module to
github.com/hanzoai/sqlite3 (wasm blob stays upstream ncruces/go-sqlite3-wasm)
and adds the hanzovfs package: a pure-Go, no-FUSE SQLite VFS that seals pages
with ChaCha20-Poly1305 (per-tenant DEK, HIP-0302) for per-tenant SQLite ⇒
hanzoai/vfs ⇒ S3.

Deprecates modernc.org/sqlite (no custom-VFS hook) and direct ncruces use.
Benchmarked 3.6x faster writes / ~92x faster point-reads than FUSE; encryption ~free.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 20:32:43 +00:00
Nuno Cruces 2900c3e2a1 CI. 2026-06-16 15:53:18 +01:00
Nuno Cruces d581398704 Issue #401. 2026-06-16 14:28:52 +01:00
Nuno Cruces f063720e7f Memory size. 2026-06-11 17:13:10 +01:00
Nuno Cruces cc974a6e22 JSON 1.27. 2026-06-11 15:37:21 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2f00389562 Bump golang.org/x/crypto from 0.52.0 to 0.53.0 (#396)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.52.0 to 0.53.0.
- [Commits](https://github.com/golang/crypto/compare/v0.52.0...v0.53.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.53.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-09 00:40:18 +01:00
Nuno Cruces 25266ab966 Pre-update hook. 2026-06-08 12:59:01 +01:00
Nuno Cruces dc662cff13 Split FTS5 and R-Tree. 2026-06-08 10:24:48 +01:00
Nuno Cruces 174151ad8f SQLite 3.53.2. 2026-06-04 16:45:52 +01:00
Nuno Cruces 312b416f2a Fix #392. 2026-06-01 10:32:37 +01:00
Nuno Cruces b7dd234638 Deps. 2026-05-28 16:33:36 +01:00
Nuno Cruces c5396f115d Reflect. 2026-05-22 11:00:03 +01:00
Nuno CrucesandGitHub 89efb65dbb Driver scanning API (#377) 2026-05-21 11:55:29 +01:00
Nuno CrucesandGitHub a2be335b4f Typo. 2026-05-20 16:33:12 +01:00
Nuno Cruces ce78ca9e71 Cleanup. 2026-05-20 14:59:47 +01:00
Sai Asish YandGitHub 79143838cd Fix #389. 2026-05-20 10:17:55 +01:00
Nuno CrucesandGitHub 3bd7753abc Update README.md 2026-05-19 18:12:14 +01:00
Nuno Cruces 57d9e999b5 CI. 2026-05-19 17:53:36 +01:00
Nuno Cruces 7c278c53d1 DSN tweaks. 2026-05-15 18:38:33 +01:00
Nuno Cruces 61d8d97579 64K pages. 2026-05-09 16:57:14 +01:00
Nuno CrucesandGitHub 892bb6a5b1 CI (#386) 2026-05-09 13:10:59 +01:00
Nuno Cruces 74832300cf Deps. 2026-05-08 16:24:28 +01:00
Nuno Cruces edf4e07234 Release. 2026-04-23 15:07:25 +01:00
Nuno Cruces f99d5fa541 Prepare release. 2026-04-23 14:35:12 +01:00
Nuno Cruces 3f5df10c4b Avoid holes. 2026-04-20 10:15:30 +01:00
Nuno Cruces 9dba3c7490 Fix #376. 2026-04-18 01:11:07 +01:00
Nuno Cruces 4ba802708b Libc math. 2026-04-17 16:10:20 +01:00
Nuno Cruces 10ce179087 Libc fixes. 2026-04-17 15:30:53 +01:00
Nuno Cruces 064bd0131f Base64. 2026-04-16 12:41:52 +01:00
Nuno Cruces 0985a110f6 Faster arrays. 2026-04-15 18:15:25 +01:00
Nuno CrucesandGitHub 85405c4f38 Vec1 vector extension (#375) 2026-04-15 16:32:32 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a2af69a0fe Bump cross-platform-actions/action from 0.32.0 to 1.0.0 (#374)
Bumps [cross-platform-actions/action](https://github.com/cross-platform-actions/action) from 0.32.0 to 1.0.0.
- [Release notes](https://github.com/cross-platform-actions/action/releases)
- [Changelog](https://github.com/cross-platform-actions/action/blob/master/changelog.md)
- [Commits](https://github.com/cross-platform-actions/action/compare/v0.32.0...v1.0.0)

---
updated-dependencies:
- dependency-name: cross-platform-actions/action
  dependency-version: 1.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-14 04:43:40 +01:00
Nuno Cruces 692e090377 Deps. 2026-04-10 00:45:36 +01:00
Nuno Cruces 02af344355 SQLite 3.53.0. 2026-04-10 00:21:45 +01:00
Nuno Cruces 61d2c859c5 CI. 2026-04-02 19:56:49 +01:00
Nuno Cruces 7daf908f50 Portable. 2026-03-29 12:47:10 +01:00
Nuno Cruces 13edc95f12 Regen. 2026-03-28 09:55:31 +00:00
Nuno Cruces b6e44cc408 Regen. 2026-03-25 15:09:35 +00:00
Nuno Cruces 2c6806ebbd Regen. 2026-03-21 10:37:12 +00:00
Nuno Cruces 361fdc52fa SQLite 3.51.3. 2026-03-18 17:56:27 +00:00
Nuno Cruces 1008ddc5cb JSON. 2026-03-15 11:10:12 +00:00
Nuno Cruces 1956c5a861 Cleanup. 2026-03-13 17:16:56 +00:00
Nuno CrucesandGitHub 1165211c3a Switch from wazero to wasm2go (#362) 2026-03-12 20:53:33 +00:00
Nuno Cruces 137e2694c0 Fix arena. 2026-03-09 21:12:30 +00:00
Nuno CrucesandGitHub d29aacb0b2 Compile SQLite with nostdlib. (#351) 2026-03-09 16:15:28 +00:00
Nuno Cruces 30649fa50a Noncombining. 2026-03-05 16:33:57 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ed9a77fa96 Bump docker/setup-qemu-action from 3 to 4 (#360)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-05 15:12:28 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
4c0daafbf7 Bump actions/attest-build-provenance from 3 to 4 (#356)
Bumps [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) from 3 to 4.
- [Release notes](https://github.com/actions/attest-build-provenance/releases)
- [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md)
- [Commits](https://github.com/actions/attest-build-provenance/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/attest-build-provenance
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-27 09:08:07 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1ad454cfae Bump golang.org/x/crypto from 0.47.0 to 0.48.0 (#353)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.47.0 to 0.48.0.
- [Commits](https://github.com/golang/crypto/compare/v0.47.0...v0.48.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-10 07:40:07 +00:00
Nuno Cruces ed7dab3edd More sqlite-createtable-parser. 2026-02-06 11:18:52 +00:00
Nuno Cruces a68b5f4364 Math benchmarks. 2026-02-05 09:42:01 +00:00
Nuno Cruces 299c57d1b8 nostdlib. 2026-01-29 17:04:45 +00:00
Nuno Cruces 501f863d20 New sqlite-createtable-parser. 2026-01-26 14:23:44 +00:00
Nuno Cruces 33bea5277f Flags. 2026-01-24 13:35:09 +00:00
Nuno Cruces c263381f1f Fix #349. 2026-01-24 12:48:33 +00:00
Nuno Cruces c4f254bc37 SQLite 3.51.2. 2026-01-16 14:55:24 +00:00
Nuno Cruces c97c5a1f86 Type affinity. 2026-01-15 14:17:06 +00:00
Nuno Cruces 3e1b1313c1 Unquote identifiers. 2026-01-15 12:59:46 +00:00
Nuno Cruces 83744cb9c8 More sqlite-createtable-parser. 2026-01-14 18:11:59 +00:00
Nuno Cruces a36d72c2dc Stricter vtabs, sqlite-createtable-parser. 2026-01-13 14:23:55 +00:00
Nuno Cruces e7f5604199 Stricter vtabs, get VFS name. 2026-01-13 11:25:25 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e50083912c Bump golang.org/x/crypto from 0.46.0 to 0.47.0 (#348)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.46.0 to 0.47.0.
- [Commits](https://github.com/golang/crypto/compare/v0.46.0...v0.47.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.47.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-13 08:03:31 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
d4764fb2fa Bump golang.org/x/sys from 0.39.0 to 0.40.0 (#346)
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.39.0 to 0.40.0.
- [Commits](https://github.com/golang/sys/compare/v0.39.0...v0.40.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-08 23:12:26 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
42df71f3ff Bump github.com/ncruces/wbt from 0.2.0 to 1.0.0 (#345)
Bumps [github.com/ncruces/wbt](https://github.com/ncruces/wbt) from 0.2.0 to 1.0.0.
- [Release notes](https://github.com/ncruces/wbt/releases)
- [Commits](https://github.com/ncruces/wbt/compare/v0.2.0...v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/ncruces/wbt
  dependency-version: 1.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-06 23:55:44 +00:00
Nuno Cruces 91e969c06b Example. 2026-01-05 12:49:21 +00:00
Nuno Cruces 8ab0ddf53e More temp files. 2025-12-29 20:45:44 +00:00
Nuno Cruces 74d22ded0a Move Litestream. 2025-12-29 12:20:05 +00:00
Nuno Cruces d962611796 Bump cross-platform-actions/action from 0.31.0 to 0.32.0 2025-12-21 13:39:41 +00:00
Nuno Cruces 7df3814c34 Improved driver metadata. 2025-12-20 22:23:05 +00:00
Nuno Cruces c5f49b835a Reduce mutex scope, use temp files. 2025-12-19 16:37:47 +00:00
Nuno Cruces 0e55451a0b Deps. 2025-12-19 12:28:57 +00:00
Nuno Cruces ea9a58ab19 Remove singleflight. 2025-12-17 13:17:50 +00:00
Nuno Cruces 0b46e74ea6 Bump cross-platform-actions/action from 0.30.0 to 0.31.0 2025-12-16 12:34:55 +00:00
Nuno Cruces 8dca850bee Stricter floats. 2025-12-12 17:21:33 +00:00
Nuno Cruces 5b78823416 ScanColumn dropped. 2025-12-12 13:06:37 +00:00
Nuno Cruces 1764a571da Litestream v0.5.3. 2025-12-12 12:44:57 +00:00
Nuno Cruces 9837310af7 Fuzz time shift. 2025-12-10 16:15:44 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ec8961a621 Bump golang.org/x/crypto from 0.45.0 to 0.46.0 (#341)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.45.0 to 0.46.0.
- [Commits](https://github.com/golang/crypto/compare/v0.45.0...v0.46.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.46.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 00:49:00 +00:00
Nuno Cruces 8c37aa2d97 Remove inprocess primary. 2025-12-04 16:31:41 +00:00
Nuno Cruces ca93c498e7 Relative time, fixes. 2025-12-04 15:55:39 +00:00
Nuno Cruces 15e9087fa8 Time travel pragma. 2025-12-03 15:01:04 +00:00
Nuno Cruces 7028e3a5b9 SQLite 3.51.1. 2025-11-30 10:24:34 +00:00
Nuno Cruces 03bb20de6e Lock tweaks. 2025-11-29 16:12:29 +00:00
Nuno Cruces 20a51a344e Remove MinLevel. 2025-11-27 17:29:47 +00:00
Nuno Cruces 2dbcc480f7 Initial pragma support. 2025-11-27 16:28:16 +00:00
Nuno Cruces 0f0716c438 Inprocess Litestream primary. 2025-11-26 14:07:44 +00:00
Nuno CrucesandGitHub 0286e50e25 Experimental file control opcode for write transaction (#339) 2025-11-21 12:48:31 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
8ac10eb8b4 Bump actions/checkout from 5 to 6 (#338)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-20 22:48:29 +00:00
Nuno Cruces 0ff41bb966 Deps. 2025-11-20 18:47:59 +00:00
Nuno Cruces ba9caf0405 Shared page cache. 2025-11-20 18:30:51 +00:00
Nuno Cruces 2c167dd116 Avoid polling intermediate levels. 2025-11-19 16:22:10 +00:00
Nuno Cruces ce0da893b4 Fix #335. 2025-11-19 11:27:42 +00:00
Nuno Cruces 9bbbab77f6 Fix define. 2025-11-18 17:58:56 +00:00
Nuno Cruces bab2d26652 Deps. 2025-11-11 23:03:31 +00:00
Nuno Cruces 3132b272de Test more, log less. 2025-11-10 11:26:28 +00:00
Nuno CrucesandGitHub 8f9a6ca4c1 Litestream lightweight read-replicas. (#328) 2025-11-09 13:16:08 +00:00
Nuno Cruces 99b097de3b Windows ARM runners. 2025-11-09 12:44:32 +00:00
Nuno Cruces 4a956e80a2 wasi-sdk-28. 2025-11-09 01:32:25 +00:00
Nuno Cruces 5f4ff03f6f wazero v1.10.0. 2025-11-09 01:32:15 +00:00
Nuno Cruces 5890049488 Shim modernc. 2025-11-09 01:32:14 +00:00
Nuno Cruces 5e73c5d714 Issue #330. 2025-11-06 12:07:37 +00:00
Nuno Cruces 6d92aa16ef SQLite 3.51.0. 2025-11-05 12:30:09 +00:00
Nuno Cruces 191d1337e7 Gorm v1.31.1. 2025-11-05 12:30:09 +00:00
Nuno CrucesandGitHub b65e894849 Improve error reporting. (#327) 2025-10-17 16:40:15 +01:00
Nuno Cruces 0b040d3f09 Prepare 3.51.0. 2025-10-16 15:18:44 +01:00
Nuno Cruces 1db4366226 VFS error handling. 2025-10-16 15:18:40 +01:00
Nuno Cruces 9e1cbfb5bb Release snapshot. 2025-10-10 17:37:44 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7f2d70a0f3 Bump golang.org/x/crypto from 0.42.0 to 0.43.0 (#324)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.42.0 to 0.43.0.
- [Commits](https://github.com/golang/crypto/compare/v0.42.0...v0.43.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 23:11:06 +01:00
Nuno Cruces ea860e407d Docs. 2025-10-01 11:00:13 +01:00
Nuno Cruces d4561d08f9 Refactor. 2025-10-01 10:48:54 +01:00
Nuno Cruces 14c1e490b4 Optimize fullfsync. 2025-09-30 12:54:18 +01:00
Nuno Cruces 23aad5f62f MVCC API. 2025-09-29 12:52:01 +01:00
Nuno Cruces e5bd10a1ff Fix TestDB. 2025-09-29 12:44:09 +01:00
Nuno Cruces 5cf06c45f7 Scan improvements. 2025-09-24 18:13:26 +01:00
Nuno Cruces 08f9fc758a JSON experiment. 2025-09-24 18:13:21 +01:00
Nuno Cruces b588d5f991 Error messages, test contexts. 2025-09-23 12:51:36 +01:00
Nuno Cruces 4c24bd0cb6 Verify download. 2025-09-23 11:49:59 +01:00
Nuno Cruces cc353e4848 Time improvements. 2025-09-23 11:49:53 +01:00
Nuno Cruces c3ebb04045 Use crypto/pbkdf2. 2025-09-18 18:41:10 +01:00
Nuno Cruces 11e064574c Weight-balanced trees. 2025-09-17 11:01:24 +01:00
Nuno Cruces 770420289a Updated dependencies. 2025-09-10 17:09:00 +01:00
Nuno Cruces 62f69011f1 Updated dependencies. 2025-09-08 13:59:41 +01:00
Nuno Cruces 4f9e3f900b binaryen-version_124. 2025-09-08 12:23:58 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
4e90618350 Bump actions/setup-go from 5 to 6 (#318)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-05 00:25:35 +02:00
Nuno CrucesandGitHub 54bb94ce58 Improve WithMemoryCapacityFromMax (#317). 2025-09-03 09:03:59 +02:00
Nuno CrucesandGitHub 07fec784e1 Grow memory geometrically. (#316) 2025-09-01 12:57:33 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
da4638cbff Bump actions/attest-build-provenance from 2 to 3 (#313)
Bumps [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) from 2 to 3.
- [Release notes](https://github.com/actions/attest-build-provenance/releases)
- [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md)
- [Commits](https://github.com/actions/attest-build-provenance/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/attest-build-provenance
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 00:19:53 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
085872c2f3 Bump github.com/ncruces/aa from 0.3.1 to 0.3.2 (#311)
Bumps [github.com/ncruces/aa](https://github.com/ncruces/aa) from 0.3.1 to 0.3.2.
- [Commits](https://github.com/ncruces/aa/compare/v0.3.1...v0.3.2)

---
updated-dependencies:
- dependency-name: github.com/ncruces/aa
  dependency-version: 0.3.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-26 17:20:13 +02:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
de49aa2b06 Bump github.com/ncruces/aa from 0.3.0 to 0.3.1 (#310)
Bumps [github.com/ncruces/aa](https://github.com/ncruces/aa) from 0.3.0 to 0.3.1.
- [Commits](https://github.com/ncruces/aa/compare/v0.3.0...v0.3.1)

---
updated-dependencies:
- dependency-name: github.com/ncruces/aa
  dependency-version: 0.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-23 05:21:11 +01:00
Nuno Cruces 1f3ad0165e SQLite 3.50.4. 2025-08-21 19:05:44 +01:00
Nuno Cruces 0bda48d1d9 Gorm v1.30.1. 2025-08-21 18:56:05 +01:00
Nuno CrucesandGitHub 0026bc91aa MVCC memory VFS. (#309) 2025-08-21 18:44:40 +01:00
Nuno Cruces d84ca9d627 Fix #308. 2025-08-16 19:45:10 +01:00
Nuno Cruces 5d14e01f94 Fix #304. 2025-08-16 19:27:00 +01:00
Nuno CrucesandGitHub 342df983d4 Fix #305. 2025-08-14 23:46:48 +01:00
Nuno Cruces 00476fb1e2 Tests. 2025-08-14 15:04:10 +01:00
Nuno Cruces 8a64ee6eaa Implement RowsColumnScanner. 2025-08-14 01:42:00 +01:00
Nuno Cruces 8f9a8e2752 Learnings from truffle. 2025-08-13 13:10:50 +01:00
Nuno Cruces d8880e4cee Fixes. 2025-08-13 03:34:21 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
4b154a842c Bump actions/checkout from 4 to 5 (#303)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 08:47:41 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
758a53e9bf Bump golang.org/x/crypto from 0.40.0 to 0.41.0 (#300)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.40.0 to 0.41.0.
- [Commits](https://github.com/golang/crypto/compare/v0.40.0...v0.41.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-08 00:30:25 +01:00
Nuno Cruces 1a42b4c590 Better fuzzing. 2025-08-07 16:27:45 +01:00
Nuno Cruces 7e4ec1df1c Fix #299. 2025-08-07 01:52:01 +01:00
Nuno Cruces 2c582a1d66 VFS improvements. 2025-08-05 14:15:21 +01:00
Nuno Cruces 20a67ca669 WAL mode serdes. 2025-08-02 11:48:37 +01:00
Nuno Cruces 789e2dc136 wasi-sdk-27. 2025-07-29 16:50:07 +01:00
Nuno Cruces 0399f10c06 VFS improvements. 2025-07-23 09:57:53 +01:00
Nuno Cruces 75c6744b5b FreeBSD 14.3. 2025-07-22 23:47:22 +01:00
Nuno Cruces 754e806164 Tests. 2025-07-22 10:34:30 +01:00
Nuno Cruces 2640c9fb54 SQLite 3.50.3. 2025-07-17 19:42:01 +01:00
Nuno Cruces 9719d4b0e3 Better tests. 2025-07-17 01:11:16 +01:00
Nuno Cruces b21c69dc1f Fix mode. 2025-07-17 00:50:39 +01:00
Nuno Cruces b0f8ff44a5 Support subtype. 2025-07-17 00:47:04 +01:00
Nuno Cruces f37bca6a80 Speedup memdb. 2025-07-15 23:08:41 +01:00
Nuno Cruces b4e8fcb752 Avoid UB. 2025-07-15 15:52:39 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
14b98a5d05 Bump golang.org/x/crypto from 0.39.0 to 0.40.0 (#295)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.39.0 to 0.40.0.
- [Commits](https://github.com/golang/crypto/compare/v0.39.0...v0.40.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-11 00:30:01 +01:00
Nuno Cruces 36a62264f9 Remove unneeded generics. 2025-07-10 13:05:55 +01:00
Nuno Cruces 33ea564f38 Deps. 2025-07-10 00:50:14 +01:00
Nuno Cruces 5c55d8692f Fix bitset. 2025-07-04 17:10:40 +01:00
Nuno Cruces be2f3036b4 SQLite 3.50.2. 2025-06-30 12:29:54 +01:00
Nuno Cruces 784f82f42f Avoid UB. 2025-06-25 15:27:11 +01:00
Nuno Cruces cd6ba43e77 Less SIMD. 2025-06-24 02:23:54 +01:00
Nuno Cruces d7aef63844 Naming, volatile. 2025-06-20 12:45:42 +01:00
Nuno Cruces 64e5046f10 Improved byteset search. 2025-06-17 11:36:53 +01:00
Nuno Cruces 0bdce8aa68 Avoid overflow. 2025-06-12 15:12:20 +01:00
Nuno Cruces 69a2881a10 SQLite 3.50.1. 2025-06-08 00:38:01 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
24ad4445f1 Bump golang.org/x/crypto from 0.38.0 to 0.39.0 (#285)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.38.0 to 0.39.0.
- [Commits](https://github.com/golang/crypto/compare/v0.38.0...v0.39.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-05 23:49:59 +01:00
Nuno Cruces c159bbd88f Docs, tweaks. 2025-06-04 12:19:01 +01:00
Nuno Cruces c90f8205f7 Remove. 2025-06-03 12:54:56 +01:00
Nuno Cruces b64b9b0415 Better strcasestr. 2025-06-02 10:25:10 +01:00
Nuno Cruces 9142e19d61 SQLite 3.50.0. 2025-05-31 01:14:16 +01:00
DaenneyandGitHub 4a76f2b064 README: Make the testing section simpler to read (#282) 2025-05-30 14:29:34 +01:00
Nuno Cruces c9b364507e Avoid copy, alloc. 2025-05-27 11:35:43 +01:00
Nuno Cruces 2204b96ff6 Gorm. 2025-05-27 10:51:42 +01:00
Nuno Cruces b46f480d79 FCNTL_NULL_IO. 2025-05-26 16:32:52 +01:00
Nuno Cruces 040a026925 More examples. 2025-05-26 12:26:14 +01:00
Michael LynchandGitHub e678040a4e Remove duplicate link to 'custom VFSes' in README (#279) 2025-05-26 11:49:03 +01:00
Michael LynchandGitHub f1cc12569c Link to example of incremental BLOB I/O usage (#280) 2025-05-26 11:48:53 +01:00
Nuno Cruces 721a987e0e Line numbers. 2025-05-26 11:48:02 +01:00
Michael LynchandGitHub f3d65142cc Use sql.Named to clarify the blobio example (#281)
I find the example SQL queries a bit difficult to read at the callsite with the magic numbers and ? placeholders. I think sql.Named makes it more obvious to the reader what the different parts of the SQL query represent.
2025-05-25 19:02:19 +01:00
Nuno Cruces 93f711c77b More fuzzing. 2025-05-21 11:36:56 +01:00
Nuno Cruces 341bd063e8 More fuzzing. 2025-05-21 08:16:52 +01:00
Nuno Cruces f765882670 Sunday's Quick Search. 2025-05-17 13:20:11 +01:00
Nuno Cruces ff3676ff4a Case insensitive search, fixes. 2025-05-16 13:10:03 +01:00
Nuno Cruces 54877a53cd Differential fuzzing. 2025-05-15 14:55:34 +01:00
Nuno Cruces fccc6c10a7 Issue #277. 2025-05-14 13:46:01 +01:00
Nuno Cruces fc21ffcc71 Tweaks. 2025-05-14 01:10:44 +01:00
Nuno Cruces 687e643d7a Case insensitive compare. 2025-05-13 18:12:14 +01:00
Nuno Cruces fc5ced209c Fix bcmp. 2025-05-13 14:28:21 +01:00
Nuno Cruces c1bed07e3a Issue #276. 2025-05-13 14:27:04 +01:00
Nuno Cruces a0771f2363 Relaxed SIMD. 2025-05-12 17:30:32 +01:00
Nuno Cruces 6bad547d3d Unknown haystack length. 2025-05-12 12:00:06 +01:00
Nuno Cruces c2c1aea578 Handle repetitive needles. 2025-05-10 01:44:25 +01:00
Nuno Cruces 60ab485b29 Comments. 2025-05-09 14:21:23 +01:00
Nuno CrucesandGitHub e17a432fde Adds strstr and memmem. (#275) 2025-05-09 00:59:39 +01:00
Nuno Cruces c780ef16e2 SQLite 3.49.2. 2025-05-07 14:08:18 +01:00
Nuno Cruces b609930142 Refactor #274. 2025-05-07 12:46:13 +01:00
Nuno Cruces fd165ce724 Issue #274. 2025-05-07 01:37:52 +01:00
Nuno Cruces d3973b23e3 More memcmp. 2025-05-06 15:48:58 +01:00
Nuno Cruces 320b68e74f More tests. 2025-05-05 14:47:43 +01:00
Nuno Cruces 2c3850e5d1 Reuse fast funcs. 2025-05-03 01:18:10 +01:00
Nuno Cruces db7aacff9f Add strrchr. 2025-05-02 14:35:14 +01:00
Nuno Cruces d748d98e39 Fix. 2025-05-01 12:49:38 +01:00
Nuno Cruces 13b8642384 Compile as C++. 2025-04-29 14:03:59 +01:00
Nuno Cruces 29c5c816cb More libc. 2025-04-27 23:35:13 +01:00
Nuno Cruces b32db76da6 Fix flake. 2025-04-25 00:40:52 +01:00
Nuno Cruces 383f620a1e Action permissions. 2025-04-25 00:28:03 +01:00
Nuno Cruces a3c3515e96 Update binaries. 2025-04-25 00:20:26 +01:00
Nuno Cruces e580f080b9 Test libc. 2025-04-24 23:59:53 +01:00
Nuno Cruces 9ea7099c24 Size optimized versions. 2025-04-24 23:17:30 +01:00
Nuno Cruces 29aa365806 Fix. 2025-04-24 00:41:48 +01:00
Nuno Cruces bb87a920f7 Fix memchr. 2025-04-22 02:27:50 +01:00
Nuno Cruces 48379336dc Improve strspn. 2025-04-21 18:59:20 +01:00
Nuno Cruces 251a92fa1a Weak symbols. 2025-04-18 14:50:57 +01:00
Nuno Cruces f5206ea8da Shellsort. 2025-04-18 09:56:18 +01:00
Nuno Cruces 68ef4593d6 Make libc easier to use. 2025-04-17 13:55:44 +01:00
Nuno Cruces 79bf171210 Fix #263. 2025-04-16 16:39:36 +01:00
Nuno CrucesandGitHub ad16d329ea Optimize strlen and strchr on ARM (#262) 2025-04-15 00:44:31 +01:00
Nuno CrucesandGitHub 9706fa9607 Benchmark more CPUs 2025-04-13 13:28:36 +01:00
Nuno Cruces 45494f5fb6 Redundant defers. 2025-04-09 10:21:54 +01:00
Nuno Cruces 1b0bf3495e Fix flake. 2025-04-09 10:21:54 +01:00
Nuno Cruces 73ac7e06f6 Use SIMD libc. 2025-04-09 10:21:44 +01:00
Nuno Cruces a3ce8f9de5 More. 2025-04-07 01:32:15 +01:00
Nuno Cruces 2043d5fca4 Benchmark. 2025-04-06 23:24:34 +01:00
Nuno Cruces 3bd11a0a86 More SIMD. 2025-04-05 11:16:47 +01:00
Nuno Cruces 39f3fa64eb More SIMD. 2025-04-05 02:03:31 +01:00
Nuno Cruces 4c19387535 SIMD. 2025-04-04 17:47:20 +01:00
Nuno Cruces e6c9f18934 Benchmark libc. 2025-04-04 10:56:12 +01:00
Nuno Cruces 970eb6a2f9 Fix. 2025-04-02 15:33:21 +01:00
Nuno Cruces fac27b8bab libc. 2025-04-02 11:55:20 +01:00
Nuno Cruces 9f626b2f52 Fix #255. 2025-03-31 16:33:31 +01:00
Nuno CrucesandGitHub 1f5d8bf7df Avoid escaping times (#256) 2025-03-31 13:02:41 +01:00
Nuno Cruces 41dc46af7e Optimize errors a bit. 2025-03-28 17:01:04 +00:00
Nuno Cruces e5c285b783 Discussion #250. 2025-03-28 11:30:47 +00:00
Nuno Cruces 6290a14990 Fix interrupt race. 2025-03-26 19:02:14 +00:00
Nuno CrucesandGitHub 948641194b Rework context cancellation. (#251) 2025-03-26 11:39:06 +00:00
Nuno Cruces befed7cf23 binaryen-version_123. 2025-03-26 11:25:54 +00:00
Nuno CrucesandGitHub 3547d9ffb0 Fix WAL flakiness on Windows (#254) 2025-03-26 10:17:19 +00:00
Nuno CrucesandGitHub a67165eb09 Fix WAL flakiness on Windows (#253) 2025-03-26 02:13:30 +00:00
Nuno Cruces 0ba393199a Mitigate #252. 2025-03-25 23:36:57 +00:00
Nuno Cruces 9e4258bc46 Better fix #249. 2025-03-25 12:45:27 +00:00
Nuno CrucesandGitHub b645721d10 IP/CIDR functions. (#246) 2025-03-24 22:38:22 +00:00
Nuno Cruces 6c296231a5 Fix #249. 2025-03-24 19:59:19 +00:00
Nuno Cruces c067e3630b Improve SetInterrupt performance. 2025-03-24 10:41:50 +00:00
Nuno CrucesandGitHub 35a2dbd847 Improve context cancellation performance. (#248) 2025-03-21 11:06:29 +00:00
Nuno Cruces b36f73c66d Optimize. 2025-03-17 12:24:36 +00:00
Nuno Cruces d36f19fd91 Docs. 2025-03-14 11:37:48 +00:00
Nuno Cruces eba71b1f42 Go 1.24.1. 2025-03-14 00:54:12 +00:00
Nuno Cruces d78239bfbf More EINTR. 2025-03-14 00:07:09 +00:00
Nuno Cruces 49852732b2 Optimize. 2025-03-12 17:29:12 +00:00
Nuno Cruces 9b90d076cb Update README.md 2025-03-12 12:01:13 +00:00
Nuno Cruces 15b94577b1 Tweak. 2025-03-11 20:15:53 +00:00
Nuno Cruces 25557244cc Global ConfigLog. 2025-03-11 17:07:56 +00:00
Nuno Cruces c2d3bf0cfc Reduce flakyness. 2025-03-11 14:57:48 +00:00
Nuno Cruces 58a5682084 Handle EINTR. 2025-03-11 12:07:14 +00:00
Nuno Cruces 1ed954e96f Fix #243. 2025-03-10 14:54:34 +00:00
Nuno Cruces 9e7a0a875d Improved arg reuse. 2025-03-10 12:01:15 +00:00
Nuno CrucesandGitHub 26adda4529 Seq aggregate functions (#229) 2025-03-08 14:07:43 +00:00
Nuno Cruces 2f6cd8de1d Docs. 2025-03-07 11:47:02 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e027e055ff Bump golang.org/x/crypto from 0.35.0 to 0.36.0 (#239)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.35.0 to 0.36.0.
- [Commits](https://github.com/golang/crypto/compare/v0.35.0...v0.36.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-05 23:11:20 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
63fdc141e5 Bump golang.org/x/text from 0.22.0 to 0.23.0 (#240)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.22.0 to 0.23.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-05 22:57:38 +00:00
Nuno Cruces 0bbd145a49 Update modules. 2025-02-28 16:57:25 +00:00
Nuno Cruces c755ef96e6 Export logging. 2025-02-28 14:50:22 +00:00
Nuno Cruces 9a69e407cc Fix #235. 2025-02-28 00:33:45 +00:00
Nuno Cruces e9db0d8e84 Issue #233. 2025-02-27 00:07:49 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dadf53e175 Bump golang.org/x/crypto from 0.33.0 to 0.35.0 (#231)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.33.0 to 0.35.0.
- [Commits](https://github.com/golang/crypto/compare/v0.33.0...v0.35.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-24 23:15:52 +00:00
Nuno Cruces f536765206 Go 1.23. 2025-02-24 14:09:55 +00:00
Nuno Cruces 12034c4f0b Retract. 2025-02-24 14:09:55 +00:00
Nuno Cruces b4e5d1a213 Issue #230. 2025-02-24 13:13:25 +00:00
Nuno Cruces b06c7dda6c Checksum robustness. 2025-02-24 13:13:25 +00:00
Nuno Cruces 5e1909a20e Issue #230. 2025-02-24 13:13:25 +00:00
Nuno Cruces 77d74baca5 Fix potential leak. 2025-02-22 12:48:41 +00:00
329 changed files with 9459 additions and 8460 deletions
-15
View File
@@ -1,15 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
echo 'set -eu' > test.sh
for p in $(go list ./...); do
dir=".${p#github.com/ncruces/go-sqlite3}"
name="$(basename "$p").test"
(cd ${dir}; go test -c ${BUILDFLAGS:-})
[ -f "${dir}/${name}" ] && echo "(cd ${dir}; ./${name} ${TESTFLAGS:-})" >> test.sh
done
if [[ -v VMACTIONS ]]; then
envsubst < .github/actions/vmactions/template.yml > .github/actions/vmactions/action.yml
fi
-34
View File
@@ -1,34 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
if [[ "$OSTYPE" == "linux"* ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-25/wasi-sdk-25.0-x86_64-linux.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_122/binaryen-version_122-x86_64-linux.tar.gz"
elif [[ "$OSTYPE" == "darwin"* ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-25/wasi-sdk-25.0-arm64-macos.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_122/binaryen-version_122-arm64-macos.tar.gz"
elif [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
WASI_SDK="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-25/wasi-sdk-25.0-x86_64-windows.tar.gz"
BINARYEN="https://github.com/WebAssembly/binaryen/releases/download/version_122/binaryen-version_122-x86_64-windows.tar.gz"
fi
# Download tools
mkdir -p tools/
[ -d "tools/wasi-sdk" ] || curl -#L "$WASI_SDK" | tar xzC tools &
[ -d "tools/binaryen" ] || curl -#L "$BINARYEN" | tar xzC tools &
wait
[ -d "tools/wasi-sdk" ] || mv "tools/wasi-sdk"* "tools/wasi-sdk"
[ -d "tools/binaryen" ] || mv "tools/binaryen"* "tools/binaryen"
# Download and build SQLite
sqlite3/download.sh
embed/build.sh
embed/bcw2/build.sh
# Download and build sqlite-createtable-parser
util/sql3util/wasm/download.sh
util/sql3util/wasm/build.sh
# Check diffs
git diff --exit-code
-32
View File
@@ -1,32 +0,0 @@
name: Reproducible build
on:
workflow_dispatch:
permissions:
contents: read
id-token: write
attestations: write
jobs:
build:
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: ilammy/msvc-dev-cmd@v1
- uses: actions/checkout@v4
- name: Build
shell: bash
run: .github/workflows/repro.sh
- uses: actions/attest-build-provenance@v2
if: matrix.os == 'ubuntu-latest'
with:
subject-path: |
embed/sqlite3.wasm
embed/bcw2/bcw2.wasm
util/sql3util/wasm/sql3parse_table.wasm
-231
View File
@@ -1,231 +0,0 @@
name: Test
on:
push:
branches: [ 'main' ]
paths:
- '**.go'
- '**.mod'
- '**.wasm'
- '**.yml'
pull_request:
branches: [ 'main' ]
paths:
- '**.go'
- '**.mod'
- '**.wasm'
- '**.yml'
workflow_dispatch:
jobs:
test:
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Format
run: gofmt -s -w . && git diff --exit-code
if: matrix.os != 'windows-latest'
- name: Tidy
run: go mod tidy && git diff --exit-code
- name: Download
run: go mod download
- name: Verify
run: go mod verify
- name: Vet
run: go vet ./...
- name: Build
run: go build -v ./...
- name: Test
run: go test -v ./... -bench . -benchtime=1x
- name: Test BSD locks
run: go test -v -tags sqlite3_flock ./...
if: matrix.os != 'windows-latest'
- name: Test dot locks
run: go test -v -tags sqlite3_dotlk ./...
if: matrix.os != 'windows-latest'
- name: Test modules
shell: bash
run: |
go work init .
go work use -r embed gormlite
go test -v ./embed/bcw2/...
- name: Test GORM
shell: bash
run: gormlite/test.sh
if: matrix.os != 'windows-latest'
- name: Collect coverage
run: |
go get -tool github.com/dave/courtney@v0.4.4
go tool courtney
if: |
github.event_name == 'push' &&
matrix.os == 'ubuntu-latest'
- uses: ncruces/go-coverage-report@v0
with:
coverage-file: coverage.out
chart: true
amend: true
if: |
github.event_name == 'push' &&
matrix.os == 'ubuntu-latest'
test-bsd:
strategy:
matrix:
os:
- name: freebsd
version: '14.2'
flags: '-test.v'
- name: netbsd
version: '10.1'
flags: '-test.v'
- name: freebsd
arch: arm64
version: '14.2'
flags: '-test.v -test.short'
- name: netbsd
arch: arm64
version: '10.1'
flags: '-test.v -test.short'
- name: openbsd
version: '7.6'
flags: '-test.v -test.short'
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- name: Build
env:
GOOS: ${{ matrix.os.name }}
GOARCH: ${{ matrix.os.arch }}
TESTFLAGS: ${{ matrix.os.flags }}
run: .github/workflows/build-test.sh
- name: Test
uses: cross-platform-actions/action@v0.27.0
with:
operating_system: ${{ matrix.os.name }}
architecture: ${{ matrix.os.arch }}
version: ${{ matrix.os.version }}
shell: bash
run: . ./test.sh
sync_files: runner-to-vm
test-vm:
strategy:
matrix:
os:
- name: dragonfly
action: 'vmactions/dragonflybsd-vm@v1'
tflags: '-test.v'
- name: illumos
action: 'vmactions/omnios-vm@v1'
tflags: '-test.v'
- name: solaris
action: 'vmactions/solaris-vm@v1'
bflags: '-tags sqlite3_dotlk'
tflags: '-test.v'
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- name: Build
env:
GOOS: ${{ matrix.os.name }}
BUILDFLAGS: ${{ matrix.os.bflags }}
TESTFLAGS: ${{ matrix.os.tflags }}
VMACTIONS: ${{ matrix.os.action }}
run: .github/workflows/build-test.sh
- name: Test
uses: ./.github/actions/vmactions
test-wasip1:
runs-on: ubuntu-latest
needs: test
steps:
- uses: bytecodealliance/actions/wasmtime/setup@v1
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Set path
run: echo "$(go env GOROOT)/lib/wasm" >> "$GITHUB_PATH"
- name: Test wasmtime
env:
GOOS: wasip1
GOARCH: wasm
GOWASIRUNTIME: wasmtime
GOWASIRUNTIMEARGS: '--env CI=true'
run: go test -v -short -tags sqlite3_dotlk -skip Example ./...
test-qemu:
runs-on: ubuntu-latest
needs: test
steps:
- uses: docker/setup-qemu-action@v3
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Test 386 (32-bit)
run: GOARCH=386 go test -v -short ./...
- name: Test riscv64 (interpreter)
run: GOARCH=riscv64 go test -v -short ./...
- name: Test ppc64le (interpreter)
run: GOARCH=ppc64le go test -v -short ./...
- name: Test s390x (big-endian)
run: GOARCH=s390x go test -v -short -tags sqlite3_dotlk ./...
test-linuxarm:
runs-on: ubuntu-24.04-arm
needs: test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Test
run: go test -v ./...
test-macintel:
runs-on: macos-13
needs: test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with: { go-version: stable }
- name: Test
run: go test -v ./...
+20
View File
@@ -0,0 +1,20 @@
# Fork of ncruces/go-sqlite3. Allowlist UPSTREAM test fixtures + example crypto
# keys (SHA-256/MD5 of the empty string used in the adiantum/XTS VFS tests, and
# the SQLite C amalgamation). Hanzo additions (hanzovfs/, cmd/) are still scanned.
[extend]
useDefault = true
[allowlist]
description = "Upstream SQLite + crypto-VFS test fixtures (not real secrets)"
paths = [
'''sqlite3/sqlite3\.c''',
'''sqlite3/sqlite3ext\.h''',
'''vfs/adiantum/.*''',
'''vfs/xts/.*''',
'''vfs/tests/.*''',
'''tests/.*''',
]
regexes = [
'''e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855''',
'''d41d8cd98f00b204e9800998ecf8427e''',
]
+35
View File
@@ -0,0 +1,35 @@
# CI/CD for hanzoai/platform (the Hanzo PaaS) — NOT GitHub Actions.
# The platform runs `make ci` as a build step and/or a scheduled (cron) task.
# Works identically locally and anywhere: `make ci`.
SHELL := /usr/bin/env bash
export GOPRIVATE := github.com/hanzoai,github.com/luxfi
export GOFLAGS := -mod=mod
BIN := $(CURDIR)/.bin
export PATH := $(BIN):$(PATH)
GITLEAKS_VERSION ?= 8.21.2
ARCH := $(shell uname -m | sed 's/x86_64/x64/; s/aarch64/arm64/')
.PHONY: ci build test scan vuln secrets sbom tools clean
ci: build test scan ## full pipeline (build -> test -> scan)
build:
go build ./...
test:
go test ./...
scan: vuln secrets sbom
vuln:
go install golang.org/x/vuln/cmd/govulncheck@latest
govulncheck ./...
secrets: $(BIN)/gitleaks
gitleaks detect --source . --redact --no-banner --exit-code 1
sbom: $(BIN)/syft
syft . -o cyclonedx-json=sbom.cyclonedx.json -q
tools: $(BIN)/gitleaks $(BIN)/syft
clean:
rm -rf $(BIN) sbom.cyclonedx.json
$(BIN)/gitleaks:
@mkdir -p $(BIN)
curl -sSfL "https://github.com/gitleaks/gitleaks/releases/download/v$(GITLEAKS_VERSION)/gitleaks_$(GITLEAKS_VERSION)_linux_$(ARCH).tar.gz" | tar -xz -C $(BIN) gitleaks
$(BIN)/syft:
@mkdir -p $(BIN)
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b $(BIN) >/dev/null
+7
View File
@@ -0,0 +1,7 @@
github.com/hanzoai/sqlite3 — canonical Hanzo SQLite.
Fork of github.com/ncruces/go-sqlite3 (MIT, Copyright (c) 2023 Nuno Cruces).
The upstream MIT LICENSE is retained in full. Hanzo additions (the hanzovfs
native encrypted VFS, module path, build pins) are likewise MIT-licensed.
Pure-Go, no cgo. WASM SQLite blob from github.com/ncruces/go-sqlite3-wasm (upstream).
+15
View File
@@ -0,0 +1,15 @@
# CI/CD on hanzoai/platform — no GitHub Actions
This repo is built, run, and scanned by **hanzoai/platform** (the Hanzo PaaS),
not GitHub Actions. `.github/workflows` is intentionally absent.
- **Build / run** — the platform builds via Dockerfile / Nixpacks / Railpack and
runs the result; auto-deploy on push.
- **Actions (CI)** — a platform **Scheduled Task** (and/or build step) runs
`make ci` = `build → test → scan` (govulncheck + gitleaks + CycloneDX SBOM),
on push and on a cron. SOC 2 CC7.x/CC8.x evidence (artifacts to hanzoai/audit).
- The platform build env carries Git credentials for `github.com/hanzoai` +
`github.com/luxfi`, so cross-org private modules resolve natively — no per-repo
token dance (the thing that made GitHub Actions painful).
Run anywhere: `make ci`.
+62 -28
View File
@@ -1,4 +1,4 @@
# Go bindings to SQLite using wazero
# Go bindings to SQLite using wasm2go
[![Go Reference](https://pkg.go.dev/badge/image)](https://pkg.go.dev/github.com/ncruces/go-sqlite3)
[![Go Report](https://goreportcard.com/badge/github.com/ncruces/go-sqlite3)](https://goreportcard.com/report/github.com/ncruces/go-sqlite3)
@@ -8,9 +8,9 @@ Go module `github.com/ncruces/go-sqlite3` is a `cgo`-free [SQLite](https://sqlit
It provides a [`database/sql`](https://pkg.go.dev/database/sql) compatible driver,
as well as direct access to most of the [C SQLite API](https://sqlite.org/cintro.html).
It wraps a [Wasm](https://webassembly.org/) [build](embed/) of SQLite,
and uses [wazero](https://wazero.io/) as the runtime.\
Go, wazero and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ direct dependencies.
It wraps a [Wasm](https://webassembly.org/) build of SQLite,
and uses [wasm2go](https://github.com/ncruces/wasm2go) to translate it to Go.\
Go and [`x/sys`](https://pkg.go.dev/golang.org/x/sys) are the _only_ direct dependencies.
### Getting started
@@ -19,7 +19,6 @@ Using the [`database/sql`](https://pkg.go.dev/database/sql) driver:
import "database/sql"
import _ "github.com/ncruces/go-sqlite3/driver"
import _ "github.com/ncruces/go-sqlite3/embed"
var version string
db, _ := sql.Open("sqlite3", "file:demo.db")
@@ -30,12 +29,10 @@ db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
- [`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3)
wraps the [C SQLite API](https://sqlite.org/cintro.html)
([example usage](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-package)).
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-package)).
- [`github.com/ncruces/go-sqlite3/driver`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver)
provides a [`database/sql`](https://pkg.go.dev/database/sql) driver
([example usage](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-package)).
- [`github.com/ncruces/go-sqlite3/embed`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/embed)
embeds a build of SQLite into your application.
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-package)).
- [`github.com/ncruces/go-sqlite3/vfs`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs)
wraps the [C SQLite VFS API](https://sqlite.org/vfs.html) and provides a pure Go implementation.
- [`github.com/ncruces/go-sqlite3/gormlite`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/gormlite)
@@ -44,52 +41,75 @@ db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
### Advanced features
- [incremental BLOB I/O](https://sqlite.org/c3ref/blob_open.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/blobio#example-package))
- [nested transactions](https://sqlite.org/lang_savepoint.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-Savepoint))
- [custom functions](https://sqlite.org/c3ref/create_function.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-Conn.CreateFunction))
- [virtual tables](https://sqlite.org/vtab.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3#example-CreateModule))
- [custom VFSes](https://sqlite.org/vfs.html)
([examples](vfs/README.md#custom-vfses))
- [online backup](https://sqlite.org/backup.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#Conn))
- [JSON support](https://sqlite.org/json1.html)
([example](https://pkg.go.dev/github.com/ncruces/go-sqlite3/driver#example-package-Json))
- [math functions](https://sqlite.org/lang_mathfunc.html)
- [full-text search](https://sqlite.org/fts5.html)
- [geospatial search](https://sqlite.org/geopoly.html)
- [Unicode support](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/unicode)
- [statistics functions](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/stats)
- [encryption at rest](vfs/adiantum/README.md)
- [many extensions](ext/README.md)
- [custom VFSes](vfs/README.md#custom-vfses)
- [and more…](embed/README.md)
### Caveats
This module replaces the SQLite [OS Interface](https://sqlite.org/vfs.html)
(aka VFS) with a [pure Go](vfs/) implementation,
which has advantages and disadvantages.
Read more about the Go VFS design [here](vfs/README.md).
Because each database connection executes within a Wasm sandboxed environment,
memory usage will be higher than alternatives.
### Testing
This project aims for [high test coverage](https://github.com/ncruces/go-sqlite3/wiki/Test-coverage-report).
It also benefits greatly from [SQLite's](https://sqlite.org/testing.html) and
[wazero's](https://tetrate.io/blog/introducing-wazero-from-tetrate/#:~:text=Rock%2Dsolid%20test%20approach) thorough testing.
It also benefits greatly from [SQLite's](https://sqlite.org/testing.html) thorough testing.
Every commit is [tested](https://github.com/ncruces/go-sqlite3/wiki/Support-matrix) on
Linux (amd64/arm64/386/riscv64/ppc64le/s390x), macOS (amd64/arm64),
Windows (amd64), FreeBSD (amd64/arm64), OpenBSD (amd64), NetBSD (amd64/arm64),
DragonFly BSD (amd64), illumos (amd64), and Solaris (amd64).
Every commit is tested on:
* Linux: amd64, arm64, 386, arm, riscv64, ppc64le, loong64, s390x
* macOS: amd64, arm64
* Windows: amd64, arm64
* BSD:
* FreeBSD: amd64, arm64
* NetBSD: amd64, arm64
* DragonFly BSD: amd64
* OpenBSD: amd64
* illumos: amd64
* Solaris: amd64
Certain operating system and CPU combinations have some limitations.
See the [support matrix](https://github.com/ncruces/go-sqlite3/wiki/Support-matrix) for a complete overview.
The Go VFS is tested by running SQLite's
[mptest](https://github.com/sqlite/sqlite/blob/master/mptest/mptest.c).
### Performance
Perfomance of the [`database/sql`](https://pkg.go.dev/database/sql) driver is
Performance of the [`database/sql`](https://pkg.go.dev/database/sql) driver is
[competitive](https://github.com/cvilsmeier/go-sqlite-bench) with alternatives.
The Wasm and VFS layers are also tested by running SQLite's
The Wasm and VFS layers are also benchmarked by running SQLite's
[speedtest1](https://github.com/sqlite/sqlite/blob/master/test/speedtest1.c).
### Concurrency
This module behaves similarly to SQLite in [multi-thread](https://sqlite.org/threadsafe.html) mode:
it is goroutine-safe, provided that no single database connection, or object derived from it,
is used concurrently by multiple goroutines.
The [`database/sql`](https://pkg.go.dev/database/sql) API is safe to use concurrently,
according to its documentation.
### FAQ, issues, new features
For questions, please see [Discussions](https://github.com/ncruces/go-sqlite3/discussions/categories/q-a).
@@ -98,12 +118,26 @@ Also, post there if you used this driver for something interesting
([_"Show and tell"_](https://github.com/ncruces/go-sqlite3/discussions/categories/show-and-tell)),
have an [idea](https://github.com/ncruces/go-sqlite3/discussions/categories/ideas)…
The [Issue](https://github.com/ncruces/go-sqlite3/issues) tracker is for bugs we want fixed,
The [Issue](https://github.com/ncruces/go-sqlite3/issues) tracker is for bugs,
and features we're working on, planning to work on, or asking for help with.
### Alternatives
---
- [`modernc.org/sqlite`](https://pkg.go.dev/modernc.org/sqlite)
- [`crawshaw.io/sqlite`](https://pkg.go.dev/crawshaw.io/sqlite)
- [`github.com/mattn/go-sqlite3`](https://pkg.go.dev/github.com/mattn/go-sqlite3)
- [`github.com/zombiezen/go-sqlite`](https://pkg.go.dev/github.com/zombiezen/go-sqlite)
## Hanzo fork
`github.com/hanzoai/sqlite3` is the **canonical Hanzo SQLite** — a fork of
`ncruces/go-sqlite3` (MIT) adding the **`hanzovfs`** native encrypted VFS for
per-tenant SQLite ⇒ `hanzoai/vfs` ⇒ S3 (HIP-0302 / HIP-0107), no FUSE, no cgo.
```go
import (
"github.com/hanzoai/sqlite3"
_ "github.com/hanzoai/sqlite3/hanzovfs" // registers vfs=hanzo
_ "github.com/hanzoai/sqlite3/embed"
)
db, _ := sqlite3.Open("file:/orgs/acme/app.db?vfs=hanzo")
```
**Deprecates** `modernc.org/sqlite` (no custom-VFS hook) and direct use of
`ncruces/go-sqlite3`. Benchmarked 3.6× faster writes / ~92× faster point-reads
than the FUSE-mounted path; per-page AEAD encryption costs ~0.
+12 -14
View File
@@ -62,22 +62,22 @@ func (src *Conn) BackupInit(srcDB, dstURI string) (*Backup, error) {
}
func (c *Conn) backupInit(dst ptr_t, dstName string, src ptr_t, srcName string) (*Backup, error) {
defer c.arena.mark()()
dstPtr := c.arena.string(dstName)
srcPtr := c.arena.string(srcName)
defer c.arena.Mark()()
dstPtr := c.arena.String(dstName)
srcPtr := c.arena.String(srcName)
other := dst
if c.handle == dst {
other = src
}
ptr := ptr_t(c.call("sqlite3_backup_init",
stk_t(dst), stk_t(dstPtr),
stk_t(src), stk_t(srcPtr)))
ptr := ptr_t(c.wrp.Xsqlite3_backup_init(
int32(dst), int32(dstPtr),
int32(src), int32(srcPtr)))
if ptr == 0 {
defer c.closeDB(other)
rc := res_t(c.call("sqlite3_errcode", stk_t(dst)))
return nil, c.sqlite.error(rc, dst)
rc := res_t(c.wrp.Xsqlite3_errcode(int32(dst)))
return nil, c.errorFor(dst, rc)
}
return &Backup{
@@ -97,7 +97,7 @@ func (b *Backup) Close() error {
return nil
}
rc := res_t(b.c.call("sqlite3_backup_finish", stk_t(b.handle)))
rc := res_t(b.c.wrp.Xsqlite3_backup_finish(int32(b.handle)))
b.c.closeDB(b.otherc)
b.handle = 0
return b.c.error(rc)
@@ -108,7 +108,7 @@ func (b *Backup) Close() error {
//
// https://sqlite.org/c3ref/backup_finish.html#sqlite3backupstep
func (b *Backup) Step(nPage int) (done bool, err error) {
rc := res_t(b.c.call("sqlite3_backup_step", stk_t(b.handle), stk_t(nPage)))
rc := res_t(b.c.wrp.Xsqlite3_backup_step(int32(b.handle), int32(nPage)))
if rc == _DONE {
return true, nil
}
@@ -120,8 +120,7 @@ func (b *Backup) Step(nPage int) (done bool, err error) {
//
// https://sqlite.org/c3ref/backup_finish.html#sqlite3backupremaining
func (b *Backup) Remaining() int {
n := int32(b.c.call("sqlite3_backup_remaining", stk_t(b.handle)))
return int(n)
return int(b.c.wrp.Xsqlite3_backup_remaining(int32(b.handle)))
}
// PageCount returns the total number of pages in the source database
@@ -129,6 +128,5 @@ func (b *Backup) Remaining() int {
//
// https://sqlite.org/c3ref/backup_finish.html#sqlite3backuppagecount
func (b *Backup) PageCount() int {
n := int32(b.c.call("sqlite3_backup_pagecount", stk_t(b.handle)))
return int(n)
return int(b.c.wrp.Xsqlite3_backup_pagecount(int32(b.handle)))
}
+40 -35
View File
@@ -3,7 +3,7 @@ package sqlite3
import (
"io"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3/internal/errutil"
)
// ZeroBlob represents a zero-filled, length n BLOB
@@ -31,29 +31,32 @@ var _ io.ReadWriteSeeker = &Blob{}
//
// https://sqlite.org/c3ref/blob_open.html
func (c *Conn) OpenBlob(db, table, column string, row int64, write bool) (*Blob, error) {
defer c.arena.mark()()
blobPtr := c.arena.new(ptrlen)
dbPtr := c.arena.string(db)
tablePtr := c.arena.string(table)
columnPtr := c.arena.string(column)
if c.interrupt.Err() != nil {
return nil, INTERRUPT
}
defer c.arena.Mark()()
blobPtr := c.arena.New(ptrlen)
dbPtr := c.arena.String(db)
tablePtr := c.arena.String(table)
columnPtr := c.arena.String(column)
var flags int32
if write {
flags = 1
}
c.checkInterrupt(c.handle)
rc := res_t(c.call("sqlite3_blob_open", stk_t(c.handle),
stk_t(dbPtr), stk_t(tablePtr), stk_t(columnPtr),
stk_t(row), stk_t(flags), stk_t(blobPtr)))
rc := res_t(c.wrp.Xsqlite3_blob_open(int32(c.handle),
int32(dbPtr), int32(tablePtr), int32(columnPtr),
row, flags, int32(blobPtr)))
if err := c.error(rc); err != nil {
return nil, err
}
blob := Blob{c: c}
blob.handle = util.Read32[ptr_t](c.mod, blobPtr)
blob.bytes = int64(int32(c.call("sqlite3_blob_bytes", stk_t(blob.handle))))
blob.handle = ptr_t(c.wrp.Read32(blobPtr))
blob.bytes = int64(c.wrp.Xsqlite3_blob_bytes(int32(blob.handle)))
return &blob, nil
}
@@ -67,8 +70,8 @@ func (b *Blob) Close() error {
return nil
}
rc := res_t(b.c.call("sqlite3_blob_close", stk_t(b.handle)))
b.c.free(b.bufptr)
rc := res_t(b.c.wrp.Xsqlite3_blob_close(int32(b.handle)))
b.c.wrp.Free(b.bufptr)
b.handle = 0
return b.c.error(rc)
}
@@ -94,12 +97,12 @@ func (b *Blob) Read(p []byte) (n int, err error) {
want = avail
}
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, want)
b.bufptr = b.c.wrp.Realloc(b.bufptr, want)
b.buflen = want
}
rc := res_t(b.c.call("sqlite3_blob_read", stk_t(b.handle),
stk_t(b.bufptr), stk_t(want), stk_t(b.offset)))
rc := res_t(b.c.wrp.Xsqlite3_blob_read(int32(b.handle),
int32(b.bufptr), int32(want), int32(b.offset)))
err = b.c.error(rc)
if err != nil {
return 0, err
@@ -109,7 +112,7 @@ func (b *Blob) Read(p []byte) (n int, err error) {
err = io.EOF
}
copy(p, util.View(b.c.mod, b.bufptr, want))
copy(p, b.c.wrp.Bytes(b.bufptr, want))
return int(want), err
}
@@ -127,19 +130,19 @@ func (b *Blob) WriteTo(w io.Writer) (n int64, err error) {
want = avail
}
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, want)
b.bufptr = b.c.wrp.Realloc(b.bufptr, want)
b.buflen = want
}
for want > 0 {
rc := res_t(b.c.call("sqlite3_blob_read", stk_t(b.handle),
stk_t(b.bufptr), stk_t(want), stk_t(b.offset)))
rc := res_t(b.c.wrp.Xsqlite3_blob_read(int32(b.handle),
int32(b.bufptr), int32(want), int32(b.offset)))
err = b.c.error(rc)
if err != nil {
return n, err
}
mem := util.View(b.c.mod, b.bufptr, want)
mem := b.c.wrp.Bytes(b.bufptr, want)
m, err := w.Write(mem[:want])
b.offset += int64(m)
n += int64(m)
@@ -165,13 +168,13 @@ func (b *Blob) WriteTo(w io.Writer) (n int64, err error) {
func (b *Blob) Write(p []byte) (n int, err error) {
want := int64(len(p))
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, want)
b.bufptr = b.c.wrp.Realloc(b.bufptr, want)
b.buflen = want
}
util.WriteBytes(b.c.mod, b.bufptr, p)
b.c.wrp.WriteBytes(b.bufptr, p)
rc := res_t(b.c.call("sqlite3_blob_write", stk_t(b.handle),
stk_t(b.bufptr), stk_t(want), stk_t(b.offset)))
rc := res_t(b.c.wrp.Xsqlite3_blob_write(int32(b.handle),
int32(b.bufptr), int32(want), int32(b.offset)))
err = b.c.error(rc)
if err != nil {
return 0, err
@@ -196,16 +199,16 @@ func (b *Blob) ReadFrom(r io.Reader) (n int64, err error) {
want = 1
}
if want > b.buflen {
b.bufptr = b.c.realloc(b.bufptr, want)
b.bufptr = b.c.wrp.Realloc(b.bufptr, want)
b.buflen = want
}
for {
mem := util.View(b.c.mod, b.bufptr, want)
mem := b.c.wrp.Bytes(b.bufptr, want)
m, err := r.Read(mem[:want])
if m > 0 {
rc := res_t(b.c.call("sqlite3_blob_write", stk_t(b.handle),
stk_t(b.bufptr), stk_t(m), stk_t(b.offset)))
rc := res_t(b.c.wrp.Xsqlite3_blob_write(int32(b.handle),
int32(b.bufptr), int32(m), int32(b.offset)))
err := b.c.error(rc)
if err != nil {
return n, err
@@ -234,7 +237,7 @@ func (b *Blob) ReadFrom(r io.Reader) (n int64, err error) {
func (b *Blob) Seek(offset int64, whence int) (int64, error) {
switch whence {
default:
return 0, util.WhenceErr
return 0, errutil.WhenceErr
case io.SeekStart:
break
case io.SeekCurrent:
@@ -243,7 +246,7 @@ func (b *Blob) Seek(offset int64, whence int) (int64, error) {
offset += b.bytes
}
if offset < 0 {
return 0, util.OffsetErr
return 0, errutil.OffsetErr
}
b.offset = offset
return offset, nil
@@ -253,9 +256,11 @@ func (b *Blob) Seek(offset int64, whence int) (int64, error) {
//
// https://sqlite.org/c3ref/blob_reopen.html
func (b *Blob) Reopen(row int64) error {
b.c.checkInterrupt(b.c.handle)
err := b.c.error(res_t(b.c.call("sqlite3_blob_reopen", stk_t(b.handle), stk_t(row))))
b.bytes = int64(int32(b.c.call("sqlite3_blob_bytes", stk_t(b.handle))))
if b.c.interrupt.Err() != nil {
return INTERRUPT
}
err := b.c.error(res_t(b.c.wrp.Xsqlite3_blob_reopen(int32(b.handle), row)))
b.bytes = int64(b.c.wrp.Xsqlite3_blob_bytes(int32(b.handle)))
b.offset = 0
return err
}
+48
View File
@@ -0,0 +1,48 @@
// hanzovfs-bench: native PQ SQLite VFS throughput (in-process, no FUSE).
// go run github.com/hanzoai/sqlite3/cmd/hanzovfs-bench@latest
package main
import (
"fmt"
"os"
"runtime"
"time"
"github.com/hanzoai/sqlite3"
_ "github.com/hanzoai/sqlite3/embed"
"github.com/hanzoai/sqlite3/hanzovfs"
"github.com/luxfi/age"
)
func bench(label, dsn string) {
db, err := sqlite3.Open(dsn)
if err != nil { fmt.Printf("%-24s OPEN ERR %v\n", label, err); return }
defer db.Close()
db.Exec(`PRAGMA journal_mode=MEMORY; PRAGMA synchronous=OFF;`)
db.Exec(`CREATE TABLE t(id INTEGER PRIMARY KEY, email TEXT, body TEXT)`)
const n = 20000
t0 := time.Now()
db.Exec("BEGIN")
st, _, _ := db.Prepare("INSERT INTO t(email,body) VALUES(?,?)")
for i := 0; i < n; i++ {
st.BindText(1, fmt.Sprintf("u%d@hanzo.ai", i))
st.BindText(2, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")
st.Step(); st.Reset()
}
st.Close(); db.Exec("COMMIT")
ins := float64(n) / time.Since(t0).Seconds()
t0 = time.Now()
sel, _, _ := db.Prepare("SELECT body FROM t WHERE id=?")
for i := 0; i < 5000; i++ { sel.BindInt(1, 1+(i*7)%n); sel.Step(); sel.Reset() }
sel.Close()
fmt.Printf("%-24s insert=%9.0f rows/s point_sel=%9.0f q/s\n", label, ins, 5000/time.Since(t0).Seconds())
}
func main() {
fmt.Printf("hanzovfs-bench %s/%s (%d CPU)\n", runtime.GOOS, runtime.GOARCH, runtime.NumCPU())
id, rcpt, err := hanzovfs.GenerateIdentity()
if err != nil { panic(err) }
dir, _ := os.MkdirTemp("", "hvfs")
hanzovfs.Register("pq", hanzovfs.Config{Dir: dir, Recipients: []age.Recipient{rcpt}, Identities: []age.Identity{id}})
bench("native PQ VFS (ML-KEM)", "file:/orgs/acme/bench.db?vfs=pq")
}
+121 -107
View File
@@ -1,14 +1,12 @@
package sqlite3
import (
"context"
"fmt"
"strconv"
"sync/atomic"
"github.com/tetratelabs/wazero/api"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/vfs"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/vfs"
)
// Config makes configuration changes to a database connection.
@@ -29,8 +27,8 @@ func (c *Conn) Config(op DBConfig, arg ...bool) (bool, error) {
// The vararg is a pointer to an array containing these arguments:
// an int and an int* pointing to that int.
defer c.arena.mark()()
argsPtr := c.arena.new(intlen + ptrlen)
defer c.arena.Mark()()
argsPtr := c.arena.New(intlen + ptrlen)
var flag int32
switch {
@@ -40,12 +38,21 @@ func (c *Conn) Config(op DBConfig, arg ...bool) (bool, error) {
flag = 1
}
util.Write32(c.mod, argsPtr+0*ptrlen, flag)
util.Write32(c.mod, argsPtr+1*ptrlen, argsPtr)
c.wrp.Write32(argsPtr+0*ptrlen, uint32(flag))
c.wrp.Write32(argsPtr+1*ptrlen, uint32(argsPtr))
rc := res_t(c.call("sqlite3_db_config", stk_t(c.handle),
stk_t(op), stk_t(argsPtr)))
return util.ReadBool(c.mod, argsPtr), c.error(rc)
rc := res_t(c.wrp.Xsqlite3_db_config(int32(c.handle),
int32(op), int32(argsPtr)))
return c.wrp.ReadBool(argsPtr), c.error(rc)
}
var defaultLogger atomic.Pointer[func(code ExtendedErrorCode, msg string)]
// ConfigLog sets up the default error logging callback for new connections.
//
// https://sqlite.org/errlog.html
func ConfigLog(cb func(code ExtendedErrorCode, msg string)) {
defaultLogger.Store(&cb)
}
// ConfigLog sets up the error logging callback for the connection.
@@ -56,7 +63,7 @@ func (c *Conn) ConfigLog(cb func(code ExtendedErrorCode, msg string)) error {
if cb != nil {
enable = 1
}
rc := res_t(c.call("sqlite3_config_log_go", stk_t(enable)))
rc := res_t(c.wrp.Xsqlite3_config_log_go(enable))
if err := c.error(rc); err != nil {
return err
}
@@ -64,9 +71,9 @@ func (c *Conn) ConfigLog(cb func(code ExtendedErrorCode, msg string)) error {
return nil
}
func logCallback(ctx context.Context, mod api.Module, _ ptr_t, iCode res_t, zMsg ptr_t) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.log != nil {
msg := util.ReadString(mod, zMsg, _MAX_LENGTH)
func (e *env) Xgo_log(_, iCode, zMsg int32) {
if c, ok := e.DB.(*Conn); ok && c.log != nil {
msg := e.ReadString(ptr_t(zMsg), _MAX_LENGTH)
c.log(xErrorCode(iCode), msg)
}
}
@@ -85,12 +92,12 @@ func (c *Conn) Log(code ExtendedErrorCode, format string, a ...any) {
//
// https://sqlite.org/c3ref/file_control.html
func (c *Conn) FileControl(schema string, op FcntlOpcode, arg ...any) (any, error) {
defer c.arena.mark()()
ptr := c.arena.new(max(ptrlen, intlen))
defer c.arena.Mark()()
ptr := c.arena.New(max(ptrlen, intlen))
var schemaPtr ptr_t
if schema != "" {
schemaPtr = c.arena.string(schema)
schemaPtr = c.arena.String(schema)
}
var rc res_t
@@ -99,10 +106,10 @@ func (c *Conn) FileControl(schema string, op FcntlOpcode, arg ...any) (any, erro
default:
return nil, MISUSE
case FCNTL_RESET_CACHE:
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), 0))
case FCNTL_RESET_CACHE, FCNTL_NULL_IO:
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), 0))
case FCNTL_PERSIST_WAL, FCNTL_POWERSAFE_OVERWRITE:
var flag int32
@@ -112,62 +119,66 @@ func (c *Conn) FileControl(schema string, op FcntlOpcode, arg ...any) (any, erro
case arg[0]:
flag = 1
}
util.Write32(c.mod, ptr, flag)
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
ret = util.ReadBool(c.mod, ptr)
c.wrp.Write32(ptr, uint32(flag))
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), int32(ptr)))
ret = c.wrp.ReadBool(ptr)
case FCNTL_CHUNK_SIZE:
util.Write32(c.mod, ptr, int32(arg[0].(int)))
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
c.wrp.Write32(ptr, uint32(arg[0].(int)))
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), int32(ptr)))
case FCNTL_RESERVE_BYTES:
bytes := -1
if len(arg) > 0 {
bytes = arg[0].(int)
}
util.Write32(c.mod, ptr, int32(bytes))
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
ret = int(util.Read32[int32](c.mod, ptr))
c.wrp.Write32(ptr, uint32(bytes))
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), int32(ptr)))
ret = int(int32(c.wrp.Read32(ptr)))
case FCNTL_DATA_VERSION:
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
ret = util.Read32[uint32](c.mod, ptr)
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), int32(ptr)))
ret = c.wrp.Read32(ptr)
case FCNTL_LOCKSTATE:
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
ret = util.Read32[vfs.LockLevel](c.mod, ptr)
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), int32(ptr)))
ret = vfs.LockLevel(c.wrp.Read32(ptr))
case FCNTL_VFS_POINTER:
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
case FCNTL_VFSNAME, FCNTL_VFS_POINTER:
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(FCNTL_VFS_POINTER), int32(ptr)))
if rc == _OK {
const zNameOffset = 16
ptr = util.Read32[ptr_t](c.mod, ptr)
ptr = util.Read32[ptr_t](c.mod, ptr+zNameOffset)
name := util.ReadString(c.mod, ptr, _MAX_NAME)
ret = vfs.Find(name)
ptr = ptr_t(c.wrp.Read32(ptr))
ptr = ptr_t(c.wrp.Read32(ptr + zNameOffset))
name := c.wrp.ReadString(ptr, _MAX_NAME)
if op == FCNTL_VFS_POINTER {
ret = vfs.Find(name)
} else {
ret = name
}
}
case FCNTL_FILE_POINTER, FCNTL_JOURNAL_POINTER:
rc = res_t(c.call("sqlite3_file_control",
stk_t(c.handle), stk_t(schemaPtr),
stk_t(op), stk_t(ptr)))
rc = res_t(c.wrp.Xsqlite3_file_control(
int32(c.handle), int32(schemaPtr),
int32(op), int32(ptr)))
if rc == _OK {
const fileHandleOffset = 4
ptr = util.Read32[ptr_t](c.mod, ptr)
ptr = util.Read32[ptr_t](c.mod, ptr+fileHandleOffset)
ret = util.GetHandle(c.ctx, ptr)
ptr = ptr_t(c.wrp.Read32(ptr))
ptr = ptr_t(c.wrp.Read32(ptr + fileHandleOffset))
ret = c.wrp.GetHandle(ptr)
}
}
@@ -182,8 +193,7 @@ func (c *Conn) FileControl(schema string, op FcntlOpcode, arg ...any) (any, erro
//
// https://sqlite.org/c3ref/limit.html
func (c *Conn) Limit(id LimitCategory, value int) int {
v := int32(c.call("sqlite3_limit", stk_t(c.handle), stk_t(id), stk_t(value)))
return int(v)
return int(c.wrp.Xsqlite3_limit(int32(c.handle), int32(id), int32(value)))
}
// SetAuthorizer registers an authorizer callback with the database connection.
@@ -194,40 +204,39 @@ func (c *Conn) SetAuthorizer(cb func(action AuthorizerActionCode, name3rd, name4
if cb != nil {
enable = 1
}
rc := res_t(c.call("sqlite3_set_authorizer_go", stk_t(c.handle), stk_t(enable)))
rc := res_t(c.wrp.Xsqlite3_set_authorizer_go(int32(c.handle), enable))
if err := c.error(rc); err != nil {
return err
}
c.authorizer = cb
return nil
}
func authorizerCallback(ctx context.Context, mod api.Module, pDB ptr_t, action AuthorizerActionCode, zName3rd, zName4th, zSchema, zInner ptr_t) (rc AuthorizerReturnCode) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.authorizer != nil {
func (e *env) Xgo_authorizer(pDB, action, zName3rd, zName4th, zSchema, zInner int32) (rc int32) {
if c, ok := e.DB.(*Conn); ok && c.handle == ptr_t(pDB) && c.authorizer != nil {
var name3rd, name4th, schema, inner string
if zName3rd != 0 {
name3rd = util.ReadString(mod, zName3rd, _MAX_NAME)
name3rd = e.ReadString(ptr_t(zName3rd), _MAX_NAME)
}
if zName4th != 0 {
name4th = util.ReadString(mod, zName4th, _MAX_NAME)
name4th = e.ReadString(ptr_t(zName4th), _MAX_NAME)
}
if zSchema != 0 {
schema = util.ReadString(mod, zSchema, _MAX_NAME)
schema = e.ReadString(ptr_t(zSchema), _MAX_NAME)
}
if zInner != 0 {
inner = util.ReadString(mod, zInner, _MAX_NAME)
inner = e.ReadString(ptr_t(zInner), _MAX_NAME)
}
rc = c.authorizer(action, name3rd, name4th, schema, inner)
return int32(c.authorizer(AuthorizerActionCode(action), name3rd, name4th, schema, inner))
}
return rc
return _OK
}
// Trace registers a trace callback function against the database connection.
//
// https://sqlite.org/c3ref/trace_v2.html
func (c *Conn) Trace(mask TraceEvent, cb func(evt TraceEvent, arg1 any, arg2 any) error) error {
rc := res_t(c.call("sqlite3_trace_go", stk_t(c.handle), stk_t(mask)))
func (c *Conn) Trace(mask TraceEvent, cb func(evt TraceEvent, arg1, arg2 any) error) error {
rc := res_t(c.wrp.Xsqlite3_trace_go(int32(c.handle), int32(mask)))
if err := c.error(rc); err != nil {
return err
}
@@ -235,45 +244,49 @@ func (c *Conn) Trace(mask TraceEvent, cb func(evt TraceEvent, arg1 any, arg2 any
return nil
}
func traceCallback(ctx context.Context, mod api.Module, evt TraceEvent, pDB, pArg1, pArg2 ptr_t) (rc res_t) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.trace != nil {
func (e *env) Xgo_trace(evt, pDB, pArg1, pArg2 int32) int32 {
if c, ok := e.DB.(*Conn); ok && c.handle == ptr_t(pDB) && c.trace != nil {
var arg1, arg2 any
if evt == TRACE_CLOSE {
if TraceEvent(evt) == TRACE_CLOSE {
arg1 = c
} else {
for _, s := range c.stmts {
if pArg1 == s.handle {
if ptr_t(pArg1) == s.handle {
arg1 = s
switch evt {
switch TraceEvent(evt) {
case TRACE_STMT:
arg2 = s.SQL()
case TRACE_PROFILE:
arg2 = util.Read64[int64](mod, pArg2)
arg2 = int64(e.Read64(ptr_t(pArg2)))
}
break
}
}
}
if arg1 != nil {
_, rc = errorCode(c.trace(evt, arg1, arg2), ERROR)
_ = c.trace(TraceEvent(evt), arg1, arg2)
}
}
return rc
return _OK
}
// WALCheckpoint checkpoints a WAL database.
//
// https://sqlite.org/c3ref/wal_checkpoint_v2.html
func (c *Conn) WALCheckpoint(schema string, mode CheckpointMode) (nLog, nCkpt int, err error) {
defer c.arena.mark()()
nLogPtr := c.arena.new(ptrlen)
nCkptPtr := c.arena.new(ptrlen)
schemaPtr := c.arena.string(schema)
rc := res_t(c.call("sqlite3_wal_checkpoint_v2",
stk_t(c.handle), stk_t(schemaPtr), stk_t(mode),
stk_t(nLogPtr), stk_t(nCkptPtr)))
nLog = int(util.Read32[int32](c.mod, nLogPtr))
nCkpt = int(util.Read32[int32](c.mod, nCkptPtr))
if c.interrupt.Err() != nil {
return 0, 0, INTERRUPT
}
defer c.arena.Mark()()
nLogPtr := c.arena.New(ptrlen)
nCkptPtr := c.arena.New(ptrlen)
schemaPtr := c.arena.String(schema)
rc := res_t(c.wrp.Xsqlite3_wal_checkpoint_v2(
int32(c.handle), int32(schemaPtr), int32(mode),
int32(nLogPtr), int32(nCkptPtr)))
nLog = int(int32(c.wrp.Read32(nLogPtr)))
nCkpt = int(int32(c.wrp.Read32(nCkptPtr)))
return nLog, nCkpt, c.error(rc)
}
@@ -281,7 +294,7 @@ func (c *Conn) WALCheckpoint(schema string, mode CheckpointMode) (nLog, nCkpt in
//
// https://sqlite.org/c3ref/wal_autocheckpoint.html
func (c *Conn) WALAutoCheckpoint(pages int) error {
rc := res_t(c.call("sqlite3_wal_autocheckpoint", stk_t(c.handle), stk_t(pages)))
rc := res_t(c.wrp.Xsqlite3_wal_autocheckpoint(int32(c.handle), int32(pages)))
return c.error(rc)
}
@@ -294,17 +307,18 @@ func (c *Conn) WALHook(cb func(db *Conn, schema string, pages int) error) {
if cb != nil {
enable = 1
}
c.call("sqlite3_wal_hook_go", stk_t(c.handle), stk_t(enable))
c.wrp.Xsqlite3_wal_hook_go(int32(c.handle), enable)
c.wal = cb
}
func walCallback(ctx context.Context, mod api.Module, _, pDB, zSchema ptr_t, pages int32) (rc res_t) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.wal != nil {
schema := util.ReadString(mod, zSchema, _MAX_NAME)
func (e *env) Xgo_wal_hook(_, pDB, zSchema, pages int32) int32 {
if c, ok := e.DB.(*Conn); ok && c.handle == ptr_t(pDB) && c.wal != nil {
schema := e.ReadString(ptr_t(zSchema), _MAX_NAME)
err := c.wal(c, schema, int(pages))
_, rc = errorCode(err, ERROR)
_, rc := errorCode(err, ERROR)
return int32(rc)
}
return rc
return _OK
}
// AutoVacuumPages registers a autovacuum compaction amount callback.
@@ -313,30 +327,30 @@ func walCallback(ctx context.Context, mod api.Module, _, pDB, zSchema ptr_t, pag
func (c *Conn) AutoVacuumPages(cb func(schema string, dbPages, freePages, bytesPerPage uint) uint) error {
var funcPtr ptr_t
if cb != nil {
funcPtr = util.AddHandle(c.ctx, cb)
funcPtr = c.wrp.AddHandle(cb)
}
rc := res_t(c.call("sqlite3_autovacuum_pages_go", stk_t(c.handle), stk_t(funcPtr)))
rc := res_t(c.wrp.Xsqlite3_autovacuum_pages_go(int32(c.handle), int32(funcPtr)))
return c.error(rc)
}
func autoVacuumCallback(ctx context.Context, mod api.Module, pApp, zSchema ptr_t, nDbPage, nFreePage, nBytePerPage uint32) uint32 {
fn := util.GetHandle(ctx, pApp).(func(schema string, dbPages, freePages, bytesPerPage uint) uint)
schema := util.ReadString(mod, zSchema, _MAX_NAME)
return uint32(fn(schema, uint(nDbPage), uint(nFreePage), uint(nBytePerPage)))
func (e *env) Xgo_autovacuum_pages(pApp, zSchema, nDbPage, nFreePage, nBytePerPage int32) int32 {
fn := e.GetHandle(ptr_t(pApp)).(func(schema string, dbPages, freePages, bytesPerPage uint) uint)
schema := e.ReadString(ptr_t(zSchema), _MAX_NAME)
return int32(fn(schema, uint(uint32(nDbPage)), uint(uint32(nFreePage)), uint(uint32(nBytePerPage))))
}
// SoftHeapLimit imposes a soft limit on heap size.
//
// https://sqlite.org/c3ref/hard_heap_limit64.html
func (c *Conn) SoftHeapLimit(n int64) int64 {
return int64(c.call("sqlite3_soft_heap_limit64", stk_t(n)))
return c.wrp.Xsqlite3_soft_heap_limit64(n)
}
// HardHeapLimit imposes a hard limit on heap size.
//
// https://sqlite.org/c3ref/hard_heap_limit64.html
func (c *Conn) HardHeapLimit(n int64) int64 {
return int64(c.call("sqlite3_hard_heap_limit64", stk_t(n)))
return c.wrp.Xsqlite3_hard_heap_limit64(n)
}
// EnableChecksums enables checksums on a database.
@@ -364,7 +378,7 @@ func (c *Conn) EnableChecksums(schema string) error {
}
if r != 8 {
// Invalid value.
return util.ErrorString("sqlite3: reserve bytes must be 8, is: " + strconv.Itoa(r.(int)))
return errutil.ErrorString("sqlite3: reserve bytes must be 8, is: " + strconv.Itoa(r.(int)))
}
// VACUUM the database.
@@ -378,6 +392,6 @@ func (c *Conn) EnableChecksums(schema string) error {
}
// Checkpoint the WAL.
_, _, err = c.WALCheckpoint(schema, CHECKPOINT_RESTART)
_, _, err = c.WALCheckpoint(schema, CHECKPOINT_FULL)
return err
}
+173 -145
View File
@@ -3,6 +3,7 @@ package sqlite3
import (
"context"
"fmt"
"iter"
"math"
"math/rand"
"net/url"
@@ -10,10 +11,9 @@ import (
"strings"
"time"
"github.com/tetratelabs/wazero/api"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/vfs"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/internal/sqlite3_wrap"
"github.com/hanzoai/sqlite3/vfs"
)
// Conn is a database connection handle.
@@ -21,10 +21,9 @@ import (
//
// https://sqlite.org/c3ref/sqlite3.html
type Conn struct {
*sqlite
wrp *sqlite3_wrap.Wrapper
interrupt context.Context
pending *Stmt
stmts []*Stmt
busy func(context.Context, int) bool
log func(xErrorCode, string)
@@ -35,11 +34,13 @@ type Conn struct {
update func(AuthorizerActionCode, string, string, int64)
commit func() bool
rollback func()
preupdate func(PreUpdateData)
busy1st time.Time
busylst time.Time
arena arena
arena sqlite3_wrap.Arena
handle ptr_t
gosched uint8
}
// Open calls [OpenFlags] with [OPEN_READWRITE], [OPEN_CREATE] and [OPEN_URI].
@@ -48,7 +49,7 @@ func Open(filename string) (*Conn, error) {
}
// OpenContext is like [Open] but includes a context,
// which is used to interrupt the process of opening the connectiton.
// which is used to interrupt the process of opening the connection.
func OpenContext(ctx context.Context, filename string) (*Conn, error) {
return newConn(ctx, filename, OPEN_READWRITE|OPEN_CREATE|OPEN_URI)
}
@@ -68,8 +69,6 @@ func OpenFlags(filename string, flags OpenFlag) (*Conn, error) {
return newConn(context.Background(), filename, flags)
}
type connKey = util.ConnKey
func newConn(ctx context.Context, filename string, flags OpenFlag) (ret *Conn, _ error) {
err := ctx.Err()
if err != nil {
@@ -77,21 +76,23 @@ func newConn(ctx context.Context, filename string, flags OpenFlag) (ret *Conn, _
}
c := &Conn{interrupt: ctx}
c.sqlite, err = instantiateSQLite()
c.wrp, err = createWrapper(ctx)
if err != nil {
return nil, err
}
defer func() {
if ret == nil {
c.Close()
c.sqlite.close()
} else {
c.interrupt = context.Background()
}
}()
c.ctx = context.WithValue(c.ctx, connKey{}, c)
c.arena = c.newArena()
c.wrp.DB = c
if logger := defaultLogger.Load(); logger != nil {
c.ConfigLog(*logger)
}
c.arena = c.wrp.NewArena()
c.handle, err = c.openDB(filename, flags)
if err == nil {
err = initExtensions(c)
@@ -103,24 +104,24 @@ func newConn(ctx context.Context, filename string, flags OpenFlag) (ret *Conn, _
}
func (c *Conn) openDB(filename string, flags OpenFlag) (ptr_t, error) {
defer c.arena.mark()()
connPtr := c.arena.new(ptrlen)
namePtr := c.arena.string(filename)
defer c.arena.Mark()()
connPtr := c.arena.New(ptrlen)
namePtr := c.arena.String(filename)
flags |= OPEN_EXRESCODE
rc := res_t(c.call("sqlite3_open_v2", stk_t(namePtr), stk_t(connPtr), stk_t(flags), 0))
rc := res_t(c.wrp.Xsqlite3_open_v2(int32(namePtr), int32(connPtr), int32(flags), 0))
handle := util.Read32[ptr_t](c.mod, connPtr)
if err := c.sqlite.error(rc, handle); err != nil {
handle := ptr_t(c.wrp.Read32(connPtr))
if err := c.errorFor(handle, rc); err != nil {
c.closeDB(handle)
return 0, err
}
c.call("sqlite3_progress_handler_go", stk_t(handle), 100)
c.wrp.Xsqlite3_progress_handler_go(int32(handle), 1000)
if flags|OPEN_URI != 0 && strings.HasPrefix(filename, "file:") {
var pragmas strings.Builder
if _, after, ok := strings.Cut(filename, "?"); ok {
query, _ := url.ParseQuery(after)
if u, err := url.Parse(filename); err == nil {
query := u.Query()
for _, p := range query["_pragma"] {
pragmas.WriteString(`PRAGMA `)
pragmas.WriteString(p)
@@ -128,10 +129,9 @@ func (c *Conn) openDB(filename string, flags OpenFlag) (ptr_t, error) {
}
}
if pragmas.Len() != 0 {
c.checkInterrupt(handle)
pragmaPtr := c.arena.string(pragmas.String())
rc := res_t(c.call("sqlite3_exec", stk_t(handle), stk_t(pragmaPtr), 0, 0, 0))
if err := c.sqlite.error(rc, handle, pragmas.String()); err != nil {
pragmaPtr := c.arena.String(pragmas.String())
rc := res_t(c.wrp.Xsqlite3_exec(int32(handle), int32(pragmaPtr), 0, 0, 0))
if err := c.errorFor(handle, rc, pragmas.String()); err != nil {
err = fmt.Errorf("sqlite3: invalid _pragma: %w", err)
c.closeDB(handle)
return 0, err
@@ -142,8 +142,8 @@ func (c *Conn) openDB(filename string, flags OpenFlag) (ptr_t, error) {
}
func (c *Conn) closeDB(handle ptr_t) {
rc := res_t(c.call("sqlite3_close_v2", stk_t(handle)))
if err := c.sqlite.error(rc, handle); err != nil {
rc := res_t(c.wrp.Xsqlite3_close_v2(int32(handle)))
if err := c.errorFor(handle, rc); err != nil {
panic(err)
}
}
@@ -162,16 +162,13 @@ func (c *Conn) Close() error {
return nil
}
c.pending.Close()
c.pending = nil
rc := res_t(c.call("sqlite3_close", stk_t(c.handle)))
rc := res_t(c.wrp.Xsqlite3_close(int32(c.handle)))
if err := c.error(rc); err != nil {
return err
}
c.handle = 0
return c.close()
return c.wrp.Close()
}
// Exec is a convenience function that allows an application to run
@@ -179,11 +176,16 @@ func (c *Conn) Close() error {
//
// https://sqlite.org/c3ref/exec.html
func (c *Conn) Exec(sql string) error {
defer c.arena.mark()()
sqlPtr := c.arena.string(sql)
if c.interrupt.Err() != nil {
return INTERRUPT
}
return c.exec(sql)
}
c.checkInterrupt(c.handle)
rc := res_t(c.call("sqlite3_exec", stk_t(c.handle), stk_t(sqlPtr), 0, 0, 0))
func (c *Conn) exec(sql string) error {
defer c.arena.Mark()()
textPtr := c.arena.String(sql)
rc := res_t(c.wrp.Xsqlite3_exec(int32(c.handle), int32(textPtr), 0, 0, 0))
return c.error(rc, sql)
}
@@ -202,20 +204,22 @@ func (c *Conn) PrepareFlags(sql string, flags PrepareFlag) (stmt *Stmt, tail str
if len(sql) > _MAX_SQL_LENGTH {
return nil, "", TOOBIG
}
if c.interrupt.Err() != nil {
return nil, "", INTERRUPT
}
defer c.arena.mark()()
stmtPtr := c.arena.new(ptrlen)
tailPtr := c.arena.new(ptrlen)
sqlPtr := c.arena.string(sql)
defer c.arena.Mark()()
stmtPtr := c.arena.New(ptrlen)
tailPtr := c.arena.New(ptrlen)
textPtr := c.arena.String(sql)
c.checkInterrupt(c.handle)
rc := res_t(c.call("sqlite3_prepare_v3", stk_t(c.handle),
stk_t(sqlPtr), stk_t(len(sql)+1), stk_t(flags),
stk_t(stmtPtr), stk_t(tailPtr)))
rc := res_t(c.wrp.Xsqlite3_prepare_v3(int32(c.handle),
int32(textPtr), int32(len(sql)+1), int32(flags),
int32(stmtPtr), int32(tailPtr)))
stmt = &Stmt{c: c}
stmt.handle = util.Read32[ptr_t](c.mod, stmtPtr)
if sql := sql[util.Read32[ptr_t](c.mod, tailPtr)-sqlPtr:]; sql != "" {
stmt = &Stmt{c: c, sql: sql}
stmt.handle = ptr_t(c.wrp.Read32(stmtPtr))
if sql := sql[ptr_t(c.wrp.Read32(tailPtr))-textPtr:]; sql != "" {
tail = sql
}
@@ -233,11 +237,11 @@ func (c *Conn) PrepareFlags(sql string, flags PrepareFlag) (stmt *Stmt, tail str
//
// https://sqlite.org/c3ref/db_name.html
func (c *Conn) DBName(n int) string {
ptr := ptr_t(c.call("sqlite3_db_name", stk_t(c.handle), stk_t(n)))
ptr := ptr_t(c.wrp.Xsqlite3_db_name(int32(c.handle), int32(n)))
if ptr == 0 {
return ""
}
return util.ReadString(c.mod, ptr, _MAX_NAME)
return c.wrp.ReadString(ptr, _MAX_NAME)
}
// Filename returns the filename for a database.
@@ -246,23 +250,23 @@ func (c *Conn) DBName(n int) string {
func (c *Conn) Filename(schema string) *vfs.Filename {
var ptr ptr_t
if schema != "" {
defer c.arena.mark()()
ptr = c.arena.string(schema)
defer c.arena.Mark()()
ptr = c.arena.String(schema)
}
ptr = ptr_t(c.call("sqlite3_db_filename", stk_t(c.handle), stk_t(ptr)))
return vfs.GetFilename(c.ctx, c.mod, ptr, vfs.OPEN_MAIN_DB)
ptr = ptr_t(c.wrp.Xsqlite3_db_filename(int32(c.handle), int32(ptr)))
return vfs.GetFilename(c.wrp, ptr, vfs.OPEN_MAIN_DB)
}
// ReadOnly determines if a database is read-only.
//
// https://sqlite.org/c3ref/db_readonly.html
func (c *Conn) ReadOnly(schema string) (ro bool, ok bool) {
func (c *Conn) ReadOnly(schema string) (ro, ok bool) {
var ptr ptr_t
if schema != "" {
defer c.arena.mark()()
ptr = c.arena.string(schema)
defer c.arena.Mark()()
ptr = c.arena.String(schema)
}
b := int32(c.call("sqlite3_db_readonly", stk_t(c.handle), stk_t(ptr)))
b := c.wrp.Xsqlite3_db_readonly(int32(c.handle), int32(ptr))
return b > 0, b < 0
}
@@ -270,7 +274,7 @@ func (c *Conn) ReadOnly(schema string) (ro bool, ok bool) {
//
// https://sqlite.org/c3ref/get_autocommit.html
func (c *Conn) GetAutocommit() bool {
b := int32(c.call("sqlite3_get_autocommit", stk_t(c.handle)))
b := c.wrp.Xsqlite3_get_autocommit(int32(c.handle))
return b != 0
}
@@ -279,7 +283,7 @@ func (c *Conn) GetAutocommit() bool {
//
// https://sqlite.org/c3ref/last_insert_rowid.html
func (c *Conn) LastInsertRowID() int64 {
return int64(c.call("sqlite3_last_insert_rowid", stk_t(c.handle)))
return c.wrp.Xsqlite3_last_insert_rowid(int32(c.handle))
}
// SetLastInsertRowID allows the application to set the value returned by
@@ -287,7 +291,7 @@ func (c *Conn) LastInsertRowID() int64 {
//
// https://sqlite.org/c3ref/set_last_insert_rowid.html
func (c *Conn) SetLastInsertRowID(id int64) {
c.call("sqlite3_set_last_insert_rowid", stk_t(c.handle), stk_t(id))
c.wrp.Xsqlite3_set_last_insert_rowid(int32(c.handle), id)
}
// Changes returns the number of rows modified, inserted or deleted
@@ -296,7 +300,7 @@ func (c *Conn) SetLastInsertRowID(id int64) {
//
// https://sqlite.org/c3ref/changes.html
func (c *Conn) Changes() int64 {
return int64(c.call("sqlite3_changes64", stk_t(c.handle)))
return c.wrp.Xsqlite3_changes64(int32(c.handle))
}
// TotalChanges returns the number of rows modified, inserted or deleted
@@ -305,14 +309,14 @@ func (c *Conn) Changes() int64 {
//
// https://sqlite.org/c3ref/total_changes.html
func (c *Conn) TotalChanges() int64 {
return int64(c.call("sqlite3_total_changes64", stk_t(c.handle)))
return c.wrp.Xsqlite3_total_changes64(int32(c.handle))
}
// ReleaseMemory frees memory used by a database connection.
//
// https://sqlite.org/c3ref/db_release_memory.html
func (c *Conn) ReleaseMemory() error {
rc := res_t(c.call("sqlite3_db_release_memory", stk_t(c.handle)))
rc := res_t(c.wrp.Xsqlite3_db_release_memory(int32(c.handle)))
return c.error(rc)
}
@@ -336,43 +340,17 @@ func (c *Conn) GetInterrupt() context.Context {
//
// https://sqlite.org/c3ref/interrupt.html
func (c *Conn) SetInterrupt(ctx context.Context) (old context.Context) {
if ctx == nil {
panic("nil Context")
}
old = c.interrupt
c.interrupt = ctx
if ctx == old || ctx.Done() == old.Done() {
return old
}
// A busy SQL statement prevents SQLite from ignoring an interrupt
// that comes before any other statements are started.
if c.pending == nil {
defer c.arena.mark()()
stmtPtr := c.arena.new(ptrlen)
loopPtr := c.arena.string(`WITH RECURSIVE c(x) AS (VALUES(0) UNION ALL SELECT x FROM c) SELECT x FROM c`)
c.call("sqlite3_prepare_v3", stk_t(c.handle), stk_t(loopPtr), math.MaxUint64,
stk_t(PREPARE_PERSISTENT), stk_t(stmtPtr), 0)
c.pending = &Stmt{c: c}
c.pending.handle = util.Read32[ptr_t](c.mod, stmtPtr)
}
if old.Done() != nil && ctx.Err() == nil {
c.pending.Reset()
}
if ctx.Done() != nil {
c.pending.Step()
}
return old
}
func (c *Conn) checkInterrupt(handle ptr_t) {
if c.interrupt.Err() != nil {
c.call("sqlite3_interrupt", stk_t(handle))
}
}
func progressCallback(ctx context.Context, mod api.Module, _ ptr_t) (interrupt int32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok {
if c.interrupt.Done() != nil {
func (e *env) Xgo_progress_handler(_ int32) (interrupt int32) {
if c, ok := e.DB.(*Conn); ok {
if c.gosched++; c.gosched%16 == 0 {
runtime.Gosched()
}
if c.interrupt.Err() != nil {
@@ -387,13 +365,13 @@ func progressCallback(ctx context.Context, mod api.Module, _ ptr_t) (interrupt i
// https://sqlite.org/c3ref/busy_timeout.html
func (c *Conn) BusyTimeout(timeout time.Duration) error {
ms := min((timeout+time.Millisecond-1)/time.Millisecond, math.MaxInt32)
rc := res_t(c.call("sqlite3_busy_timeout", stk_t(c.handle), stk_t(ms)))
rc := res_t(c.wrp.Xsqlite3_busy_timeout(int32(c.handle), int32(ms)))
return c.error(rc)
}
func timeoutCallback(ctx context.Context, mod api.Module, count, tmout int32) (retry int32) {
func (e *env) Xgo_busy_timeout(count, tmout int32) (retry int32) {
// https://fractaledmind.github.io/2024/04/15/sqlite-on-rails-the-how-and-why-of-optimal-performance/
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.interrupt.Err() == nil {
if c, ok := e.DB.(*Conn); ok && c.interrupt.Err() == nil {
switch {
case count == 0:
c.busy1st = time.Now()
@@ -418,7 +396,7 @@ func (c *Conn) BusyHandler(cb func(ctx context.Context, count int) (retry bool))
if cb != nil {
enable = 1
}
rc := res_t(c.call("sqlite3_busy_handler_go", stk_t(c.handle), stk_t(enable)))
rc := res_t(c.wrp.Xsqlite3_busy_handler_go(int32(c.handle), enable))
if err := c.error(rc); err != nil {
return err
}
@@ -426,13 +404,10 @@ func (c *Conn) BusyHandler(cb func(ctx context.Context, count int) (retry bool))
return nil
}
func busyCallback(ctx context.Context, mod api.Module, pDB ptr_t, count int32) (retry int32) {
if c, ok := ctx.Value(connKey{}).(*Conn); ok && c.handle == pDB && c.busy != nil {
interrupt := c.interrupt
if interrupt == nil {
interrupt = context.Background()
}
if interrupt.Err() == nil && c.busy(interrupt, int(count)) {
func (e *env) Xgo_busy_handler(pDB, count int32) (retry int32) {
if c, ok := e.DB.(*Conn); ok && c.handle == ptr_t(pDB) && c.busy != nil {
if interrupt := c.interrupt; interrupt.Err() == nil &&
c.busy(interrupt, int(count)) {
retry = 1
}
}
@@ -442,21 +417,21 @@ func busyCallback(ctx context.Context, mod api.Module, pDB ptr_t, count int32) (
// Status retrieves runtime status information about a database connection.
//
// https://sqlite.org/c3ref/db_status.html
func (c *Conn) Status(op DBStatus, reset bool) (current, highwater int, err error) {
defer c.arena.mark()()
hiPtr := c.arena.new(intlen)
curPtr := c.arena.new(intlen)
func (c *Conn) Status(op DBStatus, reset bool) (current, highwater int64, err error) {
defer c.arena.Mark()()
hiPtr := c.arena.New(8)
curPtr := c.arena.New(8)
var i int32
if reset {
i = 1
}
rc := res_t(c.call("sqlite3_db_status", stk_t(c.handle),
stk_t(op), stk_t(curPtr), stk_t(hiPtr), stk_t(i)))
rc := res_t(c.wrp.Xsqlite3_db_status64(int32(c.handle),
int32(op), int32(curPtr), int32(hiPtr), i))
if err = c.error(rc); err == nil {
current = int(util.Read32[int32](c.mod, curPtr))
highwater = int(util.Read32[int32](c.mod, hiPtr))
current = int64(c.wrp.Read64(curPtr))
highwater = int64(c.wrp.Read64(hiPtr))
}
return
}
@@ -465,48 +440,101 @@ func (c *Conn) Status(op DBStatus, reset bool) (current, highwater int, err erro
//
// https://sqlite.org/c3ref/table_column_metadata.html
func (c *Conn) TableColumnMetadata(schema, table, column string) (declType, collSeq string, notNull, primaryKey, autoInc bool, err error) {
defer c.arena.mark()()
var schemaPtr, columnPtr ptr_t
declTypePtr := c.arena.new(ptrlen)
collSeqPtr := c.arena.new(ptrlen)
notNullPtr := c.arena.new(ptrlen)
autoIncPtr := c.arena.new(ptrlen)
primaryKeyPtr := c.arena.new(ptrlen)
if schema != "" {
schemaPtr = c.arena.string(schema)
}
tablePtr := c.arena.string(table)
defer c.arena.Mark()()
var (
declTypePtr ptr_t
collSeqPtr ptr_t
notNullPtr ptr_t
primaryKeyPtr ptr_t
autoIncPtr ptr_t
columnPtr ptr_t
schemaPtr ptr_t
)
if column != "" {
columnPtr = c.arena.string(column)
declTypePtr = c.arena.New(ptrlen)
collSeqPtr = c.arena.New(ptrlen)
notNullPtr = c.arena.New(ptrlen)
primaryKeyPtr = c.arena.New(ptrlen)
autoIncPtr = c.arena.New(ptrlen)
columnPtr = c.arena.String(column)
}
if schema != "" {
schemaPtr = c.arena.String(schema)
}
tablePtr := c.arena.String(table)
rc := res_t(c.call("sqlite3_table_column_metadata", stk_t(c.handle),
stk_t(schemaPtr), stk_t(tablePtr), stk_t(columnPtr),
stk_t(declTypePtr), stk_t(collSeqPtr),
stk_t(notNullPtr), stk_t(primaryKeyPtr), stk_t(autoIncPtr)))
rc := res_t(c.wrp.Xsqlite3_table_column_metadata(int32(c.handle),
int32(schemaPtr), int32(tablePtr), int32(columnPtr),
int32(declTypePtr), int32(collSeqPtr),
int32(notNullPtr), int32(primaryKeyPtr), int32(autoIncPtr)))
if err = c.error(rc); err == nil && column != "" {
if ptr := util.Read32[ptr_t](c.mod, declTypePtr); ptr != 0 {
declType = util.ReadString(c.mod, ptr, _MAX_NAME)
if ptr := ptr_t(c.wrp.Read32(declTypePtr)); ptr != 0 {
declType = c.wrp.ReadString(ptr, _MAX_NAME)
}
if ptr := util.Read32[ptr_t](c.mod, collSeqPtr); ptr != 0 {
collSeq = util.ReadString(c.mod, ptr, _MAX_NAME)
if ptr := ptr_t(c.wrp.Read32(collSeqPtr)); ptr != 0 {
collSeq = c.wrp.ReadString(ptr, _MAX_NAME)
}
notNull = util.ReadBool(c.mod, notNullPtr)
autoInc = util.ReadBool(c.mod, autoIncPtr)
primaryKey = util.ReadBool(c.mod, primaryKeyPtr)
notNull = c.wrp.ReadBool(notNullPtr)
autoInc = c.wrp.ReadBool(autoIncPtr)
primaryKey = c.wrp.ReadBool(primaryKeyPtr)
}
return
}
func (c *Conn) error(rc res_t, sql ...string) error {
return c.sqlite.error(rc, c.handle, sql...)
return c.errorFor(c.handle, rc, sql...)
}
func (c *Conn) stmtsIter(yield func(*Stmt) bool) {
for _, s := range c.stmts {
if !yield(s) {
break
func (c *Conn) errorFor(handle ptr_t, rc res_t, sql ...string) error {
if rc == _OK {
return nil
}
if ErrorCode(rc) == NOMEM || xErrorCode(rc) == IOERR_NOMEM {
panic(errutil.OOMErr)
}
var msg, query string
if handle != 0 {
if ptr := ptr_t(c.wrp.Xsqlite3_errmsg(int32(handle))); ptr != 0 {
msg = c.wrp.ReadString(ptr, _MAX_LENGTH)
msg = strings.TrimPrefix(msg, "sqlite3: ")
msg = strings.TrimPrefix(msg, sqlite3_wrap.ErrorCodeString(rc)[len("sqlite3: "):])
msg = strings.TrimPrefix(msg, ": ")
if msg == "" || msg == "not an error" {
msg = ""
}
}
if len(sql) != 0 {
if i := c.wrp.Xsqlite3_error_offset(int32(handle)); i != -1 {
query = sql[0][i:]
}
}
}
var sys error
switch ErrorCode(rc) {
case CANTOPEN, IOERR:
sys = c.wrp.SysError
}
if sys != nil || msg != "" || query != "" {
return &Error{code: rc, sys: sys, msg: msg, sql: query}
}
return xErrorCode(rc)
}
// Stmts returns an iterator for the prepared statements
// associated with the database connection.
//
// https://sqlite.org/c3ref/next_stmt.html
func (c *Conn) Stmts() iter.Seq[*Stmt] {
return func(yield func(*Stmt) bool) {
for _, s := range c.stmts {
if !yield(s) {
break
}
}
}
}
-11
View File
@@ -1,11 +0,0 @@
//go:build go1.23
package sqlite3
import "iter"
// Stmts returns an iterator for the prepared statements
// associated with the database connection.
//
// https://sqlite.org/c3ref/next_stmt.html
func (c *Conn) Stmts() iter.Seq[*Stmt] { return c.stmtsIter }
-9
View File
@@ -1,9 +0,0 @@
//go:build !go1.23
package sqlite3
// Stmts returns an iterator for the prepared statements
// associated with the database connection.
//
// https://sqlite.org/c3ref/next_stmt.html
func (c *Conn) Stmts() func(func(*Stmt) bool) { return c.stmtsIter }
+34 -24
View File
@@ -3,7 +3,7 @@ package sqlite3
import (
"strconv"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3/internal/sqlite3_wrap"
)
const (
@@ -11,19 +11,17 @@ const (
_ROW = 100 /* sqlite3_step() has another row ready */
_DONE = 101 /* sqlite3_step() has finished executing */
_MAX_NAME = 1e6 // Self-imposed limit for most NUL terminated strings.
_MAX_LENGTH = 1e9
_MAX_SQL_LENGTH = 1e9
_MAX_FUNCTION_ARG = 100
_MAX_NAME = 1e6 // Self-imposed limit for most NUL terminated strings.
_MAX_LENGTH = 1e9
_MAX_SQL_LENGTH = 1e9
ptrlen = util.PtrLen
intlen = util.IntLen
ptrlen = sqlite3_wrap.PtrLen
intlen = sqlite3_wrap.IntLen
)
type (
stk_t = util.Stk_t
ptr_t = util.Ptr_t
res_t = util.Res_t
ptr_t = sqlite3_wrap.Ptr_t
res_t = sqlite3_wrap.Res_t
)
// ErrorCode is a result code that [Error.Code] might return.
@@ -74,6 +72,9 @@ const (
ERROR_MISSING_COLLSEQ ExtendedErrorCode = xErrorCode(ERROR) | (1 << 8)
ERROR_RETRY ExtendedErrorCode = xErrorCode(ERROR) | (2 << 8)
ERROR_SNAPSHOT ExtendedErrorCode = xErrorCode(ERROR) | (3 << 8)
ERROR_RESERVESIZE ExtendedErrorCode = xErrorCode(ERROR) | (4 << 8)
ERROR_KEY ExtendedErrorCode = xErrorCode(ERROR) | (5 << 8)
ERROR_UNABLE ExtendedErrorCode = xErrorCode(ERROR) | (6 << 8)
IOERR_READ ExtendedErrorCode = xErrorCode(IOERR) | (1 << 8)
IOERR_SHORT_READ ExtendedErrorCode = xErrorCode(IOERR) | (2 << 8)
IOERR_WRITE ExtendedErrorCode = xErrorCode(IOERR) | (3 << 8)
@@ -108,6 +109,8 @@ const (
IOERR_DATA ExtendedErrorCode = xErrorCode(IOERR) | (32 << 8)
IOERR_CORRUPTFS ExtendedErrorCode = xErrorCode(IOERR) | (33 << 8)
IOERR_IN_PAGE ExtendedErrorCode = xErrorCode(IOERR) | (34 << 8)
IOERR_BADKEY ExtendedErrorCode = xErrorCode(IOERR) | (35 << 8)
IOERR_CODEC ExtendedErrorCode = xErrorCode(IOERR) | (36 << 8)
LOCKED_SHAREDCACHE ExtendedErrorCode = xErrorCode(LOCKED) | (1 << 8)
LOCKED_VTAB ExtendedErrorCode = xErrorCode(LOCKED) | (2 << 8)
BUSY_RECOVERY ExtendedErrorCode = xErrorCode(BUSY) | (1 << 8)
@@ -169,7 +172,7 @@ const (
// PrepareFlag is a flag that can be passed to [Conn.PrepareFlags].
//
// https://sqlite.org/c3ref/c_prepare_normalize.html
// https://sqlite.org/c3ref/c_prepare_dont_log.html
type PrepareFlag uint32
const (
@@ -177,6 +180,7 @@ const (
PREPARE_NORMALIZE PrepareFlag = 0x02
PREPARE_NO_VTAB PrepareFlag = 0x04
PREPARE_DONT_LOG PrepareFlag = 0x10
PREPARE_FROM_DDL PrepareFlag = 0x20
)
// FunctionFlag is a flag that can be passed to
@@ -186,12 +190,12 @@ const (
type FunctionFlag uint32
const (
DETERMINISTIC FunctionFlag = 0x000000800
DIRECTONLY FunctionFlag = 0x000080000
INNOCUOUS FunctionFlag = 0x000200000
SELFORDER1 FunctionFlag = 0x002000000
// SUBTYPE FunctionFlag = 0x000100000
// RESULT_SUBTYPE FunctionFlag = 0x001000000
DETERMINISTIC FunctionFlag = 0x000000800
DIRECTONLY FunctionFlag = 0x000080000
SUBTYPE FunctionFlag = 0x000100000
INNOCUOUS FunctionFlag = 0x000200000
RESULT_SUBTYPE FunctionFlag = 0x001000000
SELFORDER1 FunctionFlag = 0x002000000
)
// StmtStatus name counter values associated with the [Stmt.Status] method.
@@ -230,7 +234,8 @@ const (
DBSTATUS_DEFERRED_FKS DBStatus = 10
DBSTATUS_CACHE_USED_SHARED DBStatus = 11
DBSTATUS_CACHE_SPILL DBStatus = 12
// DBSTATUS_MAX DBStatus = 12
DBSTATUS_TEMPBUF_SPILL DBStatus = 13
// DBSTATUS_MAX DBStatus = 13
)
// DBConfig are the available database connection configuration options.
@@ -262,7 +267,8 @@ const (
DBCONFIG_ENABLE_ATTACH_CREATE DBConfig = 1020
DBCONFIG_ENABLE_ATTACH_WRITE DBConfig = 1021
DBCONFIG_ENABLE_COMMENTS DBConfig = 1022
// DBCONFIG_MAX DBConfig = 1022
DBCONFIG_FP_DIGITS DBConfig = 1023
// DBCONFIG_MAX DBConfig = 1023
)
// FcntlOpcode are the available opcodes for [Conn.FileControl].
@@ -275,12 +281,14 @@ const (
FCNTL_CHUNK_SIZE FcntlOpcode = 6
FCNTL_FILE_POINTER FcntlOpcode = 7
FCNTL_PERSIST_WAL FcntlOpcode = 10
FCNTL_VFSNAME FcntlOpcode = 12
FCNTL_POWERSAFE_OVERWRITE FcntlOpcode = 13
FCNTL_VFS_POINTER FcntlOpcode = 27
FCNTL_JOURNAL_POINTER FcntlOpcode = 28
FCNTL_DATA_VERSION FcntlOpcode = 35
FCNTL_RESERVE_BYTES FcntlOpcode = 38
FCNTL_RESET_CACHE FcntlOpcode = 42
FCNTL_NULL_IO FcntlOpcode = 43
)
// LimitCategory are the available run-time limit categories.
@@ -301,6 +309,7 @@ const (
LIMIT_VARIABLE_NUMBER LimitCategory = 9
LIMIT_TRIGGER_DEPTH LimitCategory = 10
LIMIT_WORKER_THREADS LimitCategory = 11
LIMIT_PARSER_DEPTH LimitCategory = 12
)
// AuthorizerActionCode are the integer action codes
@@ -362,13 +371,14 @@ const (
// CheckpointMode are all the checkpoint mode values.
//
// https://sqlite.org/c3ref/c_checkpoint_full.html
type CheckpointMode uint32
type CheckpointMode int32
const (
CHECKPOINT_PASSIVE CheckpointMode = 0 /* Do as much as possible w/o blocking */
CHECKPOINT_FULL CheckpointMode = 1 /* Wait for writers, then checkpoint */
CHECKPOINT_RESTART CheckpointMode = 2 /* Like FULL but wait for readers */
CHECKPOINT_TRUNCATE CheckpointMode = 3 /* Like RESTART but also truncate WAL */
CHECKPOINT_NOOP CheckpointMode = -1 /* Do no work at all */
CHECKPOINT_PASSIVE CheckpointMode = 0 /* Do as much as possible w/o blocking */
CHECKPOINT_FULL CheckpointMode = 1 /* Wait for writers, then checkpoint */
CHECKPOINT_RESTART CheckpointMode = 2 /* Like FULL but wait for readers */
CHECKPOINT_TRUNCATE CheckpointMode = 3 /* Like RESTART but also truncate WAL */
)
// TxnState are the allowed return values from [Conn.TxnState].
+55 -61
View File
@@ -1,12 +1,10 @@
package sqlite3
import (
"encoding/json"
"errors"
"math"
"time"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3/internal/errutil"
)
// Context is the context in which an SQL function executes.
@@ -31,16 +29,16 @@ func (ctx Context) Conn() *Conn {
//
// https://sqlite.org/c3ref/get_auxdata.html
func (ctx Context) SetAuxData(n int, data any) {
ptr := util.AddHandle(ctx.c.ctx, data)
ctx.c.call("sqlite3_set_auxdata_go", stk_t(ctx.handle), stk_t(n), stk_t(ptr))
ptr := ctx.c.wrp.AddHandle(data)
ctx.c.wrp.Xsqlite3_set_auxdata_go(int32(ctx.handle), int32(n), int32(ptr))
}
// GetAuxData returns metadata for argument n of the function.
//
// https://sqlite.org/c3ref/get_auxdata.html
func (ctx Context) GetAuxData(n int) any {
ptr := ptr_t(ctx.c.call("sqlite3_get_auxdata", stk_t(ctx.handle), stk_t(n)))
return util.GetHandle(ctx.c.ctx, ptr)
ptr := ptr_t(ctx.c.wrp.Xsqlite3_get_auxdata(int32(ctx.handle), int32(n)))
return ctx.c.wrp.GetHandle(ptr)
}
// ResultBool sets the result of the function to a bool.
@@ -67,61 +65,61 @@ func (ctx Context) ResultInt(value int) {
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultInt64(value int64) {
ctx.c.call("sqlite3_result_int64",
stk_t(ctx.handle), stk_t(value))
ctx.c.wrp.Xsqlite3_result_int64(
int32(ctx.handle), value)
}
// ResultFloat sets the result of the function to a float64.
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultFloat(value float64) {
ctx.c.call("sqlite3_result_double",
stk_t(ctx.handle), stk_t(math.Float64bits(value)))
ctx.c.wrp.Xsqlite3_result_double(
int32(ctx.handle), value)
}
// ResultText sets the result of the function to a string.
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultText(value string) {
ptr := ctx.c.newString(value)
ctx.c.call("sqlite3_result_text_go",
stk_t(ctx.handle), stk_t(ptr), stk_t(len(value)))
ptr := ctx.c.wrp.NewString(value)
ctx.c.wrp.Xsqlite3_result_text_go(
int32(ctx.handle), int32(ptr), int64(len(value)))
}
// ResultRawText sets the text result of the function to a []byte.
// Returning a nil slice is the same as calling [Context.ResultNull].
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultRawText(value []byte) {
ptr := ctx.c.newBytes(value)
ctx.c.call("sqlite3_result_text_go",
stk_t(ctx.handle), stk_t(ptr), stk_t(len(value)))
ctx.ResultText(string(value)) // does not escape
}
// ResultBlob sets the result of the function to a []byte.
// Returning a nil slice is the same as calling [Context.ResultNull].
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultBlob(value []byte) {
ptr := ctx.c.newBytes(value)
ctx.c.call("sqlite3_result_blob_go",
stk_t(ctx.handle), stk_t(ptr), stk_t(len(value)))
if len(value) == 0 {
ctx.ResultZeroBlob(0)
return
}
ptr := ctx.c.wrp.NewBytes(value)
ctx.c.wrp.Xsqlite3_result_blob_go(
int32(ctx.handle), int32(ptr), int64(len(value)))
}
// ResultZeroBlob sets the result of the function to a zero-filled, length n BLOB.
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultZeroBlob(n int64) {
ctx.c.call("sqlite3_result_zeroblob64",
stk_t(ctx.handle), stk_t(n))
ctx.c.wrp.Xsqlite3_result_zeroblob64(
int32(ctx.handle), n)
}
// ResultNull sets the result of the function to NULL.
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultNull() {
ctx.c.call("sqlite3_result_null",
stk_t(ctx.handle))
ctx.c.wrp.Xsqlite3_result_null(
int32(ctx.handle))
}
// ResultTime sets the result of the function to a [time.Time].
@@ -141,19 +139,19 @@ func (ctx Context) ResultTime(value time.Time, format TimeFormat) {
case float64:
ctx.ResultFloat(v)
default:
panic(util.AssertErr())
panic(errutil.AssertErr())
}
}
func (ctx Context) resultRFC3339Nano(value time.Time) {
const maxlen = int64(len(time.RFC3339Nano)) + 5
ptr := ctx.c.new(maxlen)
buf := util.View(ctx.c.mod, ptr, maxlen)
const maxlen = 48
ptr := ctx.c.wrp.New(maxlen)
buf := ctx.c.wrp.Bytes(ptr, maxlen)
buf = value.AppendFormat(buf[:0], time.RFC3339Nano)
_ = append(buf, 0)
ctx.c.call("sqlite3_result_text_go",
stk_t(ctx.handle), stk_t(ptr), stk_t(len(buf)))
ctx.c.wrp.Xsqlite3_result_text_go(
int32(ctx.handle), int32(ptr), int64(len(buf)))
}
// ResultPointer sets the result of the function to NULL, just like [Context.ResultNull],
@@ -162,21 +160,9 @@ func (ctx Context) resultRFC3339Nano(value time.Time) {
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultPointer(ptr any) {
valPtr := util.AddHandle(ctx.c.ctx, ptr)
ctx.c.call("sqlite3_result_pointer_go",
stk_t(ctx.handle), stk_t(valPtr))
}
// ResultJSON sets the result of the function to the JSON encoding of value.
//
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultJSON(value any) {
data, err := json.Marshal(value)
if err != nil {
ctx.ResultError(err)
return // notest
}
ctx.ResultRawText(data)
valPtr := ctx.c.wrp.AddHandle(ptr)
ctx.c.wrp.Xsqlite3_result_pointer_go(
int32(ctx.handle), int32(valPtr))
}
// ResultValue sets the result of the function to a copy of [Value].
@@ -187,8 +173,8 @@ func (ctx Context) ResultValue(value Value) {
ctx.ResultError(MISUSE)
return
}
ctx.c.call("sqlite3_result_value",
stk_t(ctx.handle), stk_t(value.handle))
ctx.c.wrp.Xsqlite3_result_value(
int32(ctx.handle), int32(value.handle))
}
// ResultError sets the result of the function an error.
@@ -196,33 +182,41 @@ func (ctx Context) ResultValue(value Value) {
// https://sqlite.org/c3ref/result_blob.html
func (ctx Context) ResultError(err error) {
if errors.Is(err, NOMEM) {
ctx.c.call("sqlite3_result_error_nomem", stk_t(ctx.handle))
ctx.c.wrp.Xsqlite3_result_error_nomem(int32(ctx.handle))
return
}
if errors.Is(err, TOOBIG) {
ctx.c.call("sqlite3_result_error_toobig", stk_t(ctx.handle))
ctx.c.wrp.Xsqlite3_result_error_toobig(int32(ctx.handle))
return
}
msg, code := errorCode(err, _OK)
msg, code := errorCode(err, ERROR)
if msg != "" {
defer ctx.c.arena.mark()()
ptr := ctx.c.arena.string(msg)
ctx.c.call("sqlite3_result_error",
stk_t(ctx.handle), stk_t(ptr), stk_t(len(msg)))
defer ctx.c.arena.Mark()()
ptr := ctx.c.arena.String(msg)
ctx.c.wrp.Xsqlite3_result_error(
int32(ctx.handle), int32(ptr), int32(len(msg)))
}
if code != _OK {
ctx.c.call("sqlite3_result_error_code",
stk_t(ctx.handle), stk_t(code))
if code != res_t(ERROR) {
ctx.c.wrp.Xsqlite3_result_error_code(
int32(ctx.handle), int32(code))
}
}
// ResultSubtype sets the subtype of the result of the function.
//
// https://sqlite.org/c3ref/result_subtype.html
func (ctx Context) ResultSubtype(t uint) {
ctx.c.wrp.Xsqlite3_result_subtype(
int32(ctx.handle), int32(t))
}
// VTabNoChange may return true if a column is being fetched as part
// of an update during which the column value will not change.
//
// https://sqlite.org/c3ref/vtab_nochange.html
func (ctx Context) VTabNoChange() bool {
b := int32(ctx.c.call("sqlite3_vtab_nochange", stk_t(ctx.handle)))
b := ctx.c.wrp.Xsqlite3_vtab_nochange(int32(ctx.handle))
return b != 0
}
+208 -146
View File
@@ -1,10 +1,8 @@
// Package driver provides a database/sql driver for SQLite.
//
// Importing package driver registers a [database/sql] driver named "sqlite3".
// You may also need to import package embed.
//
// import _ "github.com/ncruces/go-sqlite3/driver"
// import _ "github.com/ncruces/go-sqlite3/embed"
// import _ "github.com/hanzoai/sqlite3/driver"
//
// The data source name for "sqlite3" databases can be a filename or a "file:" [URI].
//
@@ -20,22 +18,45 @@
// - a [serializable] transaction is always "immediate";
// - a [read-only] transaction is always "deferred".
//
// # Datatypes In SQLite
//
// SQLite is dynamically typed.
// Columns can mostly hold any value regardless of their declared type.
// SQLite supports most [driver.Value] types out of the box,
// but bool and [time.Time] require special care.
//
// Booleans can be stored on any column type and scanned back to a *bool.
// However, if scanned to a *any, booleans may either become an
// int64, string or bool, depending on the declared type of the column.
// If you use BOOLEAN for your column type,
// 1 and 0 will always scan as true and false.
//
// # Working with time
//
// Time values can similarly be stored on any column type.
// The time encoding/decoding format can be specified using "_timefmt":
//
// sql.Open("sqlite3", "file:demo.db?_timefmt=sqlite")
//
// Possible values are: "auto" (the default), "sqlite", "rfc3339";
// Special values are: "auto" (the default), "sqlite", "rfc3339";
// - "auto" encodes as RFC 3339 and decodes any [format] supported by SQLite;
// - "sqlite" encodes as SQLite and decodes any [format] supported by SQLite;
// - "rfc3339" encodes and decodes RFC 3339 only.
//
// If you encode as RFC 3339 (the default),
// consider using the TIME [collating sequence] to produce a time-ordered sequence.
// You can also set "_timefmt" to an arbitrary [sqlite3.TimeFormat] or [time.Layout].
//
// To scan values in other formats, [sqlite3.TimeFormat.Scanner] may be helpful.
// To bind values in other formats, [sqlite3.TimeFormat.Encode] them before binding.
// If you encode as RFC 3339 (the default),
// consider using the TIME [collating sequence] to produce time-ordered sequences.
//
// If you encode as RFC 3339 (the default),
// time values will scan back to a *time.Time unless your column type is TEXT.
// Otherwise, if scanned to a *any, time values may either become an
// int64, float64 or string, depending on the time format and declared type of the column.
// If you use DATE, TIME, DATETIME, or TIMESTAMP for your column type,
// "_timefmt" will be used to decode values.
//
// To scan values in custom formats, [sqlite3.TimeFormat.Scanner] may be helpful.
// To bind values in custom formats, [sqlite3.TimeFormat.Encode] them before binding.
//
// When using a custom time struct, you'll have to implement
// [database/sql/driver.Valuer] and [database/sql.Scanner].
@@ -48,7 +69,7 @@
// The Scan method needs to take into account that the value it receives can be of differing types.
// It can already be a [time.Time], if the driver decoded the value according to "_timefmt" rules.
// Or it can be a: string, int64, float64, []byte, or nil,
// depending on the column type and what whoever wrote the value.
// depending on the column type and whoever wrote the value.
// [sqlite3.TimeFormat.Decode] may help.
//
// # Setting PRAGMAs
@@ -86,13 +107,14 @@ import (
"time"
"unsafe"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/internal/util"
)
// This variable can be replaced with -ldflags:
//
// go build -ldflags="-X github.com/ncruces/go-sqlite3/driver.driverName=sqlite"
// go build -ldflags="-X github.com/hanzoai/sqlite3/driver.driverName=sqlite"
var driverName = "sqlite3"
func init() {
@@ -152,15 +174,17 @@ func newConnector(name string, init, term func(*sqlite3.Conn) error) (*connector
var txlock, timefmt string
if strings.HasPrefix(name, "file:") {
if _, after, ok := strings.Cut(name, "?"); ok {
query, err := url.ParseQuery(after)
if err != nil {
return nil, err
}
txlock = query.Get("_txlock")
timefmt = query.Get("_timefmt")
c.pragmas = query.Has("_pragma")
u, err := url.Parse(name)
if err != nil {
return nil, err
}
query, err := url.ParseQuery(u.RawQuery)
if err != nil {
return nil, err
}
txlock = query.Get("_txlock")
timefmt = query.Get("_timefmt")
c.pragmas = query.Has("_pragma")
}
switch txlock {
@@ -218,8 +242,9 @@ func (n *connector) Connect(ctx context.Context) (ret driver.Conn, err error) {
}
}()
old := c.Conn.SetInterrupt(ctx)
defer c.Conn.SetInterrupt(old)
if old := c.Conn.SetInterrupt(ctx); old != ctx {
defer c.Conn.SetInterrupt(old)
}
if !n.pragmas {
err = c.Conn.BusyTimeout(time.Minute)
@@ -239,10 +264,8 @@ func (n *connector) Connect(ctx context.Context) (ret driver.Conn, err error) {
return nil, err
}
defer s.Close()
if s.Step() && s.ColumnBool(0) {
c.readOnly = '1'
} else {
c.readOnly = '0'
if s.Step() {
c.readOnly = s.ColumnBool(0)
}
err = s.Close()
if err != nil {
@@ -298,7 +321,7 @@ type conn struct {
txReset string
tmRead sqlite3.TimeFormat
tmWrite sqlite3.TimeFormat
readOnly byte
readOnly bool
}
var (
@@ -321,7 +344,7 @@ func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, e
var txLock string
switch opts.Isolation {
default:
return nil, util.IsolationErr
return nil, errutil.IsolationErr
case driver.IsolationLevel(sql.LevelLinearizable):
txLock = "exclusive"
case driver.IsolationLevel(sql.LevelSerializable):
@@ -334,13 +357,14 @@ func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, e
c.txReset = ``
txBegin := `BEGIN ` + txLock
if opts.ReadOnly {
if opts.ReadOnly && !c.readOnly {
txBegin += ` ; PRAGMA query_only=on`
c.txReset = `; PRAGMA query_only=` + string(c.readOnly)
c.txReset = `; PRAGMA query_only=off`
}
old := c.Conn.SetInterrupt(ctx)
defer c.Conn.SetInterrupt(old)
if old := c.Conn.SetInterrupt(ctx); old != ctx {
defer c.Conn.SetInterrupt(old)
}
err := c.Conn.Exec(txBegin)
if err != nil {
@@ -358,13 +382,12 @@ func (c *conn) Commit() error {
}
func (c *conn) Rollback() error {
err := c.Conn.Exec(`ROLLBACK` + c.txReset)
if errors.Is(err, sqlite3.INTERRUPT) {
old := c.Conn.SetInterrupt(context.Background())
// ROLLBACK even if interrupted.
ctx := context.Background()
if old := c.Conn.SetInterrupt(ctx); old != ctx {
defer c.Conn.SetInterrupt(old)
err = c.Conn.Exec(`ROLLBACK` + c.txReset)
}
return err
return c.Conn.Exec(`ROLLBACK` + c.txReset)
}
func (c *conn) Prepare(query string) (driver.Stmt, error) {
@@ -373,8 +396,9 @@ func (c *conn) Prepare(query string) (driver.Stmt, error) {
}
func (c *conn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
old := c.Conn.SetInterrupt(ctx)
defer c.Conn.SetInterrupt(old)
if old := c.Conn.SetInterrupt(ctx); old != ctx {
defer c.Conn.SetInterrupt(old)
}
s, tail, err := c.Conn.Prepare(query)
if err != nil {
@@ -382,7 +406,7 @@ func (c *conn) PrepareContext(ctx context.Context, query string) (driver.Stmt, e
}
if notWhitespace(tail) {
s.Close()
return nil, util.TailErr
return nil, errutil.TailErr
}
return &stmt{Stmt: s, tmRead: c.tmRead, tmWrite: c.tmWrite, inputs: -2}, nil
}
@@ -399,8 +423,9 @@ func (c *conn) ExecContext(ctx context.Context, query string, args []driver.Name
return resultRowsAffected(0), nil
}
old := c.Conn.SetInterrupt(ctx)
defer c.Conn.SetInterrupt(old)
if old := c.Conn.SetInterrupt(ctx); old != ctx {
defer c.Conn.SetInterrupt(old)
}
err := c.Conn.Exec(query)
if err != nil {
@@ -463,8 +488,10 @@ func (s *stmt) ExecContext(ctx context.Context, args []driver.NamedValue) (drive
return nil, err
}
old := s.Stmt.Conn().SetInterrupt(ctx)
defer s.Stmt.Conn().SetInterrupt(old)
c := s.Stmt.Conn()
if old := c.SetInterrupt(ctx); old != ctx {
defer c.SetInterrupt(old)
}
err = errors.Join(
s.Stmt.Exec(),
@@ -473,7 +500,7 @@ func (s *stmt) ExecContext(ctx context.Context, args []driver.NamedValue) (drive
return nil, err
}
return newResult(s.Stmt.Conn()), nil
return newResult(c), nil
}
func (s *stmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
@@ -518,12 +545,12 @@ func (s *stmt) setupBindings(args []driver.NamedValue) (err error) {
err = s.Stmt.BindTime(id, a, s.tmWrite)
case util.JSON:
err = s.Stmt.BindJSON(id, a.Value)
case util.PointerUnwrap:
err = s.Stmt.BindPointer(id, util.UnwrapPointer(a))
case util.Pointer:
err = s.Stmt.BindPointer(id, a.Value)
case nil:
err = s.Stmt.BindNull(id)
default:
panic(util.AssertErr())
panic(errutil.AssertErr())
}
if err != nil {
return err
@@ -537,7 +564,7 @@ func (s *stmt) CheckNamedValue(arg *driver.NamedValue) error {
switch arg.Value.(type) {
case bool, int, int64, float64, string, []byte,
time.Time, sqlite3.ZeroBlob,
util.JSON, util.PointerUnwrap,
util.JSON, util.Pointer,
nil:
return nil
default:
@@ -576,28 +603,59 @@ func (r resultRowsAffected) RowsAffected() (int64, error) {
return int64(r), nil
}
type scantype byte
const (
_ANY scantype = iota
_INT
_REAL
_TEXT
_BLOB
_NULL
_BOOL
_TIME
_NOT_NULL
)
var (
_ [0]struct{} = [scantype(sqlite3.INTEGER) - _INT]struct{}{}
_ [0]struct{} = [scantype(sqlite3.FLOAT) - _REAL]struct{}{}
_ [0]struct{} = [scantype(sqlite3.TEXT) - _TEXT]struct{}{}
_ [0]struct{} = [scantype(sqlite3.BLOB) - _BLOB]struct{}{}
_ [0]struct{} = [scantype(sqlite3.NULL) - _NULL]struct{}{}
_ [0]struct{} = [_NOT_NULL & (_NOT_NULL - 1)]struct{}{}
)
func scanFromDecl(decl string) scantype {
// These types are only used before we have rows,
// and otherwise as type hints.
// The first few ensure STRICT tables are strictly typed.
// The other two are type hints for booleans and time.
switch decl {
case "INT", "INTEGER":
return _INT
case "REAL":
return _REAL
case "TEXT":
return _TEXT
case "BLOB":
return _BLOB
case "BOOLEAN":
return _BOOL
case "DATE", "TIME", "DATETIME", "TIMESTAMP":
return _TIME
}
return _ANY
}
type rows struct {
ctx context.Context
*stmt
names []string
types []string
nulls []bool
scans []scantype
}
type scantype byte
const (
_ANY scantype = iota
_INT scantype = scantype(sqlite3.INTEGER)
_REAL scantype = scantype(sqlite3.FLOAT)
_TEXT scantype = scantype(sqlite3.TEXT)
_BLOB scantype = scantype(sqlite3.BLOB)
_NULL scantype = scantype(sqlite3.NULL)
_BOOL scantype = iota
_TIME
)
var (
// Ensure these interfaces are implemented:
_ driver.RowsColumnTypeDatabaseTypeName = &rows{}
@@ -622,79 +680,69 @@ func (r *rows) Columns() []string {
return r.names
}
func (r *rows) scanType(index int) scantype {
if r.scans == nil {
count := len(r.names)
scans := make([]scantype, count)
for i := range scans {
scans[i] = scanFromDecl(strings.ToUpper(r.Stmt.ColumnDeclType(i)))
}
r.scans = scans
}
return r.scans[index] &^ _NOT_NULL
}
func (r *rows) loadColumnMetadata() {
if r.nulls == nil {
count := r.Stmt.ColumnCount()
nulls := make([]bool, count)
if r.types == nil {
c := r.Stmt.Conn()
count := len(r.names)
types := make([]string, count)
scans := make([]scantype, count)
for i := range nulls {
if col := r.Stmt.ColumnOriginName(i); col != "" {
types[i], _, nulls[i], _, _, _ = r.Stmt.Conn().TableColumnMetadata(
for i := range types {
var declType string
var notNull, autoInc bool
if column := r.Stmt.ColumnOriginName(i); column != "" {
declType, _, notNull, _, autoInc, _ = c.TableColumnMetadata(
r.Stmt.ColumnDatabaseName(i),
r.Stmt.ColumnTableName(i),
col)
types[i] = strings.ToUpper(types[i])
// These types are only used before we have rows,
// and otherwise as type hints.
// The first few ensure STRICT tables are strictly typed.
// The other two are type hints for booleans and time.
switch types[i] {
case "INT", "INTEGER":
scans[i] = _INT
case "REAL":
scans[i] = _REAL
case "TEXT":
scans[i] = _TEXT
case "BLOB":
scans[i] = _BLOB
case "BOOLEAN":
scans[i] = _BOOL
case "DATE", "TIME", "DATETIME", "TIMESTAMP":
scans[i] = _TIME
}
column)
} else {
declType = r.Stmt.ColumnDeclType(i)
}
if declType != "" {
declType = strings.ToUpper(declType)
scans[i] = scanFromDecl(declType)
types[i] = declType
}
if notNull || autoInc {
scans[i] |= _NOT_NULL
}
}
r.nulls = nulls
r.types = types
r.scans = scans
}
}
func (r *rows) declType(index int) string {
if r.types == nil {
count := r.Stmt.ColumnCount()
types := make([]string, count)
for i := range types {
types[i] = strings.ToUpper(r.Stmt.ColumnDeclType(i))
}
r.types = types
}
return r.types[index]
}
func (r *rows) ColumnTypeDatabaseTypeName(index int) string {
r.loadColumnMetadata()
decltype := r.types[index]
if len := len(decltype); len > 0 && decltype[len-1] == ')' {
if i := strings.LastIndexByte(decltype, '('); i >= 0 {
decltype = decltype[:i]
decl := r.types[index]
if len := len(decl); len > 0 && decl[len-1] == ')' {
if i := strings.LastIndexByte(decl, '('); i >= 0 {
decl = decl[:i]
}
}
return strings.TrimSpace(decltype)
return strings.TrimSpace(decl)
}
func (r *rows) ColumnTypeNullable(index int) (nullable, ok bool) {
r.loadColumnMetadata()
if r.nulls[index] {
return false, true
}
return true, false
nullable = r.scans[index]&^_NOT_NULL == 0
return nullable, !nullable
}
func (r *rows) ColumnTypeScanType(index int) (typ reflect.Type) {
r.loadColumnMetadata()
scan := r.scans[index]
scan := r.scans[index] &^ _NOT_NULL
if r.Stmt.Busy() {
// SQLite is dynamically typed and we now have a row.
@@ -706,7 +754,7 @@ func (r *rows) ColumnTypeScanType(index int) (typ reflect.Type) {
switch {
case scan == _TIME && val != _BLOB && val != _NULL:
t := r.Stmt.ColumnTime(index, r.tmRead)
useValType = t == time.Time{}
useValType = t.IsZero()
case scan == _BOOL && val == _INT:
i := r.Stmt.ColumnInt64(index)
useValType = i != 0 && i != 1
@@ -736,48 +784,62 @@ func (r *rows) ColumnTypeScanType(index int) (typ reflect.Type) {
}
}
func (r *rows) Next(dest []driver.Value) error {
old := r.Stmt.Conn().SetInterrupt(r.ctx)
defer r.Stmt.Conn().SetInterrupt(old)
func (r *rows) NextRow() error {
c := r.Stmt.Conn()
if old := c.SetInterrupt(r.ctx); old != r.ctx {
defer c.SetInterrupt(old)
}
if r.Stmt.Step() {
return nil
}
if err := r.Stmt.Err(); err != nil {
return err
}
return io.EOF
}
if !r.Stmt.Step() {
if err := r.Stmt.Err(); err != nil {
return err
}
return io.EOF
func (r *rows) Next(dest []driver.Value) error {
if err := r.NextRow(); err != nil {
return err
}
data := unsafe.Slice((*any)(unsafe.SliceData(dest)), len(dest))
err := r.Stmt.Columns(data...)
for i := range dest {
if t, ok := r.decodeTime(i, dest[i]); ok {
dest[i] = t
}
if err := r.Stmt.ColumnsRaw(data...); err != nil {
return err
}
return err
for i := range dest {
dest[i] = r.convert(i, dest[i])
}
return nil
}
func (r *rows) decodeTime(i int, v any) (_ time.Time, ok bool) {
switch v := v.(type) {
case int64, float64:
// could be a time value
case string:
if r.tmWrite != "" && r.tmWrite != time.RFC3339 && r.tmWrite != time.RFC3339Nano {
break
func (r *rows) convert(i int, val driver.Value) driver.Value {
scan := r.scanType(i)
if v, ok := val.([]byte); ok {
if len(v) == cap(v) { // a BLOB
return val
}
t, ok := maybeTime(v)
if ok {
return t, true
if scan != _TEXT {
t, ok := r.maybeTime(v)
if ok {
return t
}
}
default:
return
val = string(v)
}
switch r.declType(i) {
case "DATE", "TIME", "DATETIME", "TIMESTAMP":
// could be a time value
default:
return
switch scan {
case _TIME:
t, err := r.tmRead.Decode(val)
if err == nil {
return t
}
case _BOOL:
switch val {
case int64(0):
return false
case int64(1):
return true
}
}
t, err := r.tmRead.Decode(v)
return t, err == nil
return val
}
+146
View File
@@ -0,0 +1,146 @@
//go:build go1.27
package driver
import (
"database/sql"
"database/sql/driver"
"math"
"time"
"github.com/hanzoai/sqlite3"
)
func (r *rows) ScanColumn(ctx driver.ScanContext, i int, dest any) error {
typ := r.Stmt.ColumnType(i)
// Fast path.
var src any
switch typ {
case sqlite3.NULL:
// src = nil
case sqlite3.FLOAT:
f := r.stmt.ColumnFloat(i)
if r.scanFloat(f, dest) {
return nil
}
src = f
case sqlite3.INTEGER:
i := r.stmt.ColumnInt64(i)
if r.scanInt(i, dest) {
return nil
}
if r.scanFloat(float64(i), dest) {
return nil
}
src = i
default:
var b []byte
if typ == sqlite3.TEXT {
b = r.stmt.ColumnRawText(i)
} else {
b = r.stmt.ColumnRawBlob(i)
}
switch d := dest.(type) {
case *sql.RawBytes:
*d = b
return nil
case *string:
*d = string(b)
return nil
case *sql.NullString:
d.String = string(b)
d.Valid = true
return nil
case *[]byte:
*d = append((*d)[:0], b...)
return nil
}
src = b
}
// Time handling.
if typ != sqlite3.NULL && typ != sqlite3.BLOB {
var ok bool
switch d := dest.(type) {
case *time.Time:
*d, ok = r.scanTime(src)
case *sql.NullTime:
d.Time, ok = r.scanTime(src)
d.Valid = ok
}
if ok {
return nil
}
}
// Fallback.
return sql.ConvertAssign(ctx, dest, r.convert(i, src))
}
func (r *rows) scanTime(src any) (_ time.Time, _ bool) {
if s, ok := src.([]byte); ok {
if t, ok := r.maybeTime(s); ok {
return t, true
}
src = string(s)
}
t, err := r.tmRead.Decode(src)
return t, err == nil
}
func (r *rows) scanFloat(f float64, dest any) bool {
switch d := dest.(type) {
case *float64:
*d = f
case *float32:
*d = float32(f)
case *sql.NullFloat64:
d.Float64 = f
d.Valid = true
case *sql.Null[float32]:
d.V = float32(f)
d.Valid = true
default:
return false
}
return true
}
func (r *rows) scanInt(i int64, dest any) bool {
switch d := dest.(type) {
case *int64:
*d = i
case *sql.NullInt64:
d.Int64 = i
d.Valid = true
case *uint64:
*d = uint64(i)
return 0 <= i
case *sql.Null[uint64]:
d.V = uint64(i)
d.Valid = true
return 0 <= i
case *int:
*d = int(i)
return math.MinInt <= i && i <= math.MaxInt
case *sql.Null[int]:
d.V = int(i)
d.Valid = true
return math.MinInt <= i && i <= math.MaxInt
case *uint:
*d = uint(i)
return 0 <= i && uint64(i) <= math.MaxUint
case *sql.Null[uint]:
d.V = uint(i)
d.Valid = true
return 0 <= i && uint64(i) <= math.MaxUint
default:
return false
}
return true
}
+182 -53
View File
@@ -8,14 +8,15 @@ import (
"math"
"net/url"
"reflect"
"runtime"
"strings"
"testing"
"time"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Test_Open_error(t *testing.T) {
@@ -33,35 +34,40 @@ func Test_Open_error(t *testing.T) {
func Test_Open_dir(t *testing.T) {
t.Parallel()
db, err := sql.Open("sqlite3", ".")
ctx := testcfg.Context(t)
db, err := Open(".")
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.Conn(context.TODO())
_, err = db.Conn(ctx)
if err == nil {
t.Fatal("want error")
}
if !errors.Is(err, sqlite3.CANTOPEN) {
t.Errorf("got %v, want sqlite3.CANTOPEN", err)
if runtime.GOOS == "js" {
t.Skip("skipping on JS")
}
if !errors.Is(err, sqlite3.CANTOPEN_ISDIR) {
t.Errorf("got %v, want sqlite3.CANTOPEN_ISDIR", err)
}
}
func Test_Open_pragma(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
dsn := memdb.TestDB(t, url.Values{
"_pragma": {"busy_timeout(1000)"},
})
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var timeout int
err = db.QueryRow(`PRAGMA busy_timeout`).Scan(&timeout)
err = db.QueryRowContext(ctx, `PRAGMA busy_timeout`).Scan(&timeout)
if err != nil {
t.Fatal(err)
}
@@ -72,17 +78,18 @@ func Test_Open_pragma(t *testing.T) {
func Test_Open_pragma_invalid(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
dsn := memdb.TestDB(t, url.Values{
"_pragma": {"busy_timeout 1000"},
})
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.Conn(context.TODO())
_, err = db.Conn(ctx)
if err == nil {
t.Fatal("want error")
}
@@ -100,23 +107,24 @@ func Test_Open_pragma_invalid(t *testing.T) {
func Test_Open_txLock(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
dsn := memdb.TestDB(t, url.Values{
"_txlock": {"exclusive"},
"_pragma": {"busy_timeout(1000)"},
})
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
tx1, err := db.Begin()
tx1, err := db.BeginTx(ctx, nil)
if err != nil {
t.Fatal(err)
}
_, err = db.Begin()
_, err = db.BeginTx(ctx, nil)
if err == nil {
t.Error("want error")
}
@@ -136,11 +144,11 @@ func Test_Open_txLock(t *testing.T) {
func Test_Open_txLock_invalid(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
dsn := memdb.TestDB(t, url.Values{
"_txlock": {"xclusive"},
})
_, err := sql.Open("sqlite3", tmp+"_txlock=xclusive")
_, err := Open(dsn)
if err == nil {
t.Fatal("want error")
}
@@ -151,22 +159,20 @@ func Test_Open_txLock_invalid(t *testing.T) {
func Test_BeginTx(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t, url.Values{
dsn := memdb.TestDB(t, url.Values{
"_txlock": {"exclusive"},
"_pragma": {"busy_timeout(0)"},
})
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.BeginTx(ctx, &sql.TxOptions{Isolation: sql.LevelReadCommitted})
if err.Error() != string(util.IsolationErr) {
if err.Error() != string(errutil.IsolationErr) {
t.Error("want isolationErr")
}
@@ -199,18 +205,78 @@ func Test_BeginTx(t *testing.T) {
}
}
func Test_nested_context(t *testing.T) {
t.Parallel()
dsn := memdb.TestDB(t)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
tx, err := db.BeginTx(ctx, nil)
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
outer, err := tx.Query(`SELECT value FROM generate_series(0)`)
if err != nil {
t.Fatal(err)
}
defer outer.Close()
want := func(rows *sql.Rows, want int) {
t.Helper()
var got int
rows.Next()
if err := rows.Scan(&got); err != nil {
t.Fatal(err)
}
if got != want {
t.Errorf("got %d, want %d", got, want)
}
}
want(outer, 0)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
inner, err := tx.QueryContext(ctx, `SELECT value FROM generate_series(0)`)
if err != nil {
t.Fatal(err)
}
defer inner.Close()
want(inner, 0)
cancel()
var terr interface{ Temporary() bool }
if inner.Next() || !errors.Is(inner.Err(), context.Canceled) &&
(!errors.As(inner.Err(), &terr) || !terr.Temporary()) {
t.Fatalf("got %v, want cancellation", inner.Err())
}
want(outer, 1)
}
func Test_Prepare(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var serr *sqlite3.Error
_, err = db.Prepare(`SELECT`)
_, err = db.PrepareContext(ctx, `SELECT`)
if err == nil {
t.Error("want error")
}
@@ -224,30 +290,28 @@ func Test_Prepare(t *testing.T) {
t.Error("got message:", got)
}
_, err = db.Prepare(`SELECT 1; `)
_, err = db.PrepareContext(ctx, `SELECT 1; `)
if err != nil {
t.Error(err)
}
_, err = db.Prepare(`SELECT 1; SELECT`)
if err.Error() != string(util.TailErr) {
_, err = db.PrepareContext(ctx, `SELECT 1; SELECT`)
if err.Error() != string(errutil.TailErr) {
t.Error("want tailErr")
}
_, err = db.Prepare(`SELECT 1; SELECT 2`)
if err.Error() != string(util.TailErr) {
_, err = db.PrepareContext(ctx, `SELECT 1; SELECT 2`)
if err.Error() != string(errutil.TailErr) {
t.Error("want tailErr")
}
}
func Test_QueryRow_named(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
@@ -266,7 +330,7 @@ func Test_QueryRow_named(t *testing.T) {
defer stmt.Close()
date := time.Now()
row := stmt.QueryRow(true, sql.Named("AAA", math.Pi), nil /*3*/, nil /*4*/, date /*5*/)
row := stmt.QueryRowContext(ctx, true, sql.Named("AAA", math.Pi), nil /*3*/, nil /*4*/, date /*5*/)
var first bool
var fifth time.Time
@@ -295,15 +359,16 @@ func Test_QueryRow_named(t *testing.T) {
func Test_QueryRow_blob_null(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
rows, err := db.Query(`
rows, err := db.QueryContext(ctx, `
SELECT NULL UNION ALL
SELECT x'cafe' UNION ALL
SELECT x'babe' UNION ALL
@@ -332,11 +397,12 @@ func Test_time(t *testing.T) {
for _, fmt := range []string{"auto", "sqlite", "rfc3339", time.ANSIC} {
t.Run(fmt, func(t *testing.T) {
tmp := memdb.TestDB(t, url.Values{
dsn := memdb.TestDB(t, url.Values{
"_timefmt": {fmt},
})
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
@@ -344,18 +410,18 @@ func Test_time(t *testing.T) {
twosday := time.Date(2022, 2, 22, 22, 22, 22, 0, time.UTC)
_, err = db.Exec(`CREATE TABLE test (at DATETIME)`)
_, err = db.ExecContext(ctx, `CREATE TABLE test (at DATETIME)`)
if err != nil {
t.Fatal(err)
}
_, err = db.Exec(`INSERT INTO test VALUES (?)`, twosday)
_, err = db.ExecContext(ctx, `INSERT INTO test VALUES (?)`, twosday)
if err != nil {
t.Fatal(err)
}
var got time.Time
err = db.QueryRow(`SELECT * FROM test`).Scan(&got)
err = db.QueryRowContext(ctx, `SELECT * FROM test`).Scan(&got)
if err != nil {
t.Fatal(err)
}
@@ -379,15 +445,16 @@ func Test_ColumnType_ScanType(t *testing.T) {
)
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := sql.Open("sqlite3", tmp)
ctx := testcfg.Context(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
_, err = db.ExecContext(ctx, `
CREATE TABLE test (
col_int INTEGER,
col_real REAL,
@@ -412,7 +479,7 @@ func Test_ColumnType_ScanType(t *testing.T) {
t.Fatal(err)
}
rows, err := db.Query(`SELECT * FROM test`)
rows, err := db.QueryContext(ctx, `SELECT * FROM test`)
if err != nil {
t.Fatal(err)
}
@@ -467,3 +534,65 @@ func Test_ColumnType_ScanType(t *testing.T) {
t.Fatal(err)
}
}
func Test_rows_ScanColumn(t *testing.T) {
t.Parallel()
dsn := memdb.TestDB(t)
db, err := Open(dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var tm time.Time
err = db.QueryRow(`SELECT NULL`).Scan(&tm)
if err == nil {
t.Error("want error")
}
// Go 1.27
err = db.QueryRow(`SELECT datetime()`).Scan(&tm)
if err != nil && !strings.HasPrefix(err.Error(), "sql: Scan error") {
t.Error(err)
}
if err == nil && tm.IsZero() {
t.Error(tm)
}
var nt sql.NullTime
err = db.QueryRow(`SELECT NULL`).Scan(&nt)
if err != nil {
t.Error(err)
}
// Go 1.27
err = db.QueryRow(`SELECT datetime()`).Scan(&nt)
if err != nil && !strings.HasPrefix(err.Error(), "sql: Scan error") {
t.Error(err)
}
if err == nil && nt.Time.IsZero() {
t.Error(nt.Time)
}
}
func Benchmark_loop(b *testing.B) {
ctx := testcfg.Context(b)
db, err := Open(":memory:")
if err != nil {
b.Fatal(err)
}
defer db.Close()
var version string
err = db.QueryRowContext(ctx, `SELECT sqlite_version();`).Scan(&version)
if err != nil {
b.Fatal(err)
}
for b.Loop() {
_, err := db.ExecContext(b.Context(),
`WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x < 1000000) SELECT x FROM c;`)
if err != nil {
b.Fatal(err)
}
}
}
+4 -9
View File
@@ -1,9 +1,5 @@
//go:build linux || darwin || windows || freebsd || openbsd || netbsd || dragonfly || illumos || sqlite3_flock || sqlite3_dotlk
package driver_test
// Adapted from: https://go.dev/doc/tutorial/database-access
import (
"database/sql"
"database/sql/driver"
@@ -11,10 +7,9 @@ import (
"log"
"time"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
_ "github.com/hanzoai/sqlite3/driver"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example_customTime() {
@@ -27,7 +22,7 @@ func Example_customTime() {
_, err = db.Exec(`
CREATE TABLE data (
id INTEGER PRIMARY KEY,
date_time TEXT
date_time ANY
) STRICT;
`)
if err != nil {
+2 -3
View File
@@ -10,9 +10,8 @@ import (
"log"
"os"
_ "github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
_ "github.com/hanzoai/sqlite3/driver"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
var db *sql.DB
+3 -4
View File
@@ -4,10 +4,9 @@ import (
"fmt"
"log"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example_json() {
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"database/sql"
"time"
"github.com/ncruces/go-sqlite3"
"github.com/hanzoai/sqlite3"
)
// Savepoint establishes a new transaction savepoint.
+2 -4
View File
@@ -4,10 +4,8 @@ import (
"fmt"
"log"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3/driver"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
func ExampleSavepoint() {
+17 -6
View File
@@ -1,12 +1,23 @@
package driver
import "time"
import (
"bytes"
"time"
)
// Convert a string in [time.RFC3339Nano] format into a [time.Time]
// if it roundtrips back to the same string.
// if that's the format we're using to persist them,
// and if it roundtrips back to the same string.
// This way times can be persisted to, and recovered from, the database,
// but if a string is needed, [database/sql] will recover the same string.
func maybeTime(text string) (_ time.Time, _ bool) {
// but if a string is wanted, [database/sql] will recover the same string.
func (r *rows) maybeTime(text []byte) (_ time.Time, _ bool) {
switch r.tmWrite {
case "", time.RFC3339, time.RFC3339Nano:
break
default:
return
}
// Weed out (some) values that can't possibly be
// [time.RFC3339Nano] timestamps.
if len(text) < len("2006-01-02T15:04:05Z") {
@@ -21,8 +32,8 @@ func maybeTime(text string) (_ time.Time, _ bool) {
// Slow path.
var buf [len(time.RFC3339Nano)]byte
date, err := time.Parse(time.RFC3339Nano, text)
if err == nil && text == string(date.AppendFormat(buf[:0], time.RFC3339Nano)) {
date, err := time.Parse(time.RFC3339Nano, string(text))
if err == nil && bytes.Equal(text, date.AppendFormat(buf[:0], time.RFC3339Nano)) {
return date, true
}
return
+4 -2
View File
@@ -21,8 +21,9 @@ func Fuzz_stringOrTime_1(f *testing.F) {
f.Add("2006-01-02T15:04:05.9999999999Z")
f.Add("Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.")
r := rows{stmt: &stmt{}}
f.Fuzz(func(t *testing.T, str string) {
v, ok := maybeTime(str)
v, ok := r.maybeTime([]byte(str))
if ok {
// Make sure times round-trip to the same string:
// https://pkg.go.dev/database/sql#Rows.Scan
@@ -50,8 +51,9 @@ func Fuzz_stringOrTime_2(f *testing.F) {
f.Add(int64(639095955742), int64(222_222_222)) // twosday, year 22222AD
f.Add(int64(-763421161058), int64(222_222_222)) // twosday, year 22222BC
r := rows{stmt: &stmt{}}
checkTime := func(t testing.TB, date time.Time) {
v, ok := maybeTime(date.Format(time.RFC3339Nano))
v, ok := r.maybeTime(date.AppendFormat(nil, time.RFC3339Nano))
if ok {
// Make sure times round-trip to the same time:
if !v.Equal(date) {
+3 -4
View File
@@ -1,13 +1,11 @@
package driver
import (
"context"
"database/sql/driver"
"slices"
"testing"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/internal/testcfg"
)
func Test_namedValues(t *testing.T) {
@@ -56,7 +54,8 @@ func Fuzz_notWhitespace(f *testing.F) {
t.SkipNow()
}
c, err := db.Conn(context.Background())
ctx := testcfg.Context(t)
c, err := db.Conn(ctx)
if err != nil {
t.Fatal(err)
}
-41
View File
@@ -1,41 +0,0 @@
# Embeddable Wasm build of SQLite
This folder includes an embeddable Wasm build of SQLite 3.49.1 for use with
[`github.com/ncruces/go-sqlite3`](https://pkg.go.dev/github.com/ncruces/go-sqlite3).
The following optional features are compiled in:
- [math functions](https://sqlite.org/lang_mathfunc.html)
- [FTS5](https://sqlite.org/fts5.html)
- [JSON](https://sqlite.org/json1.html)
- [R*Tree](https://sqlite.org/rtree.html)
- [GeoPoly](https://sqlite.org/geopoly.html)
- [Spellfix1](https://sqlite.org/spellfix1.html)
- [soundex](https://sqlite.org/lang_corefunc.html#soundex)
- [stat4](https://sqlite.org/compile.html#enable_stat4)
- [base64](https://github.com/sqlite/sqlite/blob/master/ext/misc/base64.c)
- [decimal](https://github.com/sqlite/sqlite/blob/master/ext/misc/decimal.c)
- [ieee754](https://github.com/sqlite/sqlite/blob/master/ext/misc/ieee754.c)
- [regexp](https://github.com/sqlite/sqlite/blob/master/ext/misc/regexp.c)
- [series](https://github.com/sqlite/sqlite/blob/master/ext/misc/series.c)
- [uint](https://github.com/sqlite/sqlite/blob/master/ext/misc/uint.c)
- [time](../sqlite3/time.c)
See the [configuration options](../sqlite3/sqlite_opt.h),
and [patches](../sqlite3) applied.
Built using [`wasi-sdk`](https://github.com/WebAssembly/wasi-sdk),
and [`binaryen`](https://github.com/WebAssembly/binaryen).
The build is easily reproducible, and verifiable, using
[Artifact Attestations](https://github.com/ncruces/go-sqlite3/attestations).
### Customizing the build
You can use your own custom build of SQLite.
Examples of custom builds of SQLite are:
- [`github.com/ncruces/go-sqlite3/embed/bcw2`](https://github.com/ncruces/go-sqlite3/tree/main/embed/bcw2)
built from a branch supporting [`BEGIN CONCURRENT`](https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md)
and [Wal2](https://sqlite.org/cgi/src/doc/wal2/doc/wal2.md).
- [`github.com/asg017/sqlite-vec-go-bindings/ncruces`](https://github.com/asg017/sqlite-vec-go-bindings)
which includes the [`sqlite-vec`](https://github.com/asg017/sqlite-vec) vector search extension.
-2
View File
@@ -1,2 +0,0 @@
build/
sqlite/
-22
View File
@@ -1,22 +0,0 @@
# Embeddable Wasm build of SQLite
This folder includes an alternative embeddable Wasm build of SQLite,
which includes the experimental
[`BEGIN CONCURRENT`](https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md) and
[Wal2](https://sqlite.org/cgi/src/doc/wal2/doc/wal2.md) patches.
It also enables the optional
[`UPDATE … ORDER BY … LIMIT`](https://sqlite.org/lang_update.html#optional_limit_and_order_by_clauses) and
[`DELETE … ORDER BY … LIMIT`](https://sqlite.org/lang_delete.html#optional_limit_and_order_by_clauses) clauses,
and the [`WITHIN GROUP ORDER BY`](https://sqlite.org/compile.html#enable_ordered_set_aggregates) aggregate syntax.
> [!IMPORTANT]
> This package is experimental.
> It is built from the `bedrock` branch of SQLite,
> since that is _currently_ the most stable, maintained branch to include these features.
> [!CAUTION]
> The Wal2 journaling mode creates databases that other versions of SQLite cannot access.
The build is easily reproducible, and verifiable, using
[Artifact Attestations](https://github.com/ncruces/go-sqlite3/attestations).
Binary file not shown.
-59
View File
@@ -1,59 +0,0 @@
package bcw2
import (
"path/filepath"
"testing"
"github.com/ncruces/go-sqlite3/driver"
"github.com/ncruces/go-sqlite3/ext/stats"
"github.com/ncruces/go-sqlite3/vfs"
)
func Test_bcw2(t *testing.T) {
if !vfs.SupportsSharedMemory {
t.Skip("skipping without shared memory")
}
tmp := filepath.ToSlash(filepath.Join(t.TempDir(), "test.db"))
db, err := driver.Open("file:"+tmp+"?_pragma=journal_mode(wal2)&_txlock=concurrent", stats.Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
_, err = tx.Exec(`CREATE TABLE test (col)`)
if err != nil {
t.Fatal(err)
}
_, err = tx.Exec(`DELETE FROM test LIMIT 1`)
if err != nil {
t.Fatal(err)
}
_, err = tx.Exec(`SELECT median() WITHIN GROUP (ORDER BY col) FROM test`)
if err != nil {
t.Fatal(err)
}
err = tx.Commit()
if err != nil {
t.Fatal(err)
}
var version string
err = db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
if err != nil {
t.Fatal(err)
}
if version != "3.50.0" {
t.Error(version)
}
}
-66
View File
@@ -1,66 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
cd -P -- "$(dirname -- "$0")"
ROOT=../../
BINARYEN="$ROOT/tools/binaryen/bin"
WASI_SDK="$ROOT/tools/wasi-sdk/bin"
trap 'rm -rf build/ sqlite/ bcw2.tmp' EXIT
mkdir -p build/ext/
cp "$ROOT"/sqlite3/*.[ch] build/
cp "$ROOT"/sqlite3/*.patch build/
# https://sqlite.org/src/info/c09656c62155a6e8
curl -# https://sqlite.org/src/tarball/sqlite.tar.gz?r=c09656c6 | tar xz
cd sqlite
if [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
MSYS_NO_PATHCONV=1 nmake /f makefile.msc sqlite3.c "OPTS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT -DSQLITE_ENABLE_ORDERED_SET_AGGREGATES"
else
sh configure --enable-update-limit
OPTS=-DSQLITE_ENABLE_ORDERED_SET_AGGREGATES make sqlite3.c
fi
cd ~-
mv sqlite/sqlite3.c build/
mv sqlite/sqlite3.h build/
mv sqlite/sqlite3ext.h build/
mv sqlite/ext/misc/anycollseq.c build/ext/
mv sqlite/ext/misc/base64.c build/ext/
mv sqlite/ext/misc/decimal.c build/ext/
mv sqlite/ext/misc/ieee754.c build/ext/
mv sqlite/ext/misc/regexp.c build/ext/
mv sqlite/ext/misc/series.c build/ext/
mv sqlite/ext/misc/spellfix.c build/ext/
mv sqlite/ext/misc/uint.c build/ext/
cd build
cat *.patch | patch -p0 --no-backup-if-mismatch
cd ~-
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-Wall -Wextra -Wno-unused-parameter -Wno-unused-function \
-o bcw2.wasm "build/main.c" \
-I"build" \
-mexec-model=reactor \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
-Wl,--initial-memory=327680 \
-D_HAVE_SQLITE_CONFIG_H \
-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT \
-DSQLITE_CUSTOM_INCLUDE=sqlite_opt.h \
$(awk '{print "-Wl,--export="$0}' ../exports.txt)
"$BINARYEN/wasm-ctor-eval" -g -c _initialize bcw2.wasm -o bcw2.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
bcw2.tmp -o bcw2.wasm \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
-14
View File
@@ -1,14 +0,0 @@
module github.com/ncruces/go-sqlite3/embed/bcw2
go 1.22.0
toolchain go1.24.0
require github.com/ncruces/go-sqlite3 v0.23.1
require (
github.com/ncruces/julianday v1.0.0 // indirect
github.com/ncruces/sort v0.1.5 // indirect
github.com/tetratelabs/wazero v1.9.0 // indirect
golang.org/x/sys v0.30.0 // indirect
)
-12
View File
@@ -1,12 +0,0 @@
github.com/ncruces/go-sqlite3 v0.23.1 h1:zGAd76q+Tr18z/xKGatUlzBQdjR3J+rexfANUcjAgkY=
github.com/ncruces/go-sqlite3 v0.23.1/go.mod h1:Xg3FyAZl25HcBSFmcbymdfoTqD7jRnBUmv1jSrbIjdE=
github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt7M=
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/ncruces/sort v0.1.5 h1:fiFWXXAqKI8QckPf/6hu/bGFwcEPrirIOFaJqWujs4k=
github.com/ncruces/sort v0.1.5/go.mod h1:obJToO4rYr6VWP0Uw5FYymgYGt3Br4RXcs/JdKaXAPk=
github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I=
github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
-24
View File
@@ -1,24 +0,0 @@
// Package bcw2 embeds SQLite into your application.
//
// Importing package bcw2 initializes the [sqlite3.Binary] variable
// with a build of SQLite that includes the [BEGIN CONCURRENT] and [Wal2] patches:
//
// import _ "github.com/ncruces/go-sqlite3/embed/bcw2"
//
// [BEGIN CONCURRENT]: https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md
// [Wal2]: https://sqlite.org/cgi/src/doc/wal2/doc/wal2.md
package bcw2
import (
_ "embed"
"unsafe"
"github.com/ncruces/go-sqlite3"
)
//go:embed bcw2.wasm
var binary string
func init() {
sqlite3.Binary = unsafe.Slice(unsafe.StringData(binary), len(binary))
}
-33
View File
@@ -1,33 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
cd -P -- "$(dirname -- "$0")"
ROOT=../
BINARYEN="$ROOT/tools/binaryen/bin"
WASI_SDK="$ROOT/tools/wasi-sdk/bin"
trap 'rm -f sqlite3.tmp' EXIT
"$WASI_SDK/clang" --target=wasm32-wasi -std=c23 -g0 -O2 \
-Wall -Wextra -Wno-unused-parameter -Wno-unused-function \
-o sqlite3.wasm "$ROOT/sqlite3/main.c" \
-I"$ROOT/sqlite3" \
-mexec-model=reactor \
-msimd128 -mmutable-globals -mmultivalue \
-mbulk-memory -mreference-types \
-mnontrapping-fptoint -msign-ext \
-fno-stack-protector -fno-stack-clash-protection \
-Wl,--stack-first \
-Wl,--import-undefined \
-Wl,--initial-memory=327680 \
-D_HAVE_SQLITE_CONFIG_H \
-DSQLITE_CUSTOM_INCLUDE=sqlite_opt.h \
$(awk '{print "-Wl,--export="$0}' exports.txt)
"$BINARYEN/wasm-ctor-eval" -g -c _initialize sqlite3.wasm -o sqlite3.tmp
"$BINARYEN/wasm-opt" -g --strip --strip-producers -c -O3 \
sqlite3.tmp -o sqlite3.wasm \
--enable-simd --enable-mutable-globals --enable-multivalue \
--enable-bulk-memory --enable-reference-types \
--enable-nontrapping-float-to-int --enable-sign-ext
+8
View File
@@ -0,0 +1,8 @@
// Package embed is no longer needed.
//
// Deprecated: importing github.com/hanzoai/sqlite3/embed is unnecessary.
package embed
func init() {
println("If you're reading this, you're unnecessarily importing github.com/hanzoai/sqlite3/embed.")
}
-140
View File
@@ -1,140 +0,0 @@
aligned_alloc
sqlite3_anycollseq_init
sqlite3_autovacuum_pages_go
sqlite3_backup_finish
sqlite3_backup_init
sqlite3_backup_pagecount
sqlite3_backup_remaining
sqlite3_backup_step
sqlite3_bind_blob_go
sqlite3_bind_double
sqlite3_bind_int64
sqlite3_bind_null
sqlite3_bind_parameter_count
sqlite3_bind_parameter_index
sqlite3_bind_parameter_name
sqlite3_bind_pointer_go
sqlite3_bind_text_go
sqlite3_bind_value
sqlite3_bind_zeroblob64
sqlite3_blob_bytes
sqlite3_blob_close
sqlite3_blob_open
sqlite3_blob_read
sqlite3_blob_reopen
sqlite3_blob_write
sqlite3_busy_handler_go
sqlite3_busy_timeout
sqlite3_changes64
sqlite3_clear_bindings
sqlite3_close
sqlite3_close_v2
sqlite3_collation_needed_go
sqlite3_column_blob
sqlite3_column_bytes
sqlite3_column_count
sqlite3_column_database_name
sqlite3_column_decltype
sqlite3_column_double
sqlite3_column_int64
sqlite3_column_name
sqlite3_column_origin_name
sqlite3_column_table_name
sqlite3_column_text
sqlite3_column_type
sqlite3_column_value
sqlite3_columns_go
sqlite3_commit_hook_go
sqlite3_config_log_go
sqlite3_create_aggregate_function_go
sqlite3_create_collation_go
sqlite3_create_function_go
sqlite3_create_module_go
sqlite3_create_window_function_go
sqlite3_data_count
sqlite3_database_file_object
sqlite3_db_cacheflush
sqlite3_db_config
sqlite3_db_filename
sqlite3_db_name
sqlite3_db_readonly
sqlite3_db_release_memory
sqlite3_db_status
sqlite3_declare_vtab
sqlite3_errcode
sqlite3_errmsg
sqlite3_error_offset
sqlite3_errstr
sqlite3_exec
sqlite3_expanded_sql
sqlite3_file_control
sqlite3_filename_database
sqlite3_filename_journal
sqlite3_filename_wal
sqlite3_finalize
sqlite3_free
sqlite3_get_autocommit
sqlite3_get_auxdata
sqlite3_hard_heap_limit64
sqlite3_interrupt
sqlite3_invoke_busy_handler_go
sqlite3_last_insert_rowid
sqlite3_limit
sqlite3_malloc64
sqlite3_open_v2
sqlite3_overload_function
sqlite3_prepare_v3
sqlite3_progress_handler_go
sqlite3_realloc64
sqlite3_reset
sqlite3_result_blob_go
sqlite3_result_double
sqlite3_result_error
sqlite3_result_error_code
sqlite3_result_error_nomem
sqlite3_result_error_toobig
sqlite3_result_int64
sqlite3_result_null
sqlite3_result_pointer_go
sqlite3_result_text_go
sqlite3_result_value
sqlite3_result_zeroblob64
sqlite3_rollback_hook_go
sqlite3_set_authorizer_go
sqlite3_set_auxdata_go
sqlite3_set_last_insert_rowid
sqlite3_soft_heap_limit64
sqlite3_step
sqlite3_stmt_busy
sqlite3_stmt_readonly
sqlite3_stmt_status
sqlite3_table_column_metadata
sqlite3_total_changes64
sqlite3_trace_go
sqlite3_txn_state
sqlite3_update_hook_go
sqlite3_uri_key
sqlite3_value_blob
sqlite3_value_bytes
sqlite3_value_double
sqlite3_value_dup
sqlite3_value_free
sqlite3_value_frombind
sqlite3_value_int64
sqlite3_value_nochange
sqlite3_value_numeric_type
sqlite3_value_pointer_go
sqlite3_value_text
sqlite3_value_type
sqlite3_vtab_collation
sqlite3_vtab_config_go
sqlite3_vtab_distinct
sqlite3_vtab_in
sqlite3_vtab_in_first
sqlite3_vtab_in_next
sqlite3_vtab_nochange
sqlite3_vtab_on_conflict
sqlite3_vtab_rhs_value
sqlite3_wal_autocheckpoint
sqlite3_wal_checkpoint_v2
sqlite3_wal_hook_go
-21
View File
@@ -1,21 +0,0 @@
// Package embed embeds SQLite into your application.
//
// Importing package embed initializes the [sqlite3.Binary] variable
// with an appropriate build of SQLite:
//
// import _ "github.com/ncruces/go-sqlite3/embed"
package embed
import (
_ "embed"
"unsafe"
"github.com/ncruces/go-sqlite3"
)
//go:embed sqlite3.wasm
var binary string
func init() {
sqlite3.Binary = unsafe.Slice(unsafe.StringData(binary), len(binary))
}
-25
View File
@@ -1,25 +0,0 @@
package embed
import (
"testing"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
)
func Test_init(t *testing.T) {
db, err := driver.Open("file:/test.db?vfs=memdb")
if err != nil {
t.Fatal(err)
}
defer db.Close()
var version string
err = db.QueryRow(`SELECT sqlite_version()`).Scan(&version)
if err != nil {
t.Fatal(err)
}
if version != "3.49.1" {
t.Error(version)
}
}
Binary file not shown.
+33 -23
View File
@@ -2,17 +2,16 @@ package sqlite3
import (
"errors"
"strconv"
"strings"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3/internal/sqlite3_wrap"
)
// Error wraps an SQLite Error Code.
//
// https://sqlite.org/c3ref/errcode.html
type Error struct {
str string
sys error
msg string
sql string
code res_t
@@ -29,28 +28,34 @@ func (e *Error) Code() ErrorCode {
//
// https://sqlite.org/rescode.html
func (e *Error) ExtendedCode() ExtendedErrorCode {
return ExtendedErrorCode(e.code)
return xErrorCode(e.code)
}
// Error implements the error interface.
func (e *Error) Error() string {
var b strings.Builder
b.WriteString("sqlite3: ")
if e.str != "" {
b.WriteString(e.str)
} else {
b.WriteString(strconv.Itoa(int(e.code)))
}
b.WriteString(sqlite3_wrap.ErrorCodeString(e.code))
if e.msg != "" {
b.WriteString(": ")
b.WriteString(e.msg)
}
if e.sys != nil {
b.WriteString(": ")
b.WriteString(e.sys.Error())
}
return b.String()
}
// Unwrap returns the underlying operating system error
// that caused the I/O error or failure to open a file.
//
// https://sqlite.org/c3ref/system_errno.html
func (e *Error) Unwrap() error {
return e.sys
}
// Is tests whether this error matches a given [ErrorCode] or [ExtendedErrorCode].
//
// It makes it possible to do:
@@ -83,7 +88,7 @@ func (e *Error) As(err any) bool {
// Temporary returns true for [BUSY] errors.
func (e *Error) Temporary() bool {
return e.Code() == BUSY
return e.Code() == BUSY || e.Code() == INTERRUPT
}
// Timeout returns true for [BUSY_TIMEOUT] errors.
@@ -98,22 +103,31 @@ func (e *Error) SQL() string {
// Error implements the error interface.
func (e ErrorCode) Error() string {
return util.ErrorCodeString(uint32(e))
return sqlite3_wrap.ErrorCodeString(e)
}
// As converts this error to an [ExtendedErrorCode].
func (e ErrorCode) As(err any) bool {
c, ok := err.(*xErrorCode)
if ok {
*c = xErrorCode(e)
}
return ok
}
// Temporary returns true for [BUSY] errors.
func (e ErrorCode) Temporary() bool {
return e == BUSY
return e == BUSY || e == INTERRUPT
}
// ExtendedCode returns the extended error code for this error.
func (e ErrorCode) ExtendedCode() ExtendedErrorCode {
return ExtendedErrorCode(e)
return xErrorCode(e)
}
// Error implements the error interface.
func (e ExtendedErrorCode) Error() string {
return util.ErrorCodeString(uint32(e))
return sqlite3_wrap.ErrorCodeString(e)
}
// Is tests whether this error matches a given [ErrorCode].
@@ -133,7 +147,7 @@ func (e ExtendedErrorCode) As(err any) bool {
// Temporary returns true for [BUSY] errors.
func (e ExtendedErrorCode) Temporary() bool {
return ErrorCode(e) == BUSY
return ErrorCode(e) == BUSY || ErrorCode(e) == INTERRUPT
}
// Timeout returns true for [BUSY_TIMEOUT] errors.
@@ -158,14 +172,10 @@ func errorCode(err error, def ErrorCode) (msg string, code res_t) {
return code.msg, res_t(code.code)
}
var ecode ErrorCode
var xcode xErrorCode
switch {
case errors.As(err, &xcode):
if errors.As(err, &xcode) {
code = res_t(xcode)
case errors.As(err, &ecode):
code = res_t(ecode)
default:
} else {
code = res_t(def)
}
return err.Error(), code
+23 -22
View File
@@ -6,12 +6,13 @@ import (
"strings"
"testing"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/internal/sqlite3_wrap"
)
func Test_assertErr(t *testing.T) {
err := util.AssertErr()
if s := err.Error(); !strings.HasPrefix(s, "sqlite3: assertion failed") || !strings.HasSuffix(s, "error_test.go:13)") {
err := errutil.AssertErr()
if s := err.Error(); !strings.HasPrefix(s, "sqlite3: assertion failed") || !strings.HasSuffix(s, "error_test.go:14)") {
t.Errorf("got %q", s)
}
}
@@ -43,7 +44,7 @@ func TestError(t *testing.T) {
if !errors.Is(err, xErrorCode(0x8080)) {
t.Errorf("want true")
}
if s := err.Error(); s != "sqlite3: 32896" {
if s := err.Error(); s != "sqlite3: unknown error" {
t.Errorf("got %q", s)
}
if ok := errors.As(err.ExtendedCode(), &ecode); !ok || ecode != ErrorCode(0x80) {
@@ -83,7 +84,7 @@ func TestError_Temporary(t *testing.T) {
}
}
{
err := ExtendedErrorCode(tt.code)
err := xErrorCode(tt.code)
if got := err.Temporary(); got != tt.want {
t.Errorf("ExtendedErrorCode.Temporary(%d) = %v, want %v", tt.code, got, tt.want)
}
@@ -115,7 +116,7 @@ func TestError_Timeout(t *testing.T) {
}
}
{
err := ExtendedErrorCode(tt.code)
err := xErrorCode(tt.code)
if got := err.Timeout(); got != tt.want {
t.Errorf("Error.Timeout(%d) = %v, want %v", tt.code, got, tt.want)
}
@@ -127,7 +128,7 @@ func TestError_Timeout(t *testing.T) {
func Test_ErrorCode_Error(t *testing.T) {
t.Parallel()
db, err := Open(":memory:")
db, err := OpenContext(testContext(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -136,8 +137,8 @@ func Test_ErrorCode_Error(t *testing.T) {
// Test all error codes.
for i := 0; i == int(ErrorCode(i)); i++ {
want := "sqlite3: "
ptr := ptr_t(db.call("sqlite3_errstr", stk_t(i)))
want += util.ReadString(db.mod, ptr, _MAX_NAME)
ptr := ptr_t(db.wrp.Xsqlite3_errstr(int32(i)))
want += db.wrp.ReadString(ptr, _MAX_NAME)
got := ErrorCode(i).Error()
if got != want {
@@ -149,19 +150,19 @@ func Test_ErrorCode_Error(t *testing.T) {
func Test_ExtendedErrorCode_Error(t *testing.T) {
t.Parallel()
db, err := Open(":memory:")
db, err := OpenContext(testContext(t), ":memory:")
if err != nil {
t.Fatal(err)
}
defer db.Close()
// Test all extended error codes.
for i := 0; i == int(ExtendedErrorCode(i)); i++ {
for i := 0; i == int(xErrorCode(i)); i++ {
want := "sqlite3: "
ptr := ptr_t(db.call("sqlite3_errstr", stk_t(i)))
want += util.ReadString(db.mod, ptr, _MAX_NAME)
ptr := ptr_t(db.wrp.Xsqlite3_errstr(int32(i)))
want += db.wrp.ReadString(ptr, _MAX_NAME)
got := ExtendedErrorCode(i).Error()
got := xErrorCode(i).Error()
if got != want {
t.Fatalf("got %q, want %q, with %d", got, want, i)
}
@@ -175,14 +176,14 @@ func Test_errorCode(t *testing.T) {
wantCode res_t
}{
{nil, "", _OK},
{ERROR, "", util.ERROR},
{IOERR, "", util.IOERR},
{IOERR_READ, "", util.IOERR_READ},
{&Error{code: util.ERROR}, "", util.ERROR},
{fmt.Errorf("%w", ERROR), ERROR.Error(), util.ERROR},
{fmt.Errorf("%w", IOERR), IOERR.Error(), util.IOERR},
{fmt.Errorf("%w", IOERR_READ), IOERR_READ.Error(), util.IOERR_READ},
{fmt.Errorf("error"), "error", util.ERROR},
{ERROR, "", sqlite3_wrap.ERROR},
{IOERR, "", sqlite3_wrap.IOERR},
{IOERR_READ, "", sqlite3_wrap.IOERR_READ},
{&Error{code: sqlite3_wrap.ERROR}, "", sqlite3_wrap.ERROR},
{fmt.Errorf("%w", ERROR), ERROR.Error(), sqlite3_wrap.ERROR},
{fmt.Errorf("%w", IOERR), IOERR.Error(), sqlite3_wrap.IOERR},
{fmt.Errorf("%w", IOERR_READ), IOERR_READ.Error(), sqlite3_wrap.IOERR_READ},
{fmt.Errorf("error"), "error", sqlite3_wrap.ERROR},
}
for _, tt := range tests {
t.Run("", func(t *testing.T) {
+1 -2
View File
@@ -4,8 +4,7 @@ import (
"fmt"
"log"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/hanzoai/sqlite3"
)
const memory = ":memory:"
+138
View File
@@ -0,0 +1,138 @@
package sqlite3
import (
"bytes"
"encoding/base64"
"sync"
"github.com/hanzoai/sqlite3/internal/errutil"
)
var (
// +checklocks:extRegistryMtx
extRegistry []func(*Conn) error
extRegistryMtx sync.RWMutex
)
// AutoExtension causes the entryPoint function to be invoked
// for each new database connection that is created.
//
// https://sqlite.org/c3ref/auto_extension.html
func AutoExtension(entryPoint func(*Conn) error) {
extRegistryMtx.Lock()
extRegistry = append(extRegistry, entryPoint)
extRegistryMtx.Unlock()
}
func initExtensions(c *Conn) error {
c.base64()
extRegistryMtx.RLock()
defer extRegistryMtx.RUnlock()
for _, f := range extRegistry {
if err := f(c); err != nil {
return err
}
}
return nil
}
func (c *Conn) base64() error {
return c.CreateFunction("base64", 1, DETERMINISTIC, func(ctx Context, arg ...Value) {
switch a := arg[0]; a.Type() {
case NULL:
case BLOB:
data := a.RawBlob()
code := base64.StdEncoding
size := int64(code.EncodedLen(len(data)))
if size > _MAX_LENGTH {
ctx.ResultError(TOOBIG)
return
}
ptr := c.wrp.New(size)
if size > 0 {
code.Encode(c.wrp.Bytes(ptr, size), data)
}
ctx.c.wrp.Xsqlite3_result_text_go(int32(ctx.handle), int32(ptr), size)
case TEXT:
data := a.RawText()
data = bytes.Trim(data, " \t\n\v\f\r")
data = bytes.TrimRight(data, "=")
code := base64.RawStdEncoding
size := int64(code.DecodedLen(len(data)))
if size > _MAX_LENGTH {
ctx.ResultError(TOOBIG)
return
}
ptr := c.wrp.New(size)
if size > 0 {
n, _ := code.Decode(c.wrp.Bytes(ptr, size), data)
size = int64(n)
}
ctx.c.wrp.Xsqlite3_result_blob_go(int32(ctx.handle), int32(ptr), size)
default:
ctx.ResultError(errutil.ErrorString("base64: accepts only blob or text"))
}
})
}
// ExtensionLibrary represents a dynamically linked SQLite extension.
type ExtensionLibrary interface {
Xsqlite3_extension_init(db, _, _ int32) int32
}
// ExtensionInfo returns values needed to load a dynamically linked SQLite extension.
type ExtensionInfo func() (memorySize, memoryAlignment, tableSize, tableAlignment int64)
type extEnv struct {
*env
memoryBase int32
tableBase int32
}
func (e *extEnv) X__memory_base() *int32 { return &e.memoryBase }
func (e *extEnv) X__table_base() *int32 { return &e.tableBase }
// ExtensionInit loads an SQLite extension library.
//
// https://sqlite.org/loadext.html
func ExtensionInit[Env any, Mod ExtensionLibrary](db *Conn, init func(env Env) Mod, info ExtensionInfo) error {
memSize, memAlign, tableSize, tableAlign := info()
var memBase int32
if memSize > 0 {
memBase = db.wrp.Xaligned_alloc(int32(memAlign), int32(memSize))
if memBase == 0 {
panic(errutil.OOMErr)
}
}
var tableBase int
if tableSize > 0 {
// Round up to the alignment.
rnd := int(tableAlign) - 1
tab := db.wrp.X__indirect_function_table()
tableBase = (len(*tab) + rnd) &^ rnd
if add := tableBase + int(tableSize) - len(*tab); add > 0 {
*tab = append(*tab, make([]any, add)...)
}
}
e := &extEnv{
env: &env{db.wrp},
memoryBase: memBase,
tableBase: int32(tableBase),
}
mod := init(any(e).(Env))
if opt, ok := any(mod).(interface{ X__wasm_apply_data_relocs() }); ok {
opt.X__wasm_apply_data_relocs()
}
if opt, ok := any(mod).(interface{ X__wasm_call_ctors() }); ok {
opt.X__wasm_call_ctors()
}
rc := mod.Xsqlite3_extension_init(int32(db.handle), 0, 0)
return db.error(res_t(rc))
}
+14 -4
View File
@@ -17,32 +17,42 @@ you can load into your database connections.
reads [comma-separated values](https://sqlite.org/csv.html).
- [`github.com/ncruces/go-sqlite3/ext/fileio`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/fileio)
reads, writes and lists files.
- [`github.com/ncruces/go-sqlite3/ext/fts5`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/fts5)
provides [full-text search](https://sqlite.org/fts5.html).
- [`github.com/ncruces/go-sqlite3/ext/hash`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/hash)
provides cryptographic hash functions.
- [`github.com/ncruces/go-sqlite3/ext/ipaddr`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/ipaddr)
provides functions to manipulate IPs and CIDRs.
- [`github.com/ncruces/go-sqlite3/ext/lines`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/lines)
reads data [line-by-line](https://github.com/asg017/sqlite-lines).
- [`github.com/ncruces/go-sqlite3/ext/pivot`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/pivot)
creates [pivot tables](https://github.com/jakethaw/pivot_vtab).
- [`github.com/ncruces/go-sqlite3/ext/regexp`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/regexp)
provides regular expression functions.
provides [regular expression](https://github.com/nalgeon/sqlean/blob/main/docs/regexp.md) functions.
- [`github.com/ncruces/go-sqlite3/ext/rtree`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/rtree)
provides [multi-dimensional](https://sqlite.org/rtree.html) and [geospatial](https://sqlite.org/geopoly.html) indexes.
- [`github.com/ncruces/go-sqlite3/ext/serdes`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/serdes)
(de)serializes databases.
- [`github.com/ncruces/go-sqlite3/ext/spellfix1`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/spellfix1)
searches a vocabulary for [close matches](https://sqlite.org/spellfix1.html).
- [`github.com/ncruces/go-sqlite3/ext/statement`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/statement)
creates [parameterized views](https://github.com/0x09/sqlite-statement-vtab).
- [`github.com/ncruces/go-sqlite3/ext/stats`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/stats)
provides [statistics](https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html) functions.
provides [statistics](https://oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html) functions.
- [`github.com/ncruces/go-sqlite3/ext/unicode`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/unicode)
provides [Unicode aware](https://sqlite.org/src/dir/ext/icu) functions.
- [`github.com/ncruces/go-sqlite3/ext/uuid`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/uuid)
generates [UUIDs](https://en.wikipedia.org/wiki/Universally_unique_identifier).
- [`github.com/ncruces/go-sqlite3/ext/vec1`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/vec1)
provides approximate nearest-neighbor [vector search](https://sqlite.org/vec1/doc/trunk/doc/vec1.md).
- [`github.com/ncruces/go-sqlite3/ext/zorder`](https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/zorder)
maps multidimensional data to one dimension.
### Pakages
### Packages
These packages may also be useful to work with SQLite:
- [`github.com/ncruces/decimal`](https://pkg.go.dev/github.com/ncruces/decimal)
decimal arithmetic.
- [`github.com/ncruces/julianday`](https://pkg.go.dev/github.com/ncruces/julianday)
Julian day math.
Julian day math.
+30 -12
View File
@@ -7,8 +7,8 @@ import (
"fmt"
"reflect"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/util"
)
// Register registers the array single-argument, table-valued SQL function.
@@ -45,12 +45,14 @@ func (array) Open() (sqlite3.VTabCursor, error) {
}
type cursor struct {
array reflect.Value
rowID int
value reflect.Value
slice any
rowID int
length int
}
func (c *cursor) EOF() bool {
return c.rowID >= c.array.Len()
return c.rowID >= c.length
}
func (c *cursor) Next() error {
@@ -59,7 +61,8 @@ func (c *cursor) Next() error {
}
func (c *cursor) RowID() (int64, error) {
return int64(c.rowID), nil
// One-based RowID for consistency with carray and other tables.
return int64(c.rowID) + 1, nil
}
func (c *cursor) Column(ctx sqlite3.Context, n int) error {
@@ -67,7 +70,22 @@ func (c *cursor) Column(ctx sqlite3.Context, n int) error {
return nil
}
v := c.array.Index(c.rowID)
switch arr := c.slice.(type) {
case []int:
ctx.ResultInt(arr[c.rowID])
return nil
case []int64:
ctx.ResultInt64(arr[c.rowID])
return nil
case []float64:
ctx.ResultFloat(arr[c.rowID])
return nil
case []string:
ctx.ResultText(arr[c.rowID])
return nil
}
v := c.value.Index(c.rowID)
k := v.Kind()
if k == reflect.Interface {
@@ -109,14 +127,14 @@ func (c *cursor) Column(ctx sqlite3.Context, n int) error {
return nil
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
array := reflect.ValueOf(arg[0].Pointer())
array, err := indexable(array)
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) (err error) {
c.slice = arg[0].Pointer()
c.value = reflect.ValueOf(c.slice)
c.value, err = indexable(c.value)
if err != nil {
return err
}
c.array = array
c.length = c.value.Len()
c.rowID = 0
return nil
}
+18 -11
View File
@@ -1,22 +1,27 @@
package array_test
import (
"errors"
"fmt"
"log"
"math"
"reflect"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/array"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/array"
"github.com/hanzoai/sqlite3/ext/rtree"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example_driver() {
db, err := driver.Open("file:/test.db?vfs=memdb", array.Register)
db, err := driver.Open("file:/test.db?vfs=memdb", func(c *sqlite3.Conn) error {
return errors.Join(
array.Register(c),
rtree.Register(c))
})
if err != nil {
log.Fatal(err)
}
@@ -52,6 +57,7 @@ func Example_driver() {
func Example() {
sqlite3.AutoExtension(array.Register)
sqlite3.AutoExtension(rtree.Register)
db, err := sqlite3.Open(":memory:")
if err != nil {
@@ -88,15 +94,16 @@ func Example() {
func Test_cursor_Column(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, array.Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, array.Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
rows, err := db.Query(`
rows, err := db.QueryContext(ctx, `
SELECT rowid, value FROM array(?)`,
sqlite3.Pointer(&[...]any{nil, true, 1, uint(2), math.Pi, "text", []byte{1, 2, 3}}))
if err != nil {
@@ -129,7 +136,7 @@ func Test_cursor_Column(t *testing.T) {
func Test_array_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+3 -3
View File
@@ -5,8 +5,8 @@ import (
"errors"
"io"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
// Register registers the SQL functions:
@@ -111,7 +111,7 @@ func writeblob(ctx sqlite3.Context, arg ...sqlite3.Value) {
func openblob(ctx sqlite3.Context, arg ...sqlite3.Value) {
if len(arg) < 6 {
ctx.ResultError(util.ErrorString("openblob: wrong number of arguments"))
ctx.ResultError(errutil.ErrorString("openblob: wrong number of arguments"))
return
}
+29 -26
View File
@@ -1,6 +1,7 @@
package blobio_test
import (
"database/sql"
"io"
"log"
"os"
@@ -8,13 +9,12 @@ import (
"strings"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/array"
"github.com/ncruces/go-sqlite3/ext/blobio"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
_ "github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/array"
"github.com/hanzoai/sqlite3/ext/blobio"
"github.com/hanzoai/sqlite3/internal/testcfg"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example() {
@@ -34,7 +34,8 @@ func Example() {
const message = "Hello BLOB!"
// Create the BLOB.
r, err := db.Exec(`INSERT INTO test VALUES (?)`, sqlite3.ZeroBlob(len(message)))
r, err := db.Exec(`INSERT INTO test VALUES (:data)`,
sql.Named("data", sqlite3.ZeroBlob(len(message))))
if err != nil {
log.Fatal(err)
}
@@ -45,15 +46,19 @@ func Example() {
}
// Write the BLOB.
_, err = db.Exec(`SELECT writeblob('main', 'test', 'col', ?, 0, ?)`,
id, message)
_, err = db.Exec(`SELECT writeblob('main', 'test', 'col', :rowid, :offset, :message)`,
sql.Named("rowid", id),
sql.Named("offset", 0),
sql.Named("message", message))
if err != nil {
log.Fatal(err)
}
// Read the BLOB.
_, err = db.Exec(`SELECT readblob('main', 'test', 'col', ?, 0, ?)`,
id, sqlite3.Pointer(os.Stdout))
_, err = db.Exec(`SELECT readblob('main', 'test', 'col', :rowid, :offset, :writer)`,
sql.Named("rowid", id),
sql.Named("offset", 0),
sql.Named("writer", sqlite3.Pointer(os.Stdout)))
if err != nil {
log.Fatal(err)
}
@@ -64,13 +69,13 @@ func Example() {
func TestMain(m *testing.M) {
sqlite3.AutoExtension(blobio.Register)
sqlite3.AutoExtension(array.Register)
m.Run()
os.Exit(m.Run())
}
func Test_readblob(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -138,18 +143,16 @@ func Test_readblob(t *testing.T) {
}
}
if stmt.Step() {
got := stmt.ColumnText(0)
if got != tt.want1 {
t.Errorf("got %q", got)
}
if !stmt.Step() {
t.Fatal(stmt.Err())
} else if got := stmt.ColumnText(0); got != tt.want1 {
t.Errorf("got %q", got)
}
if stmt.Step() {
got := stmt.ColumnText(0)
if got != tt.want2 {
t.Errorf("got %q", got)
}
if !stmt.Step() {
t.Fatal(stmt.Err())
} else if got := stmt.ColumnText(0); got != tt.want2 {
t.Errorf("got %q", got)
}
err = stmt.Err()
@@ -163,7 +166,7 @@ func Test_readblob(t *testing.T) {
func Test_writeblob(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -218,7 +221,7 @@ func Test_writeblob(t *testing.T) {
func Test_openblob(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+25 -24
View File
@@ -14,8 +14,9 @@ import (
"github.com/dchest/siphash"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/util/sql3util"
)
// Register registers the bloom_filter virtual table:
@@ -34,6 +35,8 @@ type bloom struct {
hashes int
}
const vtab = `CREATE TABLE x(present, word TEXT HIDDEN NOT NULL PRIMARY KEY) WITHOUT ROWID`
func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom, err error) {
b := bloom{
db: db,
@@ -48,19 +51,17 @@ func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom,
return nil, err
}
if nelem <= 0 {
return nil, util.ErrorString("bloom: number of elements in filter must be positive")
return nil, errutil.ErrorString("bloom: number of elements in filter must be positive")
}
} else {
nelem = 100
}
if len(arg) > 1 {
b.prob, err = strconv.ParseFloat(arg[1], 64)
if err != nil {
return nil, err
}
if b.prob <= 0 || b.prob >= 1 {
return nil, util.ErrorString("bloom: probability must be in the range (0,1)")
var ok bool
b.prob, ok = sql3util.ParseFloat(arg[1])
if !ok || b.prob <= 0 || b.prob >= 1 {
return nil, errutil.ErrorString("bloom: probability must be in the range (0,1)")
}
} else {
b.prob = 0.01
@@ -72,7 +73,7 @@ func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom,
return nil, err
}
if b.hashes <= 0 {
return nil, util.ErrorString("bloom: number of hash functions must be positive")
return nil, errutil.ErrorString("bloom: number of hash functions must be positive")
}
} else {
b.hashes = max(1, numHashes(b.prob))
@@ -80,8 +81,7 @@ func create(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom,
b.bytes = numBytes(nelem, b.prob)
err = db.DeclareVTab(
`CREATE TABLE x(present, word HIDDEN NOT NULL PRIMARY KEY) WITHOUT ROWID`)
err = db.DeclareVTab(vtab)
if err != nil {
return nil, err
}
@@ -115,15 +115,15 @@ func connect(db *sqlite3.Conn, _, schema, table string, arg ...string) (_ *bloom
storage: table + "_storage",
}
err = db.DeclareVTab(
`CREATE TABLE x(present, word HIDDEN NOT NULL PRIMARY KEY) WITHOUT ROWID`)
err = db.DeclareVTab(vtab)
if err != nil {
return nil, err
}
load, _, err := db.Prepare(fmt.Sprintf(
load, _, err := db.PrepareFlags(fmt.Sprintf(
`SELECT m/8, p, k FROM %s.%s WHERE rowid = 1`,
sqlite3.QuoteIdentifier(b.schema), sqlite3.QuoteIdentifier(b.storage)))
sqlite3.QuoteIdentifier(b.schema), sqlite3.QuoteIdentifier(b.storage)),
sqlite3.PREPARE_DONT_LOG)
if err != nil {
return nil, err
}
@@ -166,15 +166,16 @@ func (t *bloom) ShadowTables() {
}
func (t *bloom) Integrity(schema, table string, flags int) error {
load, _, err := t.db.Prepare(fmt.Sprintf(
load, _, err := t.db.PrepareFlags(fmt.Sprintf(
`SELECT typeof(data), length(data), p, n, m, k FROM %s.%s WHERE rowid = 1`,
sqlite3.QuoteIdentifier(t.schema), sqlite3.QuoteIdentifier(t.storage)))
sqlite3.QuoteIdentifier(t.schema), sqlite3.QuoteIdentifier(t.storage)),
sqlite3.PREPARE_DONT_LOG)
if err != nil {
return fmt.Errorf("bloom: %v", err) // can't wrap!
}
defer load.Close()
err = util.ErrorString("bloom: invalid parameters")
err = errutil.ErrorString("bloom: invalid parameters")
if !load.Step() {
return err
}
@@ -216,9 +217,9 @@ func (b *bloom) BestIndex(idx *sqlite3.IndexInfo) error {
func (b *bloom) Update(arg ...sqlite3.Value) (rowid int64, err error) {
if arg[0].Type() != sqlite3.NULL {
if len(arg) == 1 {
return 0, util.ErrorString("bloom: elements cannot be deleted")
return 0, errutil.ErrorString("bloom: elements cannot be deleted")
}
return 0, util.ErrorString("bloom: elements cannot be updated")
return 0, errutil.ErrorString("bloom: elements cannot be updated")
}
if arg[2].NoChange() {
@@ -268,13 +269,13 @@ func (b *bloom) Open() (sqlite3.VTabCursor, error) {
type cursor struct {
*bloom
arg *sqlite3.Value
arg sqlite3.Value
eof bool
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
c.eof = false
c.arg = &arg[0]
c.arg = arg[0]
blob := arg[0].RawBlob()
f, err := c.db.OpenBlob(c.schema, c.storage, "data", 1, false)
@@ -312,7 +313,7 @@ func (c *cursor) Column(ctx sqlite3.Context, n int) error {
case 0:
ctx.ResultBool(true)
case 1:
ctx.ResultValue(*c.arg)
ctx.ResultValue(c.arg)
}
return nil
}
+7 -8
View File
@@ -6,21 +6,20 @@ import (
"path/filepath"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/bloom"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/ext/bloom"
"github.com/hanzoai/sqlite3/internal/testcfg"
)
func TestMain(m *testing.M) {
sqlite3.AutoExtension(bloom.Register)
m.Run()
os.Exit(m.Run())
}
func TestRegister(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -102,7 +101,7 @@ func Test_compatible(t *testing.T) {
t.Fatal(err)
}
db, err := sqlite3.Open("file:" + filepath.ToSlash(tmp) + "?nolock=1")
db, err := sqlite3.OpenContext(testcfg.Context(t), "file:"+filepath.ToSlash(tmp)+"?nolock=1")
if err != nil {
t.Fatal(err)
}
@@ -158,7 +157,7 @@ func Test_compatible(t *testing.T) {
func Test_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+6 -5
View File
@@ -10,9 +10,9 @@ import (
"fmt"
"math"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/sql3util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/util"
"github.com/hanzoai/sqlite3/util/sql3util"
)
const (
@@ -56,7 +56,7 @@ func Register(db *sqlite3.Conn) error {
done.Add(key)
}
err := db.DeclareVTab(`CREATE TABLE x(id,depth,root HIDDEN,tablename HIDDEN,idcolumn HIDDEN,parentcolumn HIDDEN)`)
err := db.DeclareVTab(`CREATE TABLE x(id INT,depth INT,root HIDDEN,tablename TEXT HIDDEN,idcolumn TEXT HIDDEN,parentcolumn TEXT HIDDEN)`)
if err != nil {
return nil, err
}
@@ -154,6 +154,7 @@ func (c *closure) BestIndex(idx *sqlite3.IndexInfo) error {
return sqlite3.CONSTRAINT
}
idx.IdxFlags = sqlite3.INDEX_SCAN_HEX
idx.EstimatedCost = cost
idx.IdxNum = plan
return nil
@@ -201,7 +202,7 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
sqlite3.QuoteIdentifier(column),
sqlite3.QuoteIdentifier(parent),
)
stmt, _, err := c.db.Prepare(sql)
stmt, _, err := c.db.PrepareFlags(sql, sqlite3.PREPARE_DONT_LOG)
if err != nil {
return err
}
+7 -7
View File
@@ -4,17 +4,17 @@ import (
_ "embed"
"fmt"
"log"
"os"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/closure"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/ext/closure"
"github.com/hanzoai/sqlite3/internal/testcfg"
)
func TestMain(m *testing.M) {
sqlite3.AutoExtension(closure.Register)
m.Run()
os.Exit(m.Run())
}
func Example() {
@@ -88,7 +88,7 @@ func Example() {
func TestRegister(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -156,7 +156,7 @@ func TestRegister(t *testing.T) {
func Test_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"strconv"
"github.com/ncruces/go-sqlite3/util/sql3util"
"github.com/hanzoai/sqlite3/util/sql3util"
)
func uintArg(key, val string) (int, error) {
+1 -1
View File
@@ -3,7 +3,7 @@ package csv
import (
"testing"
"github.com/ncruces/go-sqlite3/util/sql3util"
"github.com/hanzoai/sqlite3/util/sql3util"
)
func Test_uintArg(t *testing.T) {
+22 -27
View File
@@ -15,10 +15,11 @@ import (
"strconv"
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/osutil"
"github.com/ncruces/go-sqlite3/util/sql3util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/internal/util"
"github.com/hanzoai/sqlite3/util/osutil"
"github.com/hanzoai/sqlite3/util/sql3util"
)
// Register registers the CSV virtual table.
@@ -73,7 +74,7 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
}
if (filename == "") == (data == "") {
return nil, util.ErrorString(`csv: must specify either "filename" or "data" but not both`)
return nil, errutil.ErrorString(`csv: must specify either "filename" or "data" but not both`)
}
t := &table{
@@ -85,7 +86,8 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
header: header,
}
if schema == "" {
hadSchema := schema != ""
if !hadSchema {
var row []string
if header || columns < 0 {
csv, c, err := t.newReader()
@@ -99,17 +101,14 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
}
}
schema = getSchema(header, columns, row)
} else {
t.typs, err = getColumnAffinities(schema)
if err != nil {
return nil, err
}
}
err = db.DeclareVTab(schema)
if err == nil {
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
}
if err == nil && hadSchema {
t.typs, err = getColumnAffinities(schema)
}
if err != nil {
return nil, err
}
@@ -123,7 +122,7 @@ type table struct {
fsys fs.FS
name string
data string
typs []affinity
typs []sql3util.Affinity
comma rune
comment rune
header bool
@@ -242,31 +241,27 @@ func (c *cursor) RowID() (int64, error) {
func (c *cursor) Column(ctx sqlite3.Context, col int) error {
if col < len(c.row) {
typ := text
typ := sql3util.TEXT
if col < len(c.table.typs) {
typ = c.table.typs[col]
}
txt := c.row[col]
if txt == "" && typ != text {
if txt == "" && typ != sql3util.TEXT {
return nil
}
switch typ {
case numeric, integer:
if strings.TrimLeft(txt, "+-0123456789") == "" {
if i, err := strconv.ParseInt(txt, 10, 64); err == nil {
ctx.ResultInt64(i)
return nil
}
case sql3util.NUMERIC, sql3util.INTEGER:
if i, err := strconv.ParseInt(txt, 10, 64); err == nil {
ctx.ResultInt64(i)
return nil
}
fallthrough
case real:
if strings.TrimLeft(txt, "+-.0123456789Ee") == "" {
if f, err := strconv.ParseFloat(txt, 64); err == nil {
ctx.ResultFloat(f)
return nil
}
case sql3util.REAL:
if f, ok := sql3util.ParseFloat(txt); ok {
ctx.ResultFloat(f)
return nil
}
fallthrough
default:
+21 -20
View File
@@ -3,12 +3,12 @@ package csv_test
import (
"fmt"
"log"
"os"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/csv"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/ext/csv"
"github.com/hanzoai/sqlite3/internal/testcfg"
)
func Example() {
@@ -56,13 +56,13 @@ func Example() {
func TestMain(m *testing.M) {
sqlite3.AutoExtension(csv.Register)
m.Run()
os.Exit(m.Run())
}
func TestRegister(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -124,7 +124,7 @@ Robert "Griesemer" "gri"`
func TestAffinity(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -146,27 +146,28 @@ func TestAffinity(t *testing.T) {
}
defer stmt.Close()
if stmt.Step() {
if got := stmt.ColumnText(0); got != "1" {
t.Errorf("got %q want 1", got)
}
if !stmt.Step() {
t.Fatal(stmt.Err())
} else if got := stmt.ColumnText(0); got != "1" {
t.Errorf("got %q want 1", got)
}
if stmt.Step() {
if got := stmt.ColumnText(0); got != "0.1" {
t.Errorf("got %q want 0.1", got)
}
if !stmt.Step() {
t.Fatal(stmt.Err())
} else if got := stmt.ColumnText(0); got != "0.1" {
t.Errorf("got %q want 0.1", got)
}
if stmt.Step() {
if got := stmt.ColumnText(0); got != "e" {
t.Errorf("got %q want e", got)
}
if !stmt.Step() {
t.Fatal(stmt.Err())
} else if got := stmt.ColumnText(0); got != "e" {
t.Errorf("got %q want e", got)
}
}
func TestRegister_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+14 -14
View File
@@ -4,36 +4,36 @@ import (
"strconv"
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/hanzoai/sqlite3"
)
func getSchema(header bool, columns int, row []string) string {
var sep string
var str strings.Builder
str.WriteString("CREATE TABLE x(")
var buf strings.Builder
buf.WriteString("CREATE TABLE x(")
if 0 <= columns && columns < len(row) {
row = row[:columns]
}
for i, f := range row {
str.WriteString(sep)
buf.WriteString(sep)
if header && f != "" {
str.WriteString(sqlite3.QuoteIdentifier(f))
buf.WriteString(sqlite3.QuoteIdentifier(f))
} else {
str.WriteString("c")
str.WriteString(strconv.Itoa(i + 1))
buf.WriteString("c")
buf.WriteString(strconv.Itoa(i + 1))
}
str.WriteString(" TEXT")
buf.WriteString(" TEXT")
sep = ","
}
for i := len(row); i < columns; i++ {
str.WriteString(sep)
str.WriteString("c")
str.WriteString(strconv.Itoa(i + 1))
str.WriteString(" TEXT")
buf.WriteString(sep)
buf.WriteString("c")
buf.WriteString(strconv.Itoa(i + 1))
buf.WriteString(" TEXT")
sep = ","
}
str.WriteByte(')')
buf.WriteByte(')')
return str.String()
return buf.String()
}
+4 -39
View File
@@ -1,52 +1,17 @@
package csv
import (
"strings"
import "github.com/hanzoai/sqlite3/util/sql3util"
"github.com/ncruces/go-sqlite3/util/sql3util"
)
type affinity byte
const (
blob affinity = 0
text affinity = 1
numeric affinity = 2
integer affinity = 3
real affinity = 4
)
func getColumnAffinities(schema string) ([]affinity, error) {
func getColumnAffinities(schema string) ([]sql3util.Affinity, error) {
tab, err := sql3util.ParseTable(schema)
if err != nil {
return nil, err
}
columns := tab.Columns
types := make([]affinity, len(columns))
types := make([]sql3util.Affinity, len(columns))
for i, col := range columns {
types[i] = getAffinity(col.Type)
types[i] = sql3util.GetAffinity(col.Type)
}
return types, nil
}
func getAffinity(declType string) affinity {
// https://sqlite.org/datatype3.html#determination_of_column_affinity
if declType == "" {
return blob
}
name := strings.ToUpper(declType)
if strings.Contains(name, "INT") {
return integer
}
if strings.Contains(name, "CHAR") || strings.Contains(name, "CLOB") || strings.Contains(name, "TEXT") {
return text
}
if strings.Contains(name, "BLOB") {
return blob
}
if strings.Contains(name, "REAL") || strings.Contains(name, "FLOA") || strings.Contains(name, "DOUB") {
return real
}
return numeric
}
-32
View File
@@ -1,32 +0,0 @@
package csv
import "testing"
func Test_getAffinity(t *testing.T) {
tests := []struct {
decl string
want affinity
}{
{"", blob},
{"INTEGER", integer},
{"TINYINT", integer},
{"TEXT", text},
{"CHAR", text},
{"CLOB", text},
{"BLOB", blob},
{"REAL", real},
{"FLOAT", real},
{"DOUBLE", real},
{"NUMERIC", numeric},
{"DECIMAL", numeric},
{"BOOLEAN", numeric},
{"DATETIME", numeric},
}
for _, tt := range tests {
t.Run(tt.decl, func(t *testing.T) {
if got := getAffinity(tt.decl); got != tt.want {
t.Errorf("getAffinity() = %v, want %v", got, tt.want)
}
})
}
}
-70
View File
@@ -1,70 +0,0 @@
//go:build !go1.23
package fileio
import (
"fmt"
"github.com/ncruces/go-sqlite3/internal/util"
)
// Adapted from: https://research.swtch.com/coro
const errCoroCanceled = util.ErrorString("coroutine canceled")
func coroNew[In, Out any](f func(In, func(Out) In) Out) (resume func(In) (Out, bool), cancel func()) {
type msg[T any] struct {
panic any
val T
}
cin := make(chan msg[In])
cout := make(chan msg[Out])
running := true
resume = func(in In) (out Out, ok bool) {
if !running {
return
}
cin <- msg[In]{val: in}
m := <-cout
if m.panic != nil {
panic(m.panic)
}
return m.val, running
}
cancel = func() {
if !running {
return
}
e := fmt.Errorf("%w", errCoroCanceled)
cin <- msg[In]{panic: e}
m := <-cout
if m.panic != nil && m.panic != e {
panic(m.panic)
}
}
yield := func(out Out) In {
cout <- msg[Out]{val: out}
m := <-cin
if m.panic != nil {
panic(m.panic)
}
return m.val
}
go func() {
defer func() {
if running {
running = false
cout <- msg[Out]{panic: recover()}
}
}()
var out Out
m := <-cin
if m.panic == nil {
out = f(m.val, yield)
}
running = false
cout <- msg[Out]{val: out}
}()
return resume, cancel
}
+4 -4
View File
@@ -9,7 +9,7 @@ import (
"io/fs"
"os"
"github.com/ncruces/go-sqlite3"
"github.com/hanzoai/sqlite3"
)
// Register registers SQL functions readfile, writefile, lsmode,
@@ -18,7 +18,7 @@ func Register(db *sqlite3.Conn) error {
return RegisterFS(db, nil)
}
// Register registers SQL functions readfile, lsmode,
// RegisterFS registers SQL functions readfile, lsmode,
// and the table-valued function fsdir;
// fsys will be used to read files and list directories.
func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
@@ -30,7 +30,7 @@ func RegisterFS(db *sqlite3.Conn, fsys fs.FS) error {
db.CreateFunction("readfile", 1, sqlite3.DIRECTONLY, readfile(fsys)),
db.CreateFunction("lsmode", 1, sqlite3.DETERMINISTIC, lsmode),
sqlite3.CreateModule(db, "fsdir", nil, func(db *sqlite3.Conn, _, _, _ string, _ ...string) (fsdir, error) {
err := db.DeclareVTab(`CREATE TABLE x(name,mode,mtime TIMESTAMP,data,path HIDDEN,dir HIDDEN)`)
err := db.DeclareVTab(`CREATE TABLE x(name TEXT,mode INT,mtime TIMESTAMP,data BLOB,level INT,path HIDDEN,dir HIDDEN)`)
if err == nil {
err = db.VTabConfig(sqlite3.VTAB_DIRECTONLY)
}
@@ -42,7 +42,7 @@ func lsmode(ctx sqlite3.Context, arg ...sqlite3.Value) {
ctx.ResultText(fs.FileMode(arg[0].Int()).String())
}
func readfile(fsys fs.FS) func(ctx sqlite3.Context, arg ...sqlite3.Value) {
func readfile(fsys fs.FS) sqlite3.ScalarFunction {
return func(ctx sqlite3.Context, arg ...sqlite3.Value) {
var err error
var data []byte
+13 -12
View File
@@ -7,19 +7,19 @@ import (
"os"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/fileio"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/fileio"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Test_lsmode(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, fileio.Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, fileio.Register)
if err != nil {
t.Fatal(err)
}
@@ -36,7 +36,7 @@ func Test_lsmode(t *testing.T) {
}
var mode string
err = db.QueryRow(`SELECT lsmode(?)`, s.Mode()).Scan(&mode)
err = db.QueryRowContext(ctx, `SELECT lsmode(?)`, s.Mode()).Scan(&mode)
if err != nil {
t.Fatal(err)
}
@@ -53,9 +53,10 @@ func Test_readfile(t *testing.T) {
for _, fsys := range []fs.FS{nil, os.DirFS(".")} {
t.Run("", func(t *testing.T) {
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, func(c *sqlite3.Conn) error {
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, func(c *sqlite3.Conn) error {
fileio.RegisterFS(c, fsys)
return nil
})
@@ -64,7 +65,7 @@ func Test_readfile(t *testing.T) {
}
defer db.Close()
rows, err := db.Query(`SELECT readfile('fileio_test.go')`)
rows, err := db.QueryContext(ctx, `SELECT readfile('fileio_test.go')`)
if err != nil {
t.Fatal(err)
}
+109 -33
View File
@@ -2,27 +2,33 @@ package fileio
import (
"io/fs"
"iter"
"os"
"path"
"path/filepath"
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/hanzoai/sqlite3"
)
const (
_COL_NAME = 0
_COL_MODE = 1
_COL_TIME = 2
_COL_DATA = 3
_COL_ROOT = 4
_COL_BASE = 5
_COL_NAME = iota
_COL_MODE
_COL_MTIME
_COL_DATA
_COL_LEVEL
_COL_ROOT
_COL_BASE
)
type fsdir struct{ fsys fs.FS }
func (d fsdir) BestIndex(idx *sqlite3.IndexInfo) error {
var root, base bool
var levelOp sqlite3.IndexConstraintOp
var root bool
level := -1
base := -1
for i, cst := range idx.Constraint {
switch cst.Column {
case _COL_ROOT:
@@ -38,20 +44,44 @@ func (d fsdir) BestIndex(idx *sqlite3.IndexInfo) error {
if !cst.Usable || cst.Op != sqlite3.INDEX_CONSTRAINT_EQ {
return sqlite3.CONSTRAINT
}
idx.ConstraintUsage[i] = sqlite3.IndexConstraintUsage{
Omit: true,
ArgvIndex: 2,
base = i
case _COL_LEVEL:
if !cst.Usable {
break
}
switch cst.Op {
case sqlite3.INDEX_CONSTRAINT_EQ, sqlite3.INDEX_CONSTRAINT_LE, sqlite3.INDEX_CONSTRAINT_LT:
levelOp = cst.Op
level = i
}
base = true
}
}
if !root {
return sqlite3.CONSTRAINT
}
if base {
idx.EstimatedCost = 10
} else {
idx.EstimatedCost = 100
args := 2
idx.IdxNum = 0
idx.EstimatedCost = 1e9
if base >= 0 {
idx.IdxNum |= 1
idx.EstimatedCost /= 1e4
idx.ConstraintUsage[base] = sqlite3.IndexConstraintUsage{
Omit: true,
ArgvIndex: args,
}
args++
}
if level >= 0 {
idx.EstimatedCost /= 1e4
idx.IdxNum |= (args - 1) << 1
if levelOp == sqlite3.INDEX_CONSTRAINT_LT {
idx.IdxNum |= 1 << 3
}
idx.ConstraintUsage[level] = sqlite3.IndexConstraintUsage{
Omit: levelOp != sqlite3.INDEX_CONSTRAINT_EQ,
ArgvIndex: args,
}
}
return nil
}
@@ -62,23 +92,25 @@ func (d fsdir) Open() (sqlite3.VTabCursor, error) {
type cursor struct {
fsdir
base string
resume resume
cancel func()
curr entry
eof bool
rowID int64
base string
next func() (entry, bool)
stop func()
curr entry
eof bool
rowID int64
maxLevel int
}
type entry struct {
fs.DirEntry
err error
path string
err error
path string
level int
}
func (c *cursor) Close() error {
if c.cancel != nil {
c.cancel()
if c.stop != nil {
c.stop()
}
return nil
}
@@ -89,26 +121,54 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
}
root := arg[0].Text()
if len(arg) > 1 {
base := arg[1].Text()
if i := idxNum & 1; i > 0 {
base := arg[i].Text()
if c.fsys != nil {
root = path.Join(base, root)
base = path.Clean(base) + "/"
} else {
root = filepath.Join(base, root)
base = filepath.Clean(base) + string(filepath.Separator)
}
root = base + root
c.base = base
}
if i := idxNum >> 1; i > 0 {
c.maxLevel = arg[i&3].Int() - i>>2
}
c.resume, c.cancel = pull(c, root)
c.next, c.stop = iter.Pull(func(yield func(entry) bool) {
var stack []string
walkDir := func(p string, d fs.DirEntry, err error) error {
level := len(stack)
for level > 1 && !strings.HasPrefix(c.dir(p), stack[level-1]) {
level--
}
stack = stack[:level]
level++
if !yield(entry{d, err, p, level}) {
return fs.SkipAll
}
if d != nil && d.IsDir() {
if 0 < c.maxLevel && c.maxLevel <= level {
return fs.SkipDir
}
stack = append(stack, p)
}
return nil
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
})
c.eof = false
c.rowID = 0
return c.Next()
}
func (c *cursor) Next() error {
curr, ok := next(c)
curr, ok := c.next()
c.curr = curr
c.eof = !ok
c.rowID++
@@ -136,7 +196,7 @@ func (c *cursor) Column(ctx sqlite3.Context, n int) error {
}
ctx.ResultInt64(int64(i.Mode()))
case _COL_TIME:
case _COL_MTIME:
i, err := c.curr.Info()
if err != nil {
return err
@@ -165,6 +225,22 @@ func (c *cursor) Column(ctx sqlite3.Context, n int) error {
}
ctx.ResultText(t)
}
case _COL_LEVEL:
ctx.ResultInt(c.curr.level)
}
return nil
}
func (c *cursor) dir(p string) string {
var dir string
if c.fsys != nil {
dir, _ = path.Split(p)
} else {
dir, _ = filepath.Split(p)
}
if dir == "" {
dir = "."
}
return dir
}
-29
View File
@@ -1,29 +0,0 @@
//go:build !go1.23
package fileio
import (
"io/fs"
"path/filepath"
)
type resume = func(struct{}) (entry, bool)
func next(c *cursor) (entry, bool) {
return c.resume(struct{}{})
}
func pull(c *cursor, root string) (resume, func()) {
return coroNew(func(_ struct{}, yield func(entry) struct{}) entry {
walkDir := func(path string, d fs.DirEntry, err error) error {
yield(entry{d, err, path})
return nil
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
return entry{}
})
}
-31
View File
@@ -1,31 +0,0 @@
//go:build go1.23
package fileio
import (
"io/fs"
"iter"
"path/filepath"
)
type resume = func() (entry, bool)
func next(c *cursor) (entry, bool) {
return c.resume()
}
func pull(c *cursor, root string) (resume, func()) {
return iter.Pull(func(yield func(entry) bool) {
walkDir := func(path string, d fs.DirEntry, err error) error {
if yield(entry{d, err, path}) {
return nil
}
return fs.SkipAll
}
if c.fsys != nil {
fs.WalkDir(c.fsys, root, walkDir)
} else {
filepath.WalkDir(root, walkDir)
}
})
}
+15 -11
View File
@@ -8,12 +8,11 @@ import (
"testing"
"time"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/fileio"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/fileio"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Test_fsdir(t *testing.T) {
@@ -21,9 +20,10 @@ func Test_fsdir(t *testing.T) {
for _, fsys := range []fs.FS{nil, os.DirFS(".")} {
t.Run("", func(t *testing.T) {
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, func(c *sqlite3.Conn) error {
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, func(c *sqlite3.Conn) error {
fileio.RegisterFS(c, fsys)
return nil
})
@@ -32,7 +32,7 @@ func Test_fsdir(t *testing.T) {
}
defer db.Close()
rows, err := db.Query(`SELECT * FROM fsdir('.', '.')`)
rows, err := db.QueryContext(ctx, `SELECT * FROM fsdir('.') WHERE level <= 2`)
if err != nil {
t.Fatal(err)
}
@@ -42,7 +42,8 @@ func Test_fsdir(t *testing.T) {
var mode fs.FileMode
var mtime time.Time
var data sql.RawBytes
err := rows.Scan(&name, &mode, sqlite3.TimeFormatUnixFrac.Scanner(&mtime), &data)
var level int
err := rows.Scan(&name, &mode, sqlite3.TimeFormatUnixFrac.Scanner(&mtime), &data, &level)
if err != nil {
t.Fatal(err)
}
@@ -57,6 +58,9 @@ func Test_fsdir(t *testing.T) {
t.Errorf("got: %s", data[:min(64, len(data))])
}
}
if level > 2 {
t.Errorf("got: %v", level)
}
}
})
}
@@ -65,7 +69,7 @@ func Test_fsdir(t *testing.T) {
func Test_fsdir_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+4 -4
View File
@@ -8,14 +8,14 @@ import (
"path/filepath"
"time"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/ncruces/go-sqlite3/util/fsutil"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
"github.com/hanzoai/sqlite3/util/fsutil"
)
func writefile(ctx sqlite3.Context, arg ...sqlite3.Value) {
if len(arg) < 2 || len(arg) > 4 {
ctx.ResultError(util.ErrorString("writefile: wrong number of arguments"))
ctx.ResultError(errutil.ErrorString("writefile: wrong number of arguments"))
return
}
+17 -14
View File
@@ -1,3 +1,5 @@
//go:build !js
package fileio
import (
@@ -7,17 +9,17 @@ import (
"testing"
"time"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Test_writefile(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, Register)
if err != nil {
t.Fatal(err)
}
@@ -30,22 +32,22 @@ func Test_writefile(t *testing.T) {
sock := filepath.Join(dir, "sock")
twosday := time.Date(2022, 2, 22, 22, 22, 22, 0, time.UTC)
_, err = db.Exec(`SELECT writefile(?, 'Hello world!')`, file)
_, err = db.ExecContext(ctx, `SELECT writefile(?, 'Hello world!')`, file)
if err != nil {
t.Fatal(err)
}
_, err = db.Exec(`SELECT writefile(?, ?, ?)`, link, "test.txt", fs.ModeSymlink)
_, err = db.ExecContext(ctx, `SELECT writefile(?, ?, ?)`, link, "test.txt", fs.ModeSymlink)
if err != nil {
t.Fatal(err)
}
_, err = db.Exec(`SELECT writefile(?, ?, ?, ?)`, dir, nil, 0040700, twosday.Unix())
_, err = db.ExecContext(ctx, `SELECT writefile(?, ?, ?, ?)`, dir, nil, 0040700, twosday.Unix())
if err != nil {
t.Fatal(err)
}
rows, err := db.Query(`SELECT * FROM fsdir('.', ?)`, dir)
rows, err := db.QueryContext(ctx, `SELECT * FROM fsdir('.', ?)`, dir)
if err != nil {
t.Fatal(err)
}
@@ -55,7 +57,8 @@ func Test_writefile(t *testing.T) {
var mode fs.FileMode
var mtime time.Time
var data sql.NullString
err := rows.Scan(&name, &mode, &mtime, &data)
var level int
err := rows.Scan(&name, &mode, &mtime, &data, &level)
if err != nil {
t.Fatal(err)
}
@@ -70,19 +73,19 @@ func Test_writefile(t *testing.T) {
}
}
_, err = db.Exec(`SELECT writefile(?, 'Hello world!')`, nest)
_, err = db.ExecContext(ctx, `SELECT writefile(?, 'Hello world!')`, nest)
if err != nil {
t.Fatal(err)
}
_, err = db.Exec(`SELECT writefile(?, ?, ?)`, sock, nil, fs.ModeSocket)
_, err = db.ExecContext(ctx, `SELECT writefile(?, ?, ?)`, sock, nil, fs.ModeSocket)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
_, err = db.Exec(`SELECT writefile()`)
_, err = db.ExecContext(ctx, `SELECT writefile()`)
if err == nil {
t.Fatal("want error")
} else {
+14
View File
@@ -0,0 +1,14 @@
// Package fts5 provides the fts5 extension.
//
// https://sqlite.org/fts5.html
package fts5
import (
"github.com/hanzoai/sqlite3"
"github.com/ncruces/go-sqlite3-wasm/v3/fts5"
)
// Register registers the fts5 extension.
func Register(db *sqlite3.Conn) error {
return sqlite3.ExtensionInit(db, fts5.New, fts5.DylinkInfo)
}
+37
View File
@@ -0,0 +1,37 @@
package fts5_test
import (
"fmt"
"log"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/fts5"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example() {
db, err := driver.Open("file:/test.db?vfs=memdb", fts5.Register)
if err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
CREATE VIRTUAL TABLE docs USING fts5(title, body);
INSERT INTO docs(title, body) VALUES
('Go Programming', 'An intensive guide to Go routines.'),
('SQLite Tutorial', 'Learn how to use virtual tables efficiently.');
`)
if err != nil {
log.Fatal(err)
}
var title string
err = db.QueryRow("SELECT title FROM docs WHERE docs MATCH 'Go AND routines'").Scan(&title)
if err != nil {
log.Fatal(err)
}
fmt.Println(title)
// Output: Go Programming
}
+3 -3
View File
@@ -3,8 +3,8 @@ package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
func blake2sFunc(ctx sqlite3.Context, arg ...sqlite3.Value) {
@@ -25,6 +25,6 @@ func blake2bFunc(ctx sqlite3.Context, arg ...sqlite3.Value) {
case 512:
hashFunc(ctx, arg[0], crypto.BLAKE2b_512)
default:
ctx.ResultError(util.ErrorString("blake2b: size must be 256, 384, 512"))
ctx.ResultError(errutil.ErrorString("blake2b: size must be 256, 384, 512"))
}
}
+3 -3
View File
@@ -23,15 +23,15 @@ import (
"crypto"
"errors"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
// Register registers cryptographic hash functions for a database connection.
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
var errs util.ErrorJoiner
var errs errutil.ErrorJoiner
if crypto.MD4.Available() {
errs.Join(
db.CreateFunction("md4", 1, flags, md4Func))
+12 -12
View File
@@ -4,6 +4,7 @@ import (
_ "crypto/md5"
_ "crypto/sha1"
_ "crypto/sha256"
_ "crypto/sha3"
_ "crypto/sha512"
"testing"
@@ -11,17 +12,15 @@ import (
_ "golang.org/x/crypto/blake2s"
_ "golang.org/x/crypto/md4"
_ "golang.org/x/crypto/ripemd160"
_ "golang.org/x/crypto/sha3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func TestRegister(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
tests := []struct {
name string
@@ -55,7 +54,8 @@ func TestRegister(t *testing.T) {
{"blake2b('', 256)", "0E5751C026E543B2E8AB2EB06099DAA1D1E5DF47778F7787FAAB45CDF12FE3A8"},
}
db, err := driver.Open(tmp, Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, Register)
if err != nil {
t.Fatal(err)
}
@@ -65,7 +65,7 @@ func TestRegister(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
var hash string
err = db.QueryRow(`SELECT hex(` + tt.name + `)`).Scan(&hash)
err = db.QueryRowContext(ctx, `SELECT hex(`+tt.name+`)`).Scan(&hash)
if err != nil {
t.Fatal(err)
}
@@ -76,22 +76,22 @@ func TestRegister(t *testing.T) {
})
}
_, err = db.Exec(`SELECT sha256('', 255)`)
_, err = db.ExecContext(ctx, `SELECT sha256('', 255)`)
if err == nil {
t.Error("want error")
}
_, err = db.Exec(`SELECT sha512('', 255)`)
_, err = db.ExecContext(ctx, `SELECT sha512('', 255)`)
if err == nil {
t.Error("want error")
}
_, err = db.Exec(`SELECT sha3('', 255)`)
_, err = db.ExecContext(ctx, `SELECT sha3('', 255)`)
if err == nil {
t.Error("want error")
}
_, err = db.Exec(`SELECT blake2b('', 255)`)
_, err = db.ExecContext(ctx, `SELECT blake2b('', 255)`)
if err == nil {
t.Error("want error")
}
+4 -4
View File
@@ -3,8 +3,8 @@ package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
func sha224Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
@@ -27,7 +27,7 @@ func sha256Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
case 256:
hashFunc(ctx, arg[0], crypto.SHA256)
default:
ctx.ResultError(util.ErrorString("sha256: size must be 224, 256"))
ctx.ResultError(errutil.ErrorString("sha256: size must be 224, 256"))
}
}
@@ -47,7 +47,7 @@ func sha512Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
case 512:
hashFunc(ctx, arg[0], crypto.SHA512)
default:
ctx.ResultError(util.ErrorString("sha512: size must be 224, 256, 384, 512"))
ctx.ResultError(errutil.ErrorString("sha512: size must be 224, 256, 384, 512"))
}
}
+3 -3
View File
@@ -3,8 +3,8 @@ package hash
import (
"crypto"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
func sha3Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
@@ -23,6 +23,6 @@ func sha3Func(ctx sqlite3.Context, arg ...sqlite3.Value) {
case 512:
hashFunc(ctx, arg[0], crypto.SHA3_512)
default:
ctx.ResultError(util.ErrorString("sha3: size must be 224, 256, 384, 512"))
ctx.ResultError(errutil.ErrorString("sha3: size must be 224, 256, 384, 512"))
}
}
+113
View File
@@ -0,0 +1,113 @@
// Package ipaddr provides functions to manipulate IPs and CIDRs.
//
// It provides the following functions:
// - ipcontains(prefix, ip)
// - ipoverlaps(prefix1, prefix2)
// - ipfamily(ip/prefix)
// - iphost(ip/prefix)
// - ipmasklen(prefix)
// - ipnetwork(prefix)
package ipaddr
import (
"errors"
"net/netip"
"github.com/hanzoai/sqlite3"
)
// Register IP/CIDR functions for a database connection.
func Register(db *sqlite3.Conn) error {
const flags = sqlite3.DETERMINISTIC | sqlite3.INNOCUOUS
return errors.Join(
db.CreateFunction("ipcontains", 2, flags, contains),
db.CreateFunction("ipoverlaps", 2, flags, overlaps),
db.CreateFunction("ipfamily", 1, flags, family),
db.CreateFunction("iphost", 1, flags, host),
db.CreateFunction("ipmasklen", 1, flags, masklen),
db.CreateFunction("ipnetwork", 1, flags, network))
}
func contains(ctx sqlite3.Context, arg ...sqlite3.Value) {
prefix, err := netip.ParsePrefix(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
addr, err := netip.ParseAddr(arg[1].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
ctx.ResultBool(prefix.Contains(addr))
}
func overlaps(ctx sqlite3.Context, arg ...sqlite3.Value) {
prefix1, err := netip.ParsePrefix(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
prefix2, err := netip.ParsePrefix(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
ctx.ResultBool(prefix1.Overlaps(prefix2))
}
func family(ctx sqlite3.Context, arg ...sqlite3.Value) {
addr, err := addr(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
switch {
case addr.Is4():
ctx.ResultInt(4)
case addr.Is6():
ctx.ResultInt(6)
}
}
func host(ctx sqlite3.Context, arg ...sqlite3.Value) {
addr, err := addr(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
buf, _ := addr.MarshalText()
ctx.ResultRawText(buf)
}
func masklen(ctx sqlite3.Context, arg ...sqlite3.Value) {
prefix, err := netip.ParsePrefix(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
ctx.ResultInt(prefix.Bits())
}
func network(ctx sqlite3.Context, arg ...sqlite3.Value) {
prefix, err := netip.ParsePrefix(arg[0].Text())
if err != nil {
ctx.ResultError(err)
return // notest
}
buf, _ := prefix.Masked().MarshalText()
ctx.ResultRawText(buf)
}
func addr(text string) (netip.Addr, error) {
addr, err := netip.ParseAddr(text)
if err != nil {
if prefix, err := netip.ParsePrefix(text); err == nil {
return prefix.Addr(), nil
}
if addrpt, err := netip.ParseAddrPort(text); err == nil {
return addrpt.Addr(), nil
}
}
return addr, err
}
+88
View File
@@ -0,0 +1,88 @@
package ipaddr_test
import (
"testing"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/ipaddr"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func TestRegister(t *testing.T) {
t.Parallel()
dsn := memdb.TestDB(t)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, ipaddr.Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var got string
err = db.QueryRowContext(ctx, `SELECT ipfamily('::1')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "6" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT ipfamily('[::1]:80')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "6" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT ipfamily('192.168.1.5/24')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "4" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT iphost('192.168.1.5/24')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "192.168.1.5" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT ipmasklen('192.168.1.5/24')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "24" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT ipnetwork('192.168.1.5/24')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "192.168.1.0/24" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT ipcontains('192.168.1.0/24', '192.168.1.5')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "1" {
t.Fatalf("got %s", got)
}
err = db.QueryRowContext(ctx, `SELECT ipoverlaps('192.168.1.0/24', '192.168.1.5/32')`).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != "1" {
t.Fatalf("got %s", got)
}
}
+2 -2
View File
@@ -18,8 +18,8 @@ import (
"io"
"io/fs"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/util/osutil"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/util/osutil"
)
// Register registers the lines and lines_read table-valued functions.
+22 -19
View File
@@ -10,12 +10,11 @@ import (
"strings"
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/lines"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/lines"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example() {
@@ -67,9 +66,10 @@ func Example() {
func Test_lines(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, lines.Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, lines.Register)
if err != nil {
log.Fatal(err)
}
@@ -77,7 +77,7 @@ func Test_lines(t *testing.T) {
const data = "line 1\nline 2\r\nline 3\n"
rows, err := db.Query(`SELECT rowid, line FROM lines(?)`, data)
rows, err := db.QueryContext(ctx, `SELECT rowid, line FROM lines(?)`, data)
if err != nil {
t.Fatal(err)
}
@@ -98,22 +98,23 @@ func Test_lines(t *testing.T) {
func Test_lines_error(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, lines.Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, lines.Register)
if err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`SELECT rowid, line FROM lines(?)`, nil)
_, err = db.ExecContext(ctx, `SELECT rowid, line FROM lines(?)`, nil)
if err == nil {
t.Fatal("want error")
} else {
t.Log(err)
}
_, err = db.Exec(`SELECT rowid, line FROM lines_read(?)`, "xpto")
_, err = db.ExecContext(ctx, `SELECT rowid, line FROM lines_read(?)`, "xpto")
if err == nil {
t.Fatal("want error")
} else {
@@ -123,9 +124,10 @@ func Test_lines_error(t *testing.T) {
func Test_lines_read(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, lines.Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, lines.Register)
if err != nil {
log.Fatal(err)
}
@@ -133,7 +135,7 @@ func Test_lines_read(t *testing.T) {
const data = "line 1\nline 2\r\nline 3\n"
rows, err := db.Query(`SELECT rowid, line FROM lines_read(?)`,
rows, err := db.QueryContext(ctx, `SELECT rowid, line FROM lines_read(?)`,
sqlite3.Pointer(strings.NewReader(data)))
if err != nil {
t.Fatal(err)
@@ -155,15 +157,16 @@ func Test_lines_read(t *testing.T) {
func Test_lines_test(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, lines.Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, lines.Register)
if err != nil {
log.Fatal(err)
}
defer db.Close()
rows, err := db.Query(`SELECT rowid, line FROM lines_read(?, '}')`, "lines_test.go")
rows, err := db.QueryContext(ctx, `SELECT rowid, line FROM lines_read(?, '}')`, "lines_test.go")
if errors.Is(err, os.ErrNotExist) {
t.Skip(err)
}
+1 -1
View File
@@ -3,7 +3,7 @@ package pivot
import (
"testing"
"github.com/ncruces/go-sqlite3"
"github.com/hanzoai/sqlite3"
)
func Test_operator(t *testing.T) {
+26 -10
View File
@@ -8,8 +8,8 @@ import (
"fmt"
"strings"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
// Register registers the pivot virtual table.
@@ -43,11 +43,14 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (ret *table, err e
// Row key query.
t.scan = "SELECT * FROM\n" + arg[0]
stmt, _, err := db.Prepare(t.scan)
stmt, tail, err := db.PrepareFlags(t.scan, sqlite3.PREPARE_FROM_DDL)
if err != nil {
return nil, err
}
defer stmt.Close()
if tail != "" {
return nil, errutil.TailErr
}
t.keys = make([]string, stmt.ColumnCount())
for i := range t.keys {
@@ -55,36 +58,47 @@ func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (ret *table, err e
t.keys[i] = name
create.WriteString(sep)
create.WriteString(name)
create.WriteString(" ")
create.WriteString(stmt.ColumnDeclType(i))
sep = ","
}
stmt.Close()
// Column definition query.
stmt, _, err = db.Prepare("SELECT * FROM\n" + arg[1])
stmt, tail, err = db.PrepareFlags("SELECT * FROM\n"+arg[1], sqlite3.PREPARE_FROM_DDL)
if err != nil {
return nil, err
}
if tail != "" {
return nil, errutil.TailErr
}
if stmt.ColumnCount() != 2 {
return nil, util.ErrorString("pivot: column definition query expects 2 result columns")
return nil, errutil.ErrorString("pivot: column definition query expects 2 result columns")
}
for stmt.Step() {
name := sqlite3.QuoteIdentifier(stmt.ColumnText(1))
t.cols = append(t.cols, stmt.ColumnValue(0).Dup())
create.WriteString(",")
create.WriteString(sep)
create.WriteString(name)
create.WriteString(" ")
create.WriteString(stmt.ColumnDeclType(1))
sep = ","
}
stmt.Close()
// Pivot cell query.
t.cell = "SELECT * FROM\n" + arg[2]
stmt, _, err = db.Prepare(t.cell)
stmt, tail, err = db.PrepareFlags(t.cell, sqlite3.PREPARE_FROM_DDL)
if err != nil {
return nil, err
}
if tail != "" {
return nil, errutil.TailErr
}
if stmt.ColumnCount() != 1 {
return nil, util.ErrorString("pivot: cell query expects 1 result columns")
return nil, errutil.ErrorString("pivot: cell query expects 1 result columns")
}
if stmt.BindCount() != len(t.keys)+1 {
return nil, fmt.Errorf("pivot: cell query expects %d bound parameters", len(t.keys)+1)
@@ -182,7 +196,9 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
return err
}
c.scan, _, err = c.table.db.Prepare(idxStr)
const prepflags = sqlite3.PREPARE_DONT_LOG | sqlite3.PREPARE_FROM_DDL
c.scan, _, err = c.table.db.PrepareFlags(idxStr, prepflags)
if err != nil {
return err
}
@@ -194,7 +210,7 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
}
if c.cell == nil {
c.cell, _, err = c.table.db.Prepare(c.table.cell)
c.cell, _, err = c.table.db.PrepareFlags(c.table.cell, prepflags)
if err != nil {
return err
}
+11 -11
View File
@@ -3,13 +3,13 @@ package pivot_test
import (
"fmt"
"log"
"os"
"strings"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/pivot"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/ext/pivot"
"github.com/hanzoai/sqlite3/internal/testcfg"
)
// https://antonz.org/sqlite-pivot-table/
@@ -85,13 +85,13 @@ func Example() {
func TestMain(m *testing.M) {
sqlite3.AutoExtension(pivot.Register)
m.Run()
os.Exit(m.Run())
}
func TestRegister(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -140,10 +140,10 @@ func TestRegister(t *testing.T) {
}
defer stmt.Close()
if stmt.Step() {
if got := stmt.ColumnInt(0); got != 3 {
t.Errorf("got %d, want 3", got)
}
if !stmt.Step() {
t.Fatal(stmt.Err())
} else if got := stmt.ColumnInt(0); got != 3 {
t.Errorf("got %d, want 3", got)
}
err = db.Exec(`ALTER TABLE v_x RENAME TO v_y`)
@@ -155,7 +155,7 @@ func TestRegister(t *testing.T) {
func TestRegister_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+17 -18
View File
@@ -20,7 +20,7 @@ import (
"strings"
"unicode/utf8"
"github.com/ncruces/go-sqlite3"
"github.com/hanzoai/sqlite3"
)
// Register registers Unicode aware functions for a database connection.
@@ -111,22 +111,24 @@ loop2:
return glob.String()
}
func load(ctx sqlite3.Context, i int, expr string) (*regexp.Regexp, error) {
func load(ctx sqlite3.Context, arg []sqlite3.Value, i int) (*regexp.Regexp, error) {
re, ok := ctx.GetAuxData(i).(*regexp.Regexp)
if !ok {
r, err := regexp.Compile(expr)
if err != nil {
return nil, err
re, ok = arg[i].Pointer().(*regexp.Regexp)
if !ok {
r, err := regexp.Compile(arg[i].Text())
if err != nil {
return nil, err
}
re = r
}
re = r
ctx.SetAuxData(0, r)
ctx.SetAuxData(i, re)
}
return re, nil
}
func regex(ctx sqlite3.Context, arg ...sqlite3.Value) {
_ = arg[1] // bounds check
re, err := load(ctx, 0, arg[0].Text())
re, err := load(ctx, arg, 0)
if err != nil {
ctx.ResultError(err)
return // notest
@@ -136,18 +138,17 @@ func regex(ctx sqlite3.Context, arg ...sqlite3.Value) {
}
func regexLike(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
re, err := load(ctx, arg, 1)
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
ctx.ResultBool(re.Match(text))
}
func regexCount(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
re, err := load(ctx, arg, 1)
if err != nil {
ctx.ResultError(err)
return // notest
@@ -162,7 +163,7 @@ func regexCount(ctx sqlite3.Context, arg ...sqlite3.Value) {
}
func regexSubstr(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
re, err := load(ctx, arg, 1)
if err != nil {
ctx.ResultError(err)
return // notest
@@ -187,7 +188,7 @@ func regexSubstr(ctx sqlite3.Context, arg ...sqlite3.Value) {
}
func regexInstr(ctx sqlite3.Context, arg ...sqlite3.Value) {
re, err := load(ctx, 1, arg[1].Text())
re, err := load(ctx, arg, 1)
if err != nil {
ctx.ResultError(err)
return // notest
@@ -215,16 +216,14 @@ func regexInstr(ctx sqlite3.Context, arg ...sqlite3.Value) {
}
func regexReplace(ctx sqlite3.Context, arg ...sqlite3.Value) {
_ = arg[2] // bounds check
re, err := load(ctx, 1, arg[1].Text())
re, err := load(ctx, arg, 1)
if err != nil {
ctx.ResultError(err)
return // notest
}
text := arg[0].RawText()
repl := arg[2].RawText()
text := arg[0].RawText()
var pos, n int
if len(arg) > 3 {
pos = arg[3].Int()
+35 -11
View File
@@ -6,17 +6,18 @@ import (
"strings"
"testing"
"github.com/ncruces/go-sqlite3/driver"
_ "github.com/ncruces/go-sqlite3/embed"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/ncruces/go-sqlite3/vfs/memdb"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func TestRegister(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, Register)
if err != nil {
t.Fatal(err)
}
@@ -37,7 +38,7 @@ func TestRegister(t *testing.T) {
{`regexp_instr('Hello', '.', 6)`, ""},
{`regexp_substr('Hello', 'el.')`, "ell"},
{`regexp_replace('Hello', 'llo', 'll')`, "Hell"},
// https://www.postgresql.org/docs/current/functions-matching.html
// https://postgresql.org/docs/current/functions-matching.html
{`regexp_count('ABCABCAXYaxy', 'A.')`, "3"},
{`regexp_count('ABCABCAXYaxy', '(?i)A.', 1)`, "4"},
{`regexp_instr('number of your street, town zip, FR', '[^,]+', 1, 2)`, "23"},
@@ -67,7 +68,7 @@ func TestRegister(t *testing.T) {
for _, tt := range tests {
var got sql.NullString
err := db.QueryRow(`SELECT ` + tt.test).Scan(&got)
err := db.QueryRowContext(ctx, `SELECT `+tt.test).Scan(&got)
if err != nil {
t.Fatal(err)
}
@@ -79,9 +80,10 @@ func TestRegister(t *testing.T) {
func TestRegister_errors(t *testing.T) {
t.Parallel()
tmp := memdb.TestDB(t)
dsn := memdb.TestDB(t)
db, err := driver.Open(tmp, Register)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, Register)
if err != nil {
t.Fatal(err)
}
@@ -97,13 +99,35 @@ func TestRegister_errors(t *testing.T) {
}
for _, tt := range tests {
err := db.QueryRow(`SELECT `+tt, `\`).Scan(nil)
err := db.QueryRowContext(ctx, `SELECT `+tt, `\`).Scan(nil)
if err == nil {
t.Fatal("want error")
}
}
}
func TestRegister_pointer(t *testing.T) {
t.Parallel()
dsn := memdb.TestDB(t)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
var got int
err = db.QueryRowContext(ctx, `SELECT regexp_count('ABCABCAXYaxy', ?, 1)`,
sqlite3.Pointer(regexp.MustCompile(`(?i)A.`))).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != 4 {
t.Errorf("got %d, want %d", got, 4)
}
}
func TestGlobPrefix(t *testing.T) {
tests := []struct {
re string
+15
View File
@@ -0,0 +1,15 @@
// Package rtree provides the rtree and geopoly virtual tables.
//
// https://sqlite.org/rtree.html
// https://sqlite.org/geopoly.html
package rtree
import (
"github.com/hanzoai/sqlite3"
"github.com/ncruces/go-sqlite3-wasm/v3/rtree"
)
// Register registers the rtree and geopoly virtual tables.
func Register(db *sqlite3.Conn) error {
return sqlite3.ExtensionInit(db, rtree.New, rtree.DylinkInfo)
}
+63
View File
@@ -0,0 +1,63 @@
package rtree_test
import (
"fmt"
"log"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/rtree"
_ "github.com/hanzoai/sqlite3/vfs/memdb"
)
func Example_rtree() {
db, err := driver.Open("file:/test.db?vfs=memdb", rtree.Register)
if err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
CREATE VIRTUAL TABLE locations USING rtree(id, minX, maxX, minY, maxY);
INSERT INTO locations VALUES (1, -10.0, -5.0, 30.0, 35.0);
INSERT INTO locations VALUES (2, 100.0, 105.0, 0.0, 5.0);
`)
if err != nil {
log.Fatal(err)
}
var id int
err = db.QueryRow(`SELECT id FROM locations WHERE
minX >= -12.0 AND maxX <= -4.0 AND
minY >= 28.0 AND maxY <= 36.0`).Scan(&id)
if err != nil {
log.Fatal(err)
}
fmt.Println(id)
// Output: 1
}
func Example_geopoly() {
db, err := driver.Open("file:/test.db?vfs=memdb", rtree.Register)
if err != nil {
log.Fatal(err)
}
defer db.Close()
_, err = db.Exec(`
CREATE VIRTUAL TABLE areas USING geopoly(name);
INSERT INTO areas(name, _shape) VALUES ('Triangle Area', '[[0,0],[2,0],[1,2],[0,0]]');
`)
if err != nil {
log.Fatal(err)
}
var name string
err = db.QueryRow("SELECT name FROM areas WHERE geopoly_contains_point(_shape, 1.0, 1.0)").Scan(&name)
if err != nil {
log.Fatal(err)
}
fmt.Println(name)
// Output: Triangle Area
}
+14 -82
View File
@@ -2,28 +2,27 @@
package serdes
import (
"io"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/vfs"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/util/vfsutil"
"github.com/hanzoai/sqlite3/vfs"
)
const vfsName = "github.com/ncruces/go-sqlite3/ext/serdes.sliceVFS"
const vfsName = "github.com/hanzoai/sqlite3/ext/serdes.sliceVFS"
func init() {
vfs.Register(vfsName, sliceVFS{})
}
var fileToOpen = make(chan *sliceFile, 1)
var fileToOpen = make(chan *[]byte, 1)
// Serialize backs up a database into a byte slice.
//
// https://sqlite.org/c3ref/serialize.html
func Serialize(db *sqlite3.Conn, schema string) ([]byte, error) {
var file sliceFile
var file []byte
fileToOpen <- &file
err := db.Backup(schema, "file:serdes.db?vfs="+vfsName)
return file.data, err
err := db.Backup(schema, "file:serdes.db?nolock=1&vfs="+vfsName)
return file, err
}
// Deserialize restores a database from a byte slice,
@@ -38,11 +37,11 @@ func Serialize(db *sqlite3.Conn, schema string) ([]byte, error) {
//
// https://sqlite.org/c3ref/deserialize.html
//
// ["memdb"]: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/memdb
// ["reader"]: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readervfs
// ["memdb"]: https://pkg.go.dev/github.com/hanzoai/sqlite3/vfs/memdb
// ["reader"]: https://pkg.go.dev/github.com/hanzoai/sqlite3/vfs/readervfs
func Deserialize(db *sqlite3.Conn, schema string, data []byte) error {
fileToOpen <- &sliceFile{data}
return db.Restore(schema, "file:serdes.db?vfs="+vfsName)
fileToOpen <- &data
return db.Restore(schema, "file:serdes.db?immutable=1&vfs="+vfsName)
}
type sliceVFS struct{}
@@ -53,14 +52,14 @@ func (sliceVFS) Open(name string, flags vfs.OpenFlag) (vfs.File, vfs.OpenFlag, e
}
select {
case file := <-fileToOpen:
return file, flags | vfs.OPEN_MEMORY, nil
return (*vfsutil.SliceFile)(file), flags | vfs.OPEN_MEMORY, nil
default:
return nil, flags, sqlite3.MISUSE
}
}
func (sliceVFS) Delete(name string, dirSync bool) error {
// notest // OPEN_MEMORY
// notest // no journals to delete
return sqlite3.IOERR_DELETE
}
@@ -71,70 +70,3 @@ func (sliceVFS) Access(name string, flag vfs.AccessFlag) (bool, error) {
func (sliceVFS) FullPathname(name string) (string, error) {
return name, nil
}
type sliceFile struct{ data []byte }
func (f *sliceFile) ReadAt(b []byte, off int64) (n int, err error) {
if d := f.data; off < int64(len(d)) {
n = copy(b, d[off:])
}
if n == 0 {
err = io.EOF
}
return
}
func (f *sliceFile) WriteAt(b []byte, off int64) (n int, err error) {
if d := f.data; off > int64(len(d)) {
f.data = append(d, make([]byte, off-int64(len(d)))...)
}
d := append(f.data[:off], b...)
if len(d) > len(f.data) {
f.data = d
}
return len(b), nil
}
func (f *sliceFile) Size() (int64, error) {
return int64(len(f.data)), nil
}
func (f *sliceFile) Truncate(size int64) error {
if d := f.data; size < int64(len(d)) {
f.data = d[:size]
}
return nil
}
func (f *sliceFile) SizeHint(size int64) error {
if d := f.data; size > int64(len(d)) {
f.data = append(d, make([]byte, size-int64(len(d)))...)
}
return nil
}
func (*sliceFile) Close() error { return nil }
func (*sliceFile) Sync(flag vfs.SyncFlag) error { return nil }
func (*sliceFile) Lock(lock vfs.LockLevel) error { return nil }
func (*sliceFile) Unlock(lock vfs.LockLevel) error { return nil }
func (*sliceFile) CheckReservedLock() (bool, error) {
// notest // OPEN_MEMORY
return false, nil
}
func (*sliceFile) SectorSize() int {
// notest // IOCAP_POWERSAFE_OVERWRITE
return 0
}
func (*sliceFile) DeviceCharacteristics() vfs.DeviceCharacteristic {
return vfs.IOCAP_ATOMIC |
vfs.IOCAP_SAFE_APPEND |
vfs.IOCAP_SEQUENTIAL |
vfs.IOCAP_POWERSAFE_OVERWRITE |
vfs.IOCAP_SUBPAGE_READ
}
+45 -12
View File
@@ -1,17 +1,46 @@
package serdes_test
import (
_ "embed"
"errors"
"io"
"net/http"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/serdes"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/ext/serdes"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs"
)
func TestDeserialize(t *testing.T) {
//go:embed testdata/wal.db
var walDB []byte
func Test_wal(t *testing.T) {
if !vfs.SupportsFileLocking {
t.Skip("skipping without locks")
}
db, err := sqlite3.OpenContext(testcfg.Context(t), "testdata/wal.db")
if err != nil {
t.Fatal(err)
}
defer db.Close()
data, err := serdes.Serialize(db, "main")
if err != nil {
t.Fatal(err)
}
compareDBs(t, data, walDB)
err = serdes.Deserialize(db, "temp", walDB)
if err != nil {
t.Fatal(err)
}
}
func Test_northwind(t *testing.T) {
if testing.Short() {
t.Skip("skipping in short mode")
}
@@ -21,7 +50,7 @@ func TestDeserialize(t *testing.T) {
t.Fatal(err)
}
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -37,10 +66,14 @@ func TestDeserialize(t *testing.T) {
t.Fatal(err)
}
if len(input) != len(output) {
compareDBs(t, input, output)
}
func compareDBs(t *testing.T, a, b []byte) {
if len(a) != len(b) {
t.Fatal("lengths are different")
}
for i := range input {
for i := range a {
// These may be different.
switch {
case 24 <= i && i < 28:
@@ -53,14 +86,14 @@ func TestDeserialize(t *testing.T) {
// SQLite version that wrote the file.
continue
}
if input[i] != output[i] {
t.Errorf("difference at %d: %d %d", i, input[i], output[i])
if a[i] != b[i] {
t.Errorf("difference at %d: %d %d", i, a[i], b[i])
}
}
}
func httpGet() ([]byte, error) {
res, err := http.Get("https://raw.githubusercontent.com/jpwhite3/northwind-SQLite3/refs/heads/main/dist/northwind.db")
res, err := http.Get("https://github.com/jpwhite3/northwind-SQLite3/raw/refs/heads/main/dist/northwind.db")
if err != nil {
return nil, err
}
@@ -69,7 +102,7 @@ func httpGet() ([]byte, error) {
}
func TestOpen_errors(t *testing.T) {
_, err := sqlite3.Open("file:test.db?vfs=github.com/ncruces/go-sqlite3/ext/serdes.sliceVFS")
_, err := sqlite3.OpenContext(testcfg.Context(t), "file:test.db?vfs=github.com/hanzoai/sqlite3/ext/serdes.sliceVFS")
if err == nil {
t.Error("want error")
}
@@ -77,7 +110,7 @@ func TestOpen_errors(t *testing.T) {
t.Errorf("got %v, want sqlite3.CANTOPEN", err)
}
_, err = sqlite3.Open("file:serdes.db?vfs=github.com/ncruces/go-sqlite3/ext/serdes.sliceVFS")
_, err = sqlite3.OpenContext(testcfg.Context(t), "file:serdes.db?vfs=github.com/hanzoai/sqlite3/ext/serdes.sliceVFS")
if err == nil {
t.Error("want error")
}
BIN
View File
Binary file not shown.
+14
View File
@@ -0,0 +1,14 @@
// Package spellfix1 provides the spellfix1 virtual table.
//
// https://sqlite.org/spellfix1.html
package spellfix1
import (
"github.com/hanzoai/sqlite3"
"github.com/ncruces/go-sqlite3-wasm/v3/spellfix"
)
// Register registers the spellfix1 virtual table.
func Register(db *sqlite3.Conn) error {
return sqlite3.ExtensionInit(db, spellfix.New, spellfix.DylinkInfo)
}
+65
View File
@@ -0,0 +1,65 @@
package spellfix1_test
import (
"testing"
"github.com/hanzoai/sqlite3/driver"
"github.com/hanzoai/sqlite3/ext/spellfix1"
"github.com/hanzoai/sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3/vfs/memdb"
)
func Test(t *testing.T) {
dsn := memdb.TestDB(t)
ctx := testcfg.Context(t)
db, err := driver.Open(dsn, spellfix1.Register)
if err != nil {
t.Fatal(err)
}
defer db.Close()
_, err = db.ExecContext(ctx, `
CREATE TABLE cost1(iLang, cFrom, cTo, iCost);
INSERT INTO cost1 VALUES
(0, '', '?', 97),
(0, '?', '', 98),
(0, '?', '?', 99),
(0, 'm', 'n', 50),
(0, 'n', 'm', 50);
SELECT editdist3('cost1');
`)
if err != nil {
t.Fatal(err)
}
tests := []struct {
s1 string
s2 string
want int
}{
{"anchor", "amchor", 50},
{"anchor", "anchoxr", 97},
{"anchor", "anchorx", 97},
{"anchor", "anchr", 98},
{"anchor", "ancho", 98},
{"anchor", "nchor", 98},
{"anchor", "anchur", 99},
{"anchor", "onchor", 99},
{"anchor", "anchot", 99},
{"anchor", "omchor", 149},
}
for _, tt := range tests {
t.Run("", func(t *testing.T) {
var got int
err := db.QueryRowContext(ctx, `SELECT editdist3(?, ?)`, tt.s1, tt.s2).Scan(&got)
if err != nil {
t.Fatal(err)
}
if got != tt.want {
t.Errorf("got %d, want %d", got, tt.want)
}
})
}
}
+27 -21
View File
@@ -13,8 +13,8 @@ import (
"strings"
"unsafe"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/internal/util"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/internal/errutil"
)
// Register registers the statement virtual table.
@@ -30,45 +30,50 @@ type table struct {
func declare(db *sqlite3.Conn, _, _, _ string, arg ...string) (*table, error) {
if len(arg) != 1 {
return nil, util.ErrorString("statement: wrong number of arguments")
return nil, errutil.ErrorString("statement: wrong number of arguments")
}
sql := "SELECT * FROM\n" + arg[0]
stmt, _, err := db.PrepareFlags(sql, sqlite3.PREPARE_PERSISTENT)
stmt, tail, err := db.PrepareFlags(sql,
sqlite3.PREPARE_PERSISTENT|sqlite3.PREPARE_FROM_DDL)
if err != nil {
return nil, err
}
if tail != "" {
stmt.Close()
return nil, errutil.TailErr
}
var sep string
var str strings.Builder
str.WriteString("CREATE TABLE x(")
var buf strings.Builder
buf.WriteString("CREATE TABLE x(")
outputs := stmt.ColumnCount()
for i := range outputs {
name := sqlite3.QuoteIdentifier(stmt.ColumnName(i))
str.WriteString(sep)
str.WriteString(name)
str.WriteString(" ")
str.WriteString(stmt.ColumnDeclType(i))
buf.WriteString(sep)
buf.WriteString(name)
buf.WriteString(" ")
buf.WriteString(stmt.ColumnDeclType(i))
sep = ","
}
inputs := stmt.BindCount()
for i := 1; i <= inputs; i++ {
str.WriteString(sep)
buf.WriteString(sep)
name := stmt.BindName(i)
if name == "" {
str.WriteString("[")
str.WriteString(strconv.Itoa(i))
str.WriteString("] HIDDEN")
buf.WriteString("[")
buf.WriteString(strconv.Itoa(i))
buf.WriteString("] HIDDEN")
} else {
str.WriteString(sqlite3.QuoteIdentifier(name[1:]))
str.WriteString(" HIDDEN")
buf.WriteString(sqlite3.QuoteIdentifier(name[1:]))
buf.WriteString(" HIDDEN")
}
sep = ","
}
str.WriteByte(')')
buf.WriteByte(')')
err = db.DeclareVTab(str.String())
err = db.DeclareVTab(buf.String())
if err != nil {
stmt.Close()
return nil, err
@@ -129,7 +134,8 @@ func (t *table) Open() (_ sqlite3.VTabCursor, err error) {
if !t.inuse {
t.inuse = true
} else {
stmt, _, err = t.stmt.Conn().Prepare(t.sql)
stmt, _, err = t.stmt.Conn().PrepareFlags(t.sql,
sqlite3.PREPARE_DONT_LOG|sqlite3.PREPARE_FROM_DDL)
if err != nil {
return nil, err
}
@@ -159,8 +165,6 @@ func (c *cursor) Close() error {
}
func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
c.arg = arg
c.rowID = 0
err := errors.Join(
c.stmt.Reset(),
c.stmt.ClearBindings())
@@ -187,6 +191,8 @@ func (c *cursor) Filter(idxNum int, idxStr string, arg ...sqlite3.Value) error {
return err
}
}
c.arg = append(c.arg[:0], arg...)
c.rowID = 0
return c.Next()
}
+10 -8
View File
@@ -3,12 +3,12 @@ package statement_test
import (
"fmt"
"log"
"os"
"testing"
"github.com/ncruces/go-sqlite3"
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/ext/statement"
_ "github.com/ncruces/go-sqlite3/internal/testcfg"
"github.com/hanzoai/sqlite3"
"github.com/hanzoai/sqlite3/ext/statement"
"github.com/hanzoai/sqlite3/internal/testcfg"
)
func Example() {
@@ -50,13 +50,13 @@ func Example() {
func TestMain(m *testing.M) {
sqlite3.AutoExtension(statement.Register)
m.Run()
os.Exit(m.Run())
}
func TestRegister(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
@@ -91,7 +91,9 @@ func TestRegister(t *testing.T) {
}
defer stmt.Close()
if stmt.Step() {
if !stmt.Step() {
t.Fatal(stmt.Err())
} else {
x := stmt.ColumnInt(0)
y := stmt.ColumnInt(1)
hypot := stmt.ColumnInt(2)
@@ -104,7 +106,7 @@ func TestRegister(t *testing.T) {
func TestRegister_errors(t *testing.T) {
t.Parallel()
db, err := sqlite3.Open(":memory:")
db, err := sqlite3.OpenContext(testcfg.Context(t), ":memory:")
if err != nil {
t.Fatal(err)
}
+1 -1
View File
@@ -1,6 +1,6 @@
# ANSI SQL Aggregate Functions
https://www.oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html
https://oreilly.com/library/view/sql-in-a/9780596155322/ch04s02.html
## Built in aggregates
+2 -2
View File
@@ -1,13 +1,13 @@
package stats
import "github.com/ncruces/go-sqlite3"
import "github.com/hanzoai/sqlite3"
const (
every = iota
some
)
func newBoolean(kind int) func() sqlite3.AggregateFunction {
func newBoolean(kind int) sqlite3.AggregateConstructor {
return func() sqlite3.AggregateFunction { return &boolean{kind: kind} }
}

Some files were not shown because too many files have changed in this diff Show More