37f9a9fa94
# Description ## Checklist Please put an x inside each checkbox to indicate that you've read and followed it: `[ ]` -> `[x]` If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want). - [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md). - [ ] I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat. - [x] I/we have not leveraged AI to create the proposed changes. - [ ] I/we have performed a self-review of added code. - [ ] I/we have written code that is legible and maintainable by others. - [ ] I/we have commented the added code, particularly in hard-to-understand areas. - [ ] I/we have made any necessary changes to documentation. - [ ] I/we have added tests that cover new code. - [ ] I/we have run tests and they pass locally with the changes. - [ ] I/we have run `go fmt ./...` and `golangci-lint run`. Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4255 Co-authored-by: Daenney <daenney@noreply.codeberg.org> Co-committed-by: Daenney <daenney@noreply.codeberg.org>
5.2 KiB
Go SQLite VFS API
This package implements the SQLite OS Interface (aka VFS).
It replaces the default SQLite VFS with a pure Go implementation, and exposes interfaces that should allow you to implement your own custom VFSes.
See the support matrix for the list of supported OS and CPU architectures.
Since this is a from scratch reimplementation, there are naturally some ways it deviates from the original. It's also not as battle tested as the original.
The main differences to be aware of are file locking and WAL mode support.
File Locking
POSIX advisory locks, which SQLite uses on Unix, are broken by design. Instead, on Linux and macOS, this package uses OFD locks to synchronize access to database files.
This package can also use
BSD locks,
albeit with reduced concurrency (BEGIN IMMEDIATE behaves like BEGIN EXCLUSIVE,
docs).
BSD locks are the default on BSD and illumos,
but you can opt into them with the sqlite3_flock build tag.
On Windows, this package uses LockFileEx and UnlockFileEx,
like SQLite.
You can also opt into a cross-platform locking implementation
with the sqlite3_dotlk build tag.
Otherwise, file locking is not supported, and you must use
nolock=1
(or immutable=1)
to open database files.
To use the database/sql driver
with nolock=1 you must disable connection pooling by calling
db.SetMaxOpenConns(1).
You can use vfs.SupportsFileLocking
to check if your build supports file locking.
Write-Ahead Logging
On Unix, this package uses mmap to implement
shared-memory for the WAL-index,
like SQLite.
On Windows, this package uses MapViewOfFile, like SQLite.
You can also opt into a cross-platform, in-process, memory sharing implementation
with the sqlite3_dotlk build tag.
Otherwise, WAL support is limited,
and EXCLUSIVE locking mode must be set to create, read, and write WAL databases.
To use EXCLUSIVE locking mode with the
database/sql driver
you must disable connection pooling by calling
db.SetMaxOpenConns(1).
You can use vfs.SupportsSharedMemory
to check if your build supports shared memory.
Blocking Locks
On Windows and macOS, this package implements Wal-mode blocking locks.
Batch-Atomic Write
On Linux, this package may support batch-atomic writes on the F2FS filesystem.
Checksums
This package can be configured
to add an 8-byte checksum to the end of every page in an SQLite database.
The checksum is added as each page is written
and verified as each page is read.
The checksum is intended to help detect database corruption
caused by random bit-flips in the mass storage device.
The implementation is compatible with SQLite's Checksum VFS Shim.
Build Tags
The VFS can be customized with a few build tags:
sqlite3_flockforces the use of BSD locks.sqlite3_dotlkforces the use of dot-file locks.
Important
The default configuration of this package is compatible with the standard Unix and Windows SQLite VFSes;
sqlite3_flockbuilds are compatible with theunix-flockVFS;sqlite3_dotlkbuilds are compatible with theunix-dotfileVFS.
Caution
Concurrently accessing databases using incompatible VFSes will eventually corrupt data.
Custom VFSes
github.com/ncruces/go-sqlite3/vfs/memdbimplements an in-memory VFS.github.com/ncruces/go-sqlite3/vfs/readervfsimplements a VFS for immutable databases.github.com/ncruces/go-sqlite3/vfs/adiantumwraps a VFS to offer encryption at rest.github.com/ncruces/go-sqlite3/vfs/xtswraps a VFS to offer encryption at rest.