[chore] update dependencies (#4423)

- codeberg.org/gruf/go-ffmpreg: v0.6.10 -> v0.6.11
- github.com/spf13/cast: v1.9.2 -> v1.10.0
- github.com/spf13/viper: v1.20.1 -> v1.21.0
- golang.org/x/crypto: v0.41.0 -> v0.42.0
- golang.org/x/image: v0.30.0 -> v0.31.0

Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4423
Co-authored-by: kim <grufwub@gmail.com>
Co-committed-by: kim <grufwub@gmail.com>
This commit is contained in:
kim 2025-09-09 16:12:29 +02:00 committed by kim
commit c949b9f2d1
97 changed files with 14611 additions and 3494 deletions

View file

@ -10,3 +10,6 @@ trim_trailing_whitespace = true
[*.go]
indent_style = tab
[{*.yml,*.yaml}]
indent_size = 2

View file

@ -1,18 +1,48 @@
linters-settings:
gci:
sections:
- standard
- default
- prefix(github.com/spf13/afero)
version: "2"
run:
timeout: 10m
linters:
disable-all: true
enable:
- gci
- gofmt
- gofumpt
- staticcheck
enable:
- govet
- ineffassign
- misspell
- nolintlint
# - revive
- staticcheck
- unused
issues:
exclude-dirs:
- gcsfs/internal/stiface
disable:
- errcheck
# - staticcheck
settings:
misspell:
locale: US
nolintlint:
allow-unused: false # report any unused nolint directives
require-specific: false # don't require nolint directives to be specific about which linter is being skipped
exclusions:
paths:
- gcsfs/internal/stiface
formatters:
enable:
- gci
- gofmt
- gofumpt
- goimports
- golines
settings:
gci:
sections:
- standard
- default
- localmodule
exclusions:
paths:
- gcsfs/internal/stiface

View file

@ -1,442 +1,474 @@
![afero logo-sm](https://cloud.githubusercontent.com/assets/173412/11490338/d50e16dc-97a5-11e5-8b12-019a300d0fcb.png)
A FileSystem Abstraction System for Go
[![Test](https://github.com/spf13/afero/actions/workflows/test.yml/badge.svg)](https://github.com/spf13/afero/actions/workflows/test.yml) [![GoDoc](https://godoc.org/github.com/spf13/afero?status.svg)](https://godoc.org/github.com/spf13/afero) [![Join the chat at https://gitter.im/spf13/afero](https://badges.gitter.im/Dev%20Chat.svg)](https://gitter.im/spf13/afero?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Overview
Afero is a filesystem framework providing a simple, uniform and universal API
interacting with any filesystem, as an abstraction layer providing interfaces,
types and methods. Afero has an exceptionally clean interface and simple design
without needless constructors or initialization methods.
Afero is also a library providing a base set of interoperable backend
filesystems that make it easy to work with, while retaining all the power
and benefit of the os and ioutil packages.
Afero provides significant improvements over using the os package alone, most
notably the ability to create mock and testing filesystems without relying on the disk.
It is suitable for use in any situation where you would consider using the OS
package as it provides an additional abstraction that makes it easy to use a
memory backed file system during testing. It also adds support for the http
filesystem for full interoperability.
<img src="https://cloud.githubusercontent.com/assets/173412/11490338/d50e16dc-97a5-11e5-8b12-019a300d0fcb.png" alt="afero logo-sm"/>
## Afero Features
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/spf13/afero/ci.yaml?branch=master&amp;style=flat-square)](https://github.com/spf13/afero/actions?query=workflow%3ACI)
[![GoDoc](https://pkg.go.dev/badge/mod/github.com/spf13/afero)](https://pkg.go.dev/mod/github.com/spf13/afero)
[![Go Report Card](https://goreportcard.com/badge/github.com/spf13/afero)](https://goreportcard.com/report/github.com/spf13/afero)
![Go Version](https://img.shields.io/badge/go%20version-%3E=1.23-61CFDD.svg?style=flat-square")
* A single consistent API for accessing a variety of filesystems
* Interoperation between a variety of file system types
* A set of interfaces to encourage and enforce interoperability between backends
* An atomic cross platform memory backed file system
* Support for compositional (union) file systems by combining multiple file systems acting as one
* Specialized backends which modify existing filesystems (Read Only, Regexp filtered)
* A set of utility functions ported from io, ioutil & hugo to be afero aware
* Wrapper for go 1.16 filesystem abstraction `io/fs.FS`
# Using Afero
# Afero: The Universal Filesystem Abstraction for Go
Afero is easy to use and easier to adopt.
Afero is a powerful and extensible filesystem abstraction system for Go. It provides a single, unified API for interacting with diverse filesystems—including the local disk, memory, archives, and network storage.
A few different ways you could use Afero:
Afero acts as a drop-in replacement for the standard `os` package, enabling you to write modular code that is agnostic to the underlying storage, dramatically simplifies testing, and allows for sophisticated architectural patterns through filesystem composition.
* Use the interfaces alone to define your own file system.
* Wrapper for the OS packages.
* Define different filesystems for different parts of your application.
* Use Afero for mock filesystems while testing
## Why Afero?
## Step 1: Install Afero
Afero elevates filesystem interaction beyond simple file reading and writing, offering solutions for testability, flexibility, and advanced architecture.
First use go get to install the latest version of the library.
🔑 **Key Features:**
$ go get github.com/spf13/afero
* **Universal API:** Write your code once. Run it against the local OS, in-memory storage, ZIP/TAR archives, or remote systems (SFTP, GCS).
* **Ultimate Testability:** Utilize `MemMapFs`, a fully concurrent-safe, read/write in-memory filesystem. Write fast, isolated, and reliable unit tests without touching the physical disk or worrying about cleanup.
* **Powerful Composition:** Afero's hidden superpower. Layer filesystems on top of each other to create sophisticated behaviors:
* **Sandboxing:** Use `CopyOnWriteFs` to create temporary scratch spaces that isolate changes from the base filesystem.
* **Caching:** Use `CacheOnReadFs` to automatically layer a fast cache (like memory) over a slow backend (like a network drive).
* **Security Jails:** Use `BasePathFs` to restrict application access to a specific subdirectory (chroot).
* **`os` Package Compatibility:** Afero mirrors the functions in the standard `os` package, making adoption and refactoring seamless.
* **`io/fs` Compatibility:** Fully compatible with the Go standard library's `io/fs` interfaces.
## Installation
```bash
go get github.com/spf13/afero
```
Next include Afero in your application.
```go
import "github.com/spf13/afero"
```
## Step 2: Declare a backend
## Quick Start: The Power of Abstraction
First define a package variable and set it to a pointer to a filesystem.
```go
var AppFs = afero.NewMemMapFs()
The core of Afero is the `afero.Fs` interface. By designing your functions to accept this interface rather than calling `os.*` functions directly, your code instantly becomes more flexible and testable.
or
### 1. Refactor Your Code
var AppFs = afero.NewOsFs()
```
It is important to note that if you repeat the composite literal you
will be using a completely new and isolated filesystem. In the case of
OsFs it will still use the same underlying filesystem but will reduce
the ability to drop in other filesystems as desired.
## Step 3: Use it like you would the OS package
Throughout your application use any function and method like you normally
would.
So if my application before had:
```go
os.Open("/tmp/foo")
```
We would replace it with:
```go
AppFs.Open("/tmp/foo")
```
`AppFs` being the variable we defined above.
## List of all available functions
File System Methods Available:
```go
Chmod(name string, mode os.FileMode) : error
Chown(name string, uid, gid int) : error
Chtimes(name string, atime time.Time, mtime time.Time) : error
Create(name string) : File, error
Mkdir(name string, perm os.FileMode) : error
MkdirAll(path string, perm os.FileMode) : error
Name() : string
Open(name string) : File, error
OpenFile(name string, flag int, perm os.FileMode) : File, error
Remove(name string) : error
RemoveAll(path string) : error
Rename(oldname, newname string) : error
Stat(name string) : os.FileInfo, error
```
File Interfaces and Methods Available:
```go
io.Closer
io.Reader
io.ReaderAt
io.Seeker
io.Writer
io.WriterAt
Name() : string
Readdir(count int) : []os.FileInfo, error
Readdirnames(n int) : []string, error
Stat() : os.FileInfo, error
Sync() : error
Truncate(size int64) : error
WriteString(s string) : ret int, err error
```
In some applications it may make sense to define a new package that
simply exports the file system variable for easy access from anywhere.
## Using Afero's utility functions
Afero provides a set of functions to make it easier to use the underlying file systems.
These functions have been primarily ported from io & ioutil with some developed for Hugo.
The afero utilities support all afero compatible backends.
The list of utilities includes:
Change functions that rely on the `os` package to accept `afero.Fs`.
```go
DirExists(path string) (bool, error)
Exists(path string) (bool, error)
FileContainsBytes(filename string, subslice []byte) (bool, error)
GetTempDir(subPath string) string
IsDir(path string) (bool, error)
IsEmpty(path string) (bool, error)
ReadDir(dirname string) ([]os.FileInfo, error)
ReadFile(filename string) ([]byte, error)
SafeWriteReader(path string, r io.Reader) (err error)
TempDir(dir, prefix string) (name string, err error)
TempFile(dir, prefix string) (f File, err error)
Walk(root string, walkFn filepath.WalkFunc) error
WriteFile(filename string, data []byte, perm os.FileMode) error
WriteReader(path string, r io.Reader) (err error)
```
For a complete list see [Afero's GoDoc](https://godoc.org/github.com/spf13/afero)
// Before: Coupled to the OS and difficult to test
// func ProcessConfiguration(path string) error {
// data, err := os.ReadFile(path)
// ...
// }
They are available under two different approaches to use. You can either call
them directly where the first parameter of each function will be the file
system, or you can declare a new `Afero`, a custom type used to bind these
functions as methods to a given filesystem.
import "github.com/spf13/afero"
### Calling utilities directly
```go
fs := new(afero.MemMapFs)
f, err := afero.TempFile(fs,"", "ioutil-test")
```
### Calling via Afero
```go
fs := afero.NewMemMapFs()
afs := &afero.Afero{Fs: fs}
f, err := afs.TempFile("", "ioutil-test")
```
## Using Afero for Testing
There is a large benefit to using a mock filesystem for testing. It has a
completely blank state every time it is initialized and can be easily
reproducible regardless of OS. You could create files to your hearts content
and the file access would be fast while also saving you from all the annoying
issues with deleting temporary files, Windows file locking, etc. The MemMapFs
backend is perfect for testing.
* Much faster than performing I/O operations on disk
* Avoid security issues and permissions
* Far more control. 'rm -rf /' with confidence
* Test setup is far more easier to do
* No test cleanup needed
One way to accomplish this is to define a variable as mentioned above.
In your application this will be set to afero.NewOsFs() during testing you
can set it to afero.NewMemMapFs().
It wouldn't be uncommon to have each test initialize a blank slate memory
backend. To do this I would define my `appFS = afero.NewOsFs()` somewhere
appropriate in my application code. This approach ensures that Tests are order
independent, with no test relying on the state left by an earlier test.
Then in my tests I would initialize a new MemMapFs for each test:
```go
func TestExist(t *testing.T) {
appFS := afero.NewMemMapFs()
// create test files and directories
appFS.MkdirAll("src/a", 0755)
afero.WriteFile(appFS, "src/a/b", []byte("file b"), 0644)
afero.WriteFile(appFS, "src/c", []byte("file c"), 0644)
name := "src/c"
_, err := appFS.Stat(name)
if os.IsNotExist(err) {
t.Errorf("file \"%s\" does not exist.\n", name)
}
// After: Decoupled, flexible, and testable
func ProcessConfiguration(fs afero.Fs, path string) error {
// Use Afero utility functions which mirror os/ioutil
data, err := afero.ReadFile(fs, path)
// ... process the data
return err
}
```
# Available Backends
### 2. Usage in Production
## Operating System Native
### OsFs
The first is simply a wrapper around the native OS calls. This makes it
very easy to use as all of the calls are the same as the existing OS
calls. It also makes it trivial to have your code use the OS during
operation and a mock filesystem during testing or as needed.
In your production environment, inject the `OsFs` backend, which wraps the standard operating system calls.
```go
appfs := afero.NewOsFs()
appfs.MkdirAll("src/a", 0755)
func main() {
// Use the real OS filesystem
AppFs := afero.NewOsFs()
ProcessConfiguration(AppFs, "/etc/myapp.conf")
}
```
## Memory Backed Storage
### 3. Usage in Testing
### MemMapFs
Afero also provides a fully atomic memory backed filesystem perfect for use in
mocking and to speed up unnecessary disk io when persistence isnt
necessary. It is fully concurrent and will work within go routines
safely.
In your tests, inject `MemMapFs`. This provides a blazing-fast, isolated, in-memory filesystem that requires no disk I/O and no cleanup.
```go
mm := afero.NewMemMapFs()
mm.MkdirAll("src/a", 0755)
func TestProcessConfiguration(t *testing.T) {
// Use the in-memory filesystem
AppFs := afero.NewMemMapFs()
// Pre-populate the memory filesystem for the test
configPath := "/test/config.json"
afero.WriteFile(AppFs, configPath, []byte(`{"feature": true}`), 0644)
// Run the test entirely in memory
err := ProcessConfiguration(AppFs, configPath)
if err != nil {
t.Fatal(err)
}
}
```
#### InMemoryFile
## Afero's Superpower: Composition
As part of MemMapFs, Afero also provides an atomic, fully concurrent memory
backed file implementation. This can be used in other memory backed file
systems with ease. Plans are to add a radix tree memory stored file
system using InMemoryFile.
Afero's most unique feature is its ability to combine filesystems. This allows you to build complex behaviors out of simple components, keeping your application logic clean.
## Network Interfaces
### Example 1: Sandboxing with Copy-on-Write
### SftpFs
Afero has experimental support for secure file transfer protocol (sftp). Which can
be used to perform file operations over a encrypted channel.
### GCSFs
Afero has experimental support for Google Cloud Storage (GCS). You can either set the
`GOOGLE_APPLICATION_CREDENTIALS_JSON` env variable to your JSON credentials or use `opts` in
`NewGcsFS` to configure access to your GCS bucket.
Some known limitations of the existing implementation:
* No Chmod support - The GCS ACL could probably be mapped to *nix style permissions but that would add another level of complexity and is ignored in this version.
* No Chtimes support - Could be simulated with attributes (gcs a/m-times are set implicitly) but that's is left for another version.
* Not thread safe - Also assumes all file operations are done through the same instance of the GcsFs. File operations between different GcsFs instances are not guaranteed to be consistent.
## Filtering Backends
### BasePathFs
The BasePathFs restricts all operations to a given path within an Fs.
The given file name to the operations on this Fs will be prepended with
the base path before calling the source Fs.
Create a temporary environment where an application can "modify" system files without affecting the actual disk.
```go
bp := afero.NewBasePathFs(afero.NewOsFs(), "/base/path")
// 1. The base layer is the real OS, made read-only for safety.
baseFs := afero.NewReadOnlyFs(afero.NewOsFs())
// 2. The overlay layer is a temporary in-memory filesystem for changes.
overlayFs := afero.NewMemMapFs()
// 3. Combine them. Reads fall through to the base; writes only hit the overlay.
sandboxFs := afero.NewCopyOnWriteFs(baseFs, overlayFs)
// The application can now "modify" /etc/hosts, but the changes are isolated in memory.
afero.WriteFile(sandboxFs, "/etc/hosts", []byte("127.0.0.1 sandboxed-app"), 0644)
// The real /etc/hosts on disk is untouched.
```
### ReadOnlyFs
### Example 2: Caching a Slow Filesystem
A thin wrapper around the source Fs providing a read only view.
Improve performance by layering a fast cache (like memory) over a slow backend (like a network drive or cloud storage).
```go
fs := afero.NewReadOnlyFs(afero.NewOsFs())
_, err := fs.Create("/file.txt")
// err = syscall.EPERM
import "time"
// Assume 'remoteFs' is a slow backend (e.g., SFTP or GCS)
var remoteFs afero.Fs
// 'cacheFs' is a fast in-memory backend
cacheFs := afero.NewMemMapFs()
// Create the caching layer. Cache items for 5 minutes upon first read.
cachedFs := afero.NewCacheOnReadFs(remoteFs, cacheFs, 5*time.Minute)
// The first read is slow (fetches from remote, then caches)
data1, _ := afero.ReadFile(cachedFs, "data.json")
// The second read is instant (serves from memory cache)
data2, _ := afero.ReadFile(cachedFs, "data.json")
```
# RegexpFs
### Example 3: Security Jails (chroot)
A filtered view on file names, any file NOT matching
the passed regexp will be treated as non-existing.
Files not matching the regexp provided will not be created.
Directories are not filtered.
Restrict an application component's access to a specific subdirectory.
```go
fs := afero.NewRegexpFs(afero.NewMemMapFs(), regexp.MustCompile(`\.txt$`))
_, err := fs.Create("/file.html")
// err = syscall.ENOENT
osFs := afero.NewOsFs()
// Create a filesystem rooted at /home/user/public
// The application cannot access anything above this directory.
jailedFs := afero.NewBasePathFs(osFs, "/home/user/public")
// To the application, this is reading "/"
// In reality, it's reading "/home/user/public/"
dirInfo, err := afero.ReadDir(jailedFs, "/")
// Attempts to access parent directories fail
_, err = jailedFs.Open("../secrets.txt") // Returns an error
```
### HttpFs
## Real-World Use Cases
Afero provides an http compatible backend which can wrap any of the existing
backends.
### Build Cloud-Agnostic Applications
The Http package requires a slightly specific version of Open which
returns an http.File type.
Afero provides an httpFs file system which satisfies this requirement.
Any Afero FileSystem can be used as an httpFs.
Write applications that seamlessly work with different storage backends:
```go
httpFs := afero.NewHttpFs(<ExistingFS>)
fileserver := http.FileServer(httpFs.Dir(<PATH>))
http.Handle("/", fileserver)
type DocumentProcessor struct {
fs afero.Fs
}
func NewDocumentProcessor(fs afero.Fs) *DocumentProcessor {
return &DocumentProcessor{fs: fs}
}
func (p *DocumentProcessor) Process(inputPath, outputPath string) error {
// This code works whether fs is local disk, cloud storage, or memory
content, err := afero.ReadFile(p.fs, inputPath)
if err != nil {
return err
}
processed := processContent(content)
return afero.WriteFile(p.fs, outputPath, processed, 0644)
}
// Use with local filesystem
processor := NewDocumentProcessor(afero.NewOsFs())
// Use with Google Cloud Storage
processor := NewDocumentProcessor(gcsFS)
// Use with in-memory filesystem for testing
processor := NewDocumentProcessor(afero.NewMemMapFs())
```
## Composite Backends
### Treating Archives as Filesystems
Afero provides the ability have two filesystems (or more) act as a single
file system.
### CacheOnReadFs
The CacheOnReadFs will lazily make copies of any accessed files from the base
layer into the overlay. Subsequent reads will be pulled from the overlay
directly permitting the request is within the cache duration of when it was
created in the overlay.
If the base filesystem is writeable, any changes to files will be
done first to the base, then to the overlay layer. Write calls to open file
handles like `Write()` or `Truncate()` to the overlay first.
To writing files to the overlay only, you can use the overlay Fs directly (not
via the union Fs).
Cache files in the layer for the given time.Duration, a cache duration of 0
means "forever" meaning the file will not be re-requested from the base ever.
A read-only base will make the overlay also read-only but still copy files
from the base to the overlay when they're not present (or outdated) in the
caching layer.
Read files directly from `.zip` or `.tar` archives without unpacking them to disk first.
```go
base := afero.NewOsFs()
layer := afero.NewMemMapFs()
ufs := afero.NewCacheOnReadFs(base, layer, 100 * time.Second)
import (
"archive/zip"
"github.com/spf13/afero/zipfs"
)
// Assume 'zipReader' is a *zip.Reader initialized from a file or memory
var zipReader *zip.Reader
// Create a read-only ZipFs
archiveFS := zipfs.New(zipReader)
// Read a file from within the archive using the standard Afero API
content, err := afero.ReadFile(archiveFS, "/docs/readme.md")
```
### CopyOnWriteFs()
### Serving Any Filesystem over HTTP
The CopyOnWriteFs is a read only base file system with a potentially
writeable layer on top.
Read operations will first look in the overlay and if not found there, will
serve the file from the base.
Changes to the file system will only be made in the overlay.
Any attempt to modify a file found only in the base will copy the file to the
overlay layer before modification (including opening a file with a writable
handle).
Removing and Renaming files present only in the base layer is not currently
permitted. If a file is present in the base layer and the overlay, only the
overlay will be removed/renamed.
Use `HttpFs` to expose any Afero filesystem—even one created dynamically in memory—through a standard Go web server.
```go
base := afero.NewOsFs()
roBase := afero.NewReadOnlyFs(base)
ufs := afero.NewCopyOnWriteFs(roBase, afero.NewMemMapFs())
import (
"net/http"
"github.com/spf13/afero"
)
fh, _ = ufs.Create("/home/test/file2.txt")
fh.WriteString("This is a test")
fh.Close()
func main() {
memFS := afero.NewMemMapFs()
afero.WriteFile(memFS, "index.html", []byte("<h1>Hello from Memory!</h1>"), 0644)
// Wrap the memory filesystem to make it compatible with http.FileServer.
httpFS := afero.NewHttpFs(memFS)
http.Handle("/", http.FileServer(httpFS.Dir("/")))
http.ListenAndServe(":8080", nil)
}
```
In this example all write operations will only occur in memory (MemMapFs)
leaving the base filesystem (OsFs) untouched.
### Testing Made Simple
One of Afero's greatest strengths is making filesystem-dependent code easily testable:
## Desired/possible backends
```go
func SaveUserData(fs afero.Fs, userID string, data []byte) error {
filename := fmt.Sprintf("users/%s.json", userID)
return afero.WriteFile(fs, filename, data, 0644)
}
The following is a short list of possible backends we hope someone will
implement:
func TestSaveUserData(t *testing.T) {
// Create a clean, fast, in-memory filesystem for testing
testFS := afero.NewMemMapFs()
userData := []byte(`{"name": "John", "email": "john@example.com"}`)
err := SaveUserData(testFS, "123", userData)
if err != nil {
t.Fatalf("SaveUserData failed: %v", err)
}
// Verify the file was saved correctly
saved, err := afero.ReadFile(testFS, "users/123.json")
if err != nil {
t.Fatalf("Failed to read saved file: %v", err)
}
if string(saved) != string(userData) {
t.Errorf("Data mismatch: got %s, want %s", saved, userData)
}
}
```
* SSH
* S3
**Benefits of testing with Afero:**
- ⚡ **Fast** - No disk I/O, tests run in memory
- 🔄 **Reliable** - Each test starts with a clean slate
- 🧹 **No cleanup** - Memory is automatically freed
- 🔒 **Safe** - Can't accidentally modify real files
- 🏃 **Parallel** - Tests can run concurrently without conflicts
# About the project
## Backend Reference
## What's in the name
| Type | Backend | Constructor | Description | Status |
| :--- | :--- | :--- | :--- | :--- |
| **Core** | **OsFs** | `afero.NewOsFs()` | Interacts with the real operating system filesystem. Use in production. | ✅ Official |
| | **MemMapFs** | `afero.NewMemMapFs()` | A fast, atomic, concurrent-safe, in-memory filesystem. Ideal for testing. | ✅ Official |
| **Composition** | **CopyOnWriteFs**| `afero.NewCopyOnWriteFs(base, overlay)` | A read-only base with a writable overlay. Ideal for sandboxing. | ✅ Official |
| | **CacheOnReadFs**| `afero.NewCacheOnReadFs(base, cache, ttl)` | Lazily caches files from a slow base into a fast layer on first read. | ✅ Official |
| | **BasePathFs** | `afero.NewBasePathFs(source, path)` | Restricts operations to a subdirectory (chroot/jail). | ✅ Official |
| | **ReadOnlyFs** | `afero.NewReadOnlyFs(source)` | Provides a read-only view, preventing any modifications. | ✅ Official |
| | **RegexpFs** | `afero.NewRegexpFs(source, regexp)` | Filters a filesystem, only showing files that match a regex. | ✅ Official |
| **Utility** | **HttpFs** | `afero.NewHttpFs(source)` | Wraps any Afero filesystem to be served via `http.FileServer`. | ✅ Official |
| **Archives** | **ZipFs** | `zipfs.New(zipReader)` | Read-only access to files within a ZIP archive. | ✅ Official |
| | **TarFs** | `tarfs.New(tarReader)` | Read-only access to files within a TAR archive. | ✅ Official |
| **Network** | **GcsFs** | `gcsfs.NewGcsFs(...)` | Google Cloud Storage backend. | ⚡ Experimental |
| | **SftpFs** | `sftpfs.New(...)` | SFTP backend. | ⚡ Experimental |
| **3rd Party Cloud** | **S3Fs** | [`fclairamb/afero-s3`](https://github.com/fclairamb/afero-s3) | Production-ready S3 backend built on official AWS SDK. | 🔹 3rd Party |
| | **MinioFs** | [`cpyun/afero-minio`](https://github.com/cpyun/afero-minio) | MinIO object storage backend with S3 compatibility. | 🔹 3rd Party |
| | **DriveFs** | [`fclairamb/afero-gdrive`](https://github.com/fclairamb/afero-gdrive) | Google Drive backend with streaming support. | 🔹 3rd Party |
| | **DropboxFs** | [`fclairamb/afero-dropbox`](https://github.com/fclairamb/afero-dropbox) | Dropbox backend with streaming support. | 🔹 3rd Party |
| **3rd Party Specialized** | **GitFs** | [`tobiash/go-gitfs`](https://github.com/tobiash/go-gitfs) | Git repository filesystem (read-only, Afero compatible). | 🔹 3rd Party |
| | **DockerFs** | [`unmango/aferox`](https://github.com/unmango/aferox) | Docker container filesystem access. | 🔹 3rd Party |
| | **GitHubFs** | [`unmango/aferox`](https://github.com/unmango/aferox) | GitHub repository and releases filesystem. | 🔹 3rd Party |
| | **FilterFs** | [`unmango/aferox`](https://github.com/unmango/aferox) | Filesystem filtering with predicates. | 🔹 3rd Party |
| | **IgnoreFs** | [`unmango/aferox`](https://github.com/unmango/aferox) | .gitignore-aware filtering filesystem. | 🔹 3rd Party |
| | **FUSEFs** | [`JakWai01/sile-fystem`](https://github.com/JakWai01/sile-fystem) | Generic FUSE implementation using any Afero backend. | 🔹 3rd Party |
Afero comes from the latin roots Ad-Facere.
## Afero vs. `io/fs` (Go 1.16+)
**"Ad"** is a prefix meaning "to".
Go 1.16 introduced the `io/fs` package, which provides a standard abstraction for **read-only** filesystems.
**"Facere"** is a form of the root "faciō" making "make or do".
Afero complements `io/fs` by focusing on different needs:
The literal meaning of afero is "to make" or "to do" which seems very fitting
for a library that allows one to make files and directories and do things with them.
* **Use `io/fs` when:** You only need to read files and want to conform strictly to the standard library interfaces.
* **Use Afero when:**
* Your application needs to **create, write, modify, or delete** files.
* You need to test complex read/write interactions (e.g., renaming, concurrent writes).
* You need advanced compositional features (Copy-on-Write, Caching, etc.).
The English word that shares the same roots as Afero is "affair". Affair shares
the same concept but as a noun it means "something that is made or done" or "an
object of a particular type".
Afero is fully compatible with `io/fs`. You can wrap any Afero filesystem to satisfy the `fs.FS` interface using `afero.NewIOFS`:
It's also nice that unlike some of my other libraries (hugo, cobra, viper) it
Googles very well.
```go
import "io/fs"
## Release Notes
// Create an Afero filesystem (writable)
var myAferoFs afero.Fs = afero.NewMemMapFs()
See the [Releases Page](https://github.com/spf13/afero/releases).
// Convert it to a standard library fs.FS (read-only view)
var myIoFs fs.FS = afero.NewIOFS(myAferoFs)
```
## Third-Party Backends & Ecosystem
The Afero community has developed numerous backends and tools that extend the library's capabilities. Below are curated, well-maintained options organized by maturity and reliability.
### Featured Community Backends
These are mature, reliable backends that we can confidently recommend for production use:
#### **Amazon S3** - [`fclairamb/afero-s3`](https://github.com/fclairamb/afero-s3)
Production-ready S3 backend built on the official AWS SDK for Go.
```go
import "github.com/fclairamb/afero-s3"
s3fs := s3.NewFs(bucket, session)
```
#### **MinIO** - [`cpyun/afero-minio`](https://github.com/cpyun/afero-minio)
MinIO object storage backend providing S3-compatible object storage with deduplication and optimization features.
```go
import "github.com/cpyun/afero-minio"
minioFs := miniofs.NewMinioFs(ctx, "minio://endpoint/bucket")
```
### Community & Specialized Backends
#### Cloud Storage
- **Google Drive** - [`fclairamb/afero-gdrive`](https://github.com/fclairamb/afero-gdrive)
Streaming support; no write-seeking or POSIX permissions; no files listing cache
- **Dropbox** - [`fclairamb/afero-dropbox`](https://github.com/fclairamb/afero-dropbox)
Streaming support; no write-seeking or POSIX permissions
#### Version Control Systems
- **Git Repositories** - [`tobiash/go-gitfs`](https://github.com/tobiash/go-gitfs)
Read-only filesystem abstraction for Git repositories. Works with bare repositories and provides filesystem view of any git reference. Uses go-git for repository access.
#### Container and Remote Systems
- **Docker Containers** - [`unmango/aferox`](https://github.com/unmango/aferox)
Access Docker container filesystems as if they were local filesystems
- **GitHub API** - [`unmango/aferox`](https://github.com/unmango/aferox)
Turn GitHub repositories, releases, and assets into browsable filesystems
#### FUSE Integration
- **Generic FUSE** - [`JakWai01/sile-fystem`](https://github.com/JakWai01/sile-fystem)
Mount any Afero filesystem as a FUSE filesystem, allowing any Afero backend to be used as a real mounted filesystem
#### Specialized Filesystems
- **FAT32 Support** - [`aligator/GoFAT`](https://github.com/aligator/GoFAT)
Pure Go FAT filesystem implementation (currently read-only)
### Interface Adapters & Utilities
**Cross-Interface Compatibility:**
- [`jfontan/go-billy-desfacer`](https://github.com/jfontan/go-billy-desfacer) - Adapter between Afero and go-billy interfaces (for go-git compatibility)
- [`Maldris/go-billy-afero`](https://github.com/Maldris/go-billy-afero) - Alternative wrapper for using Afero with go-billy
- [`c4milo/afero2billy`](https://github.com/c4milo/afero2billy) - Another Afero to billy filesystem adapter
**Working Directory Management:**
- [`carolynvs/aferox`](https://github.com/carolynvs/aferox) - Working directory-aware filesystem wrapper
**Advanced Filtering:**
- [`unmango/aferox`](https://github.com/unmango/aferox) includes multiple specialized filesystems:
- **FilterFs** - Predicate-based file filtering
- **IgnoreFs** - .gitignore-aware filtering
- **WriterFs** - Dump writes to io.Writer for debugging
#### Developer Tools & Utilities
**nhatthm Utility Suite** - Essential tools for Afero development:
- [`nhatthm/aferocopy`](https://github.com/nhatthm/aferocopy) - Copy files between any Afero filesystems
- [`nhatthm/aferomock`](https://github.com/nhatthm/aferomock) - Mocking toolkit for testing
- [`nhatthm/aferoassert`](https://github.com/nhatthm/aferoassert) - Assertion helpers for filesystem testing
### Ecosystem Showcase
**Windows Virtual Drives** - [`balazsgrill/potatodrive`](https://github.com/balazsgrill/potatodrive)
Mount any Afero filesystem as a Windows drive letter. Brilliant demonstration of Afero's power!
### Modern Asset Embedding (Go 1.16+)
Instead of third-party tools, use Go's native `//go:embed` with Afero:
```go
import (
"embed"
"github.com/spf13/afero"
)
//go:embed assets/*
var assetsFS embed.FS
func main() {
// Convert embedded files to Afero filesystem
fs := afero.FromIOFS(assetsFS)
// Use like any other Afero filesystem
content, _ := afero.ReadFile(fs, "assets/config.json")
}
```
## Contributing
1. Fork it
We welcome contributions! The project is mature, but we are actively looking for contributors to help implement and stabilize network/cloud backends.
* 🔥 **Microsoft Azure Blob Storage**
* 🔒 **Modern Encryption Backend** - Built on secure, contemporary crypto (not legacy EncFS)
* 🐙 **Canonical go-git Adapter** - Unified solution for Git integration
* 📡 **SSH/SCP Backend** - Secure remote file operations
* Stabilization of existing experimental backends (GCS, SFTP)
To contribute:
1. Fork the repository
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request
5. Create a new Pull Request
## Contributors
## 📄 License
Names in no particular order:
Afero is released under the Apache 2.0 license. See [LICENSE.txt](https://github.com/spf13/afero/blob/master/LICENSE.txt) for details.
* [spf13](https://github.com/spf13)
* [jaqx0r](https://github.com/jaqx0r)
* [mbertschler](https://github.com/mbertschler)
* [xor-gate](https://github.com/xor-gate)
## 🔗 Additional Resources
## License
- [📖 Full API Documentation](https://pkg.go.dev/github.com/spf13/afero)
- [🎯 Examples Repository](https://github.com/spf13/afero/tree/master/examples)
- [📋 Release Notes](https://github.com/spf13/afero/releases)
- [❓ GitHub Discussions](https://github.com/spf13/afero/discussions)
Afero is released under the Apache 2.0 license. See
[LICENSE.txt](https://github.com/spf13/afero/blob/master/LICENSE.txt)
---
*Afero comes from the Latin roots Ad-Facere, meaning "to make" or "to do" - fitting for a library that empowers you to make and do amazing things with filesystems.*

View file

@ -34,7 +34,8 @@ func (u *CopyOnWriteFs) isBaseFile(name string) (bool, error) {
_, err := u.base.Stat(name)
if err != nil {
if oerr, ok := err.(*os.PathError); ok {
if oerr.Err == os.ErrNotExist || oerr.Err == syscall.ENOENT || oerr.Err == syscall.ENOTDIR {
if oerr.Err == os.ErrNotExist || oerr.Err == syscall.ENOENT ||
oerr.Err == syscall.ENOTDIR {
return false, nil
}
}
@ -237,7 +238,11 @@ func (u *CopyOnWriteFs) OpenFile(name string, flag int, perm os.FileMode) (File,
return u.layer.OpenFile(name, flag, perm)
}
return nil, &os.PathError{Op: "open", Path: name, Err: syscall.ENOTDIR} // ...or os.ErrNotExist?
return nil, &os.PathError{
Op: "open",
Path: name,
Err: syscall.ENOTDIR,
} // ...or os.ErrNotExist?
}
if b {
return u.base.OpenFile(name, flag, perm)

View file

@ -137,7 +137,7 @@ type readDirFile struct {
var _ fs.ReadDirFile = readDirFile{}
func (r readDirFile) ReadDir(n int) ([]fs.DirEntry, error) {
items, err := r.File.Readdir(n)
items, err := r.Readdir(n)
if err != nil {
return nil, err
}
@ -161,7 +161,12 @@ var _ Fs = FromIOFS{}
func (f FromIOFS) Create(name string) (File, error) { return nil, notImplemented("create", name) }
func (f FromIOFS) Mkdir(name string, perm os.FileMode) error { return notImplemented("mkdir", name) }
func (f FromIOFS) Mkdir(
name string,
perm os.FileMode,
) error {
return notImplemented("mkdir", name)
}
func (f FromIOFS) MkdirAll(path string, perm os.FileMode) error {
return notImplemented("mkdirall", path)

View file

@ -19,9 +19,9 @@ import (
// Lstater is an optional interface in Afero. It is only implemented by the
// filesystems saying so.
// It will call Lstat if the filesystem iself is, or it delegates to, the os filesystem.
// It will call Lstat if the filesystem itself is, or it delegates to, the os filesystem.
// Else it will call Stat.
// In addtion to the FileInfo, it will return a boolean telling whether Lstat was called or not.
// In addition to the FileInfo, it will return a boolean telling whether Lstat was called or not.
type Lstater interface {
LstatIfPossible(name string) (os.FileInfo, bool, error)
}

View file

@ -150,7 +150,11 @@ func (f *File) Sync() error {
func (f *File) Readdir(count int) (res []os.FileInfo, err error) {
if !f.fileData.dir {
return nil, &os.PathError{Op: "readdir", Path: f.fileData.name, Err: errors.New("not a dir")}
return nil, &os.PathError{
Op: "readdir",
Path: f.fileData.name,
Err: errors.New("not a dir"),
}
}
var outLength int64
@ -236,7 +240,11 @@ func (f *File) Truncate(size int64) error {
return ErrFileClosed
}
if f.readOnly {
return &os.PathError{Op: "truncate", Path: f.fileData.name, Err: errors.New("file handle is read only")}
return &os.PathError{
Op: "truncate",
Path: f.fileData.name,
Err: errors.New("file handle is read only"),
}
}
if size < 0 {
return ErrOutOfRange
@ -273,7 +281,11 @@ func (f *File) Write(b []byte) (n int, err error) {
return 0, ErrFileClosed
}
if f.readOnly {
return 0, &os.PathError{Op: "write", Path: f.fileData.name, Err: errors.New("file handle is read only")}
return 0, &os.PathError{
Op: "write",
Path: f.fileData.name,
Err: errors.New("file handle is read only"),
}
}
n = len(b)
cur := atomic.LoadInt64(&f.at)
@ -285,7 +297,9 @@ func (f *File) Write(b []byte) (n int, err error) {
tail = f.fileData.data[n+int(cur):]
}
if diff > 0 {
f.fileData.data = append(f.fileData.data, append(bytes.Repeat([]byte{0o0}, int(diff)), b...)...)
f.fileData.data = append(
f.fileData.data,
append(bytes.Repeat([]byte{0o0}, int(diff)), b...)...)
f.fileData.data = append(f.fileData.data, tail...)
} else {
f.fileData.data = append(f.fileData.data[:cur], b...)

View file

@ -92,7 +92,8 @@ func (f *UnionFile) Seek(o int64, w int) (pos int64, err error) {
func (f *UnionFile) Write(s []byte) (n int, err error) {
if f.Layer != nil {
n, err = f.Layer.Write(s)
if err == nil && f.Base != nil { // hmm, do we have fixed size files where a write may hit the EOF mark?
if err == nil &&
f.Base != nil { // hmm, do we have fixed size files where a write may hit the EOF mark?
_, err = f.Base.Write(s)
}
return n, err
@ -157,7 +158,7 @@ var defaultUnionMergeDirsFn = func(lofi, bofi []os.FileInfo) ([]os.FileInfo, err
// return a single view of the overlayed directories.
// At the end of the directory view, the error is io.EOF if c > 0.
func (f *UnionFile) Readdir(c int) (ofi []os.FileInfo, err error) {
var merge DirsMerger = f.Merger
merge := f.Merger
if merge == nil {
merge = defaultUnionMergeDirsFn
}

View file

@ -113,11 +113,11 @@ func GetTempDir(fs Fs, subPath string) string {
if subPath != "" {
// preserve windows backslash :-(
if FilePathSeparator == "\\" {
subPath = strings.Replace(subPath, "\\", "____", -1)
subPath = strings.ReplaceAll(subPath, "\\", "____")
}
dir = dir + UnicodeSanitize((subPath))
if FilePathSeparator == "\\" {
dir = strings.Replace(dir, "____", "\\", -1)
dir = strings.ReplaceAll(dir, "____", "\\")
}
if exists, _ := Exists(fs, dir); exists {

30
vendor/github.com/spf13/cast/map.go generated vendored
View file

@ -15,7 +15,7 @@ func toMapE[K comparable, V any](i any, keyFn func(any) K, valFn func(any) V) (m
m := map[K]V{}
if i == nil {
return nil, fmt.Errorf(errorMsg, i, i, m)
return m, fmt.Errorf(errorMsg, i, i, m)
}
switch v := i.(type) {
@ -45,14 +45,10 @@ func toMapE[K comparable, V any](i any, keyFn func(any) K, valFn func(any) V) (m
case string:
err := jsonStringToObject(v, &m)
if err != nil {
return nil, err
}
return m, nil
return m, err
default:
return nil, fmt.Errorf(errorMsg, i, i, m)
return m, fmt.Errorf(errorMsg, i, i, m)
}
}
@ -112,23 +108,19 @@ func ToStringMapStringSliceE(i any) (map[string][]string, error) {
for k, val := range v {
key, err := ToStringE(k)
if err != nil {
return nil, fmt.Errorf(errorMsg, i, i, m)
return m, fmt.Errorf(errorMsg, i, i, m)
}
value, err := ToStringSliceE(val)
if err != nil {
return nil, fmt.Errorf(errorMsg, i, i, m)
return m, fmt.Errorf(errorMsg, i, i, m)
}
m[key] = value
}
case string:
err := jsonStringToObject(v, &m)
if err != nil {
return nil, err
}
return m, nil
return m, err
default:
return nil, fmt.Errorf(errorMsg, i, i, m)
return m, fmt.Errorf(errorMsg, i, i, m)
}
return m, nil
@ -180,15 +172,11 @@ func toStringMapIntE[T int | int64](i any, fn func(any) T, fnE func(any) (T, err
case string:
err := jsonStringToObject(v, &m)
if err != nil {
return nil, err
}
return m, nil
return m, err
}
if reflect.TypeOf(i).Kind() != reflect.Map {
return nil, fmt.Errorf(errorMsg, i, i, m)
return m, fmt.Errorf(errorMsg, i, i, m)
}
mVal := reflect.ValueOf(m)

View file

@ -16,3 +16,6 @@ indent_style = tab
[*.nix]
indent_size = 2
[.golangci.yaml]
indent_size = 2

View file

@ -1,105 +1,118 @@
run:
timeout: 5m
version: "2"
linters-settings:
gci:
sections:
- standard
- default
- prefix(github.com/spf13/viper)
gocritic:
# Enable multiple checks by tags. See "Tags" section in https://github.com/go-critic/go-critic#usage.
enabled-tags:
- diagnostic
- experimental
- opinionated
- style
disabled-checks:
- importShadow
- unnamedResult
goimports:
local-prefixes: github.com/spf13/viper
run:
timeout: 5m
linters:
disable-all: true
enable:
- bodyclose
- dogsled
- dupl
- durationcheck
- exhaustive
- gci
- gocritic
- godot
- gofmt
- gofumpt
- goimports
- gomoddirectives
- goprintffuncname
- govet
- importas
- ineffassign
- makezero
- misspell
- nakedret
- nilerr
enable:
- bodyclose
- dogsled
- dupl
- durationcheck
- exhaustive
- gocritic
- godot
- gomoddirectives
- goprintffuncname
- govet
- importas
- ineffassign
- makezero
- misspell
- nakedret
- nilerr
- noctx
- nolintlint
- prealloc
- predeclared
- revive
- rowserrcheck
- sqlclosecheck
- staticcheck
- tparallel
- unconvert
- unparam
- unused
- wastedassign
- whitespace
# fixme
# - cyclop
# - errcheck
# - errorlint
# - exhaustivestruct
# - forbidigo
# - forcetypeassert
# - gochecknoglobals
# - gochecknoinits
# - gocognit
# - goconst
# - gocyclo
# - gosec
# - gosimple
# - ifshort
# - lll
# - nlreturn
# - paralleltest
# - scopelint
# - thelper
# - wrapcheck
# unused
# - depguard
# - goheader
# - gomodguard
# don't enable:
# - asciicheck
# - funlen
# - godox
# - goerr113
# - gomnd
# - interfacer
# - maligned
# - nestif
# - testpackage
# - wsl
exclusions:
rules:
- linters:
- errcheck
- noctx
- nolintlint
- prealloc
- predeclared
- revive
- rowserrcheck
- sqlclosecheck
- staticcheck
- stylecheck
- tparallel
- typecheck
- unconvert
- unparam
- unused
- wastedassign
- whitespace
path: _test.go
presets:
- comments
- std-error-handling
# fixme
# - cyclop
# - errcheck
# - errorlint
# - exhaustivestruct
# - forbidigo
# - forcetypeassert
# - gochecknoglobals
# - gochecknoinits
# - gocognit
# - goconst
# - gocyclo
# - gosec
# - gosimple
# - ifshort
# - lll
# - nlreturn
# - paralleltest
# - scopelint
# - thelper
# - wrapcheck
settings:
misspell:
locale: US
nolintlint:
allow-unused: false # report any unused nolint directives
require-specific: false # don't require nolint directives to be specific about which linter is being skipped
gocritic:
# Enable multiple checks by tags. See "Tags" section in https://github.com/go-critic/go-critic#usage.
enabled-tags:
- diagnostic
- experimental
- opinionated
- style
disabled-checks:
- importShadow
- unnamedResult
# unused
# - depguard
# - goheader
# - gomodguard
formatters:
enable:
- gci
- gofmt
- gofumpt
- goimports
# - golines
# deprecated
# - deadcode
# - structcheck
# - varcheck
# don't enable:
# - asciicheck
# - funlen
# - godox
# - goerr113
# - gomnd
# - interfacer
# - maligned
# - nestif
# - testpackage
# - wsl
settings:
gci:
sections:
- standard
- default
- localmodule

View file

@ -12,7 +12,7 @@
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/spf13/viper/ci.yaml?branch=master&style=flat-square)](https://github.com/spf13/viper/actions?query=workflow%3ACI)
[![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![Go Report Card](https://goreportcard.com/badge/github.com/spf13/viper?style=flat-square)](https://goreportcard.com/report/github.com/spf13/viper)
![Go Version](https://img.shields.io/badge/go%20version-%3E=1.21-61CFDD.svg?style=flat-square)
![Go Version](https://img.shields.io/badge/go%20version-%3E=1.23-61CFDD.svg?style=flat-square)
[![PkgGoDev](https://pkg.go.dev/badge/mod/github.com/spf13/viper)](https://pkg.go.dev/mod/github.com/spf13/viper)
**Go configuration with fangs!**
@ -821,7 +821,7 @@ You can use your favorite format's marshaller with the config returned by `AllSe
```go
import (
yaml "gopkg.in/yaml.v2"
yaml "go.yaml.in/yaml/v3"
// ...
)

View file

@ -83,6 +83,27 @@ v := viper.NewWithOptions(
)
```
### BREAKING: "github.com/mitchellh/mapstructure" depedency replaced
The original [mapstructure](https://github.com/mitchellh/mapstructure) has been [archived](https://github.com/mitchellh/mapstructure/issues/349) and was replaced with a [fork](https://github.com/go-viper/mapstructure) maintained by Viper ([#1723](https://github.com/spf13/viper/pull/1723)).
As a result, the package import path needs to be changed in cases where `mapstructure` is directly referenced in your code.
For example, when providing a custom decoder config:
```go
err := viper.Unmarshal(&appConfig, func(config *mapstructure.DecoderConfig) {
config.TagName = "yaml"
})
```
The change is fairly straightforward, just replace all occurrences of the import path `github.com/mitchellh/mapstructure` with `github.com/go-viper/mapstructure/v2`:
```diff
- import "github.com/mitchellh/mapstructure"
+ import "github.com/go-viper/mapstructure/v2"
```
### BREAKING: HCL, Java properties, INI removed from core
In order to reduce third-party dependencies, Viper dropped support for the following formats from the core:

View file

@ -2,30 +2,32 @@
"nodes": {
"cachix": {
"inputs": {
"devenv": "devenv_2",
"devenv": [
"devenv"
],
"flake-compat": [
"devenv"
],
"git-hooks": [
"devenv",
"flake-compat"
"git-hooks"
],
"nixpkgs": [
"devenv",
"nixpkgs"
],
"pre-commit-hooks": [
"devenv",
"pre-commit-hooks"
]
},
"locked": {
"lastModified": 1712055811,
"narHash": "sha256-7FcfMm5A/f02yyzuavJe06zLa9hcMHsagE28ADcmQvk=",
"lastModified": 1748883665,
"narHash": "sha256-R0W7uAg+BLoHjMRMQ8+oiSbTq8nkGz5RDpQ+ZfxxP3A=",
"owner": "cachix",
"repo": "cachix",
"rev": "02e38da89851ec7fec3356a5c04bc8349cae0e30",
"rev": "f707778d902af4d62d8dd92c269f8e70de09acbe",
"type": "github"
},
"original": {
"owner": "cachix",
"ref": "latest",
"repo": "cachix",
"type": "github"
}
@ -33,52 +35,21 @@
"devenv": {
"inputs": {
"cachix": "cachix",
"flake-compat": "flake-compat_2",
"nix": "nix_2",
"nixpkgs": "nixpkgs_2",
"pre-commit-hooks": "pre-commit-hooks"
},
"locked": {
"lastModified": 1724763216,
"narHash": "sha256-oW2bwCrJpIzibCNK6zfIDaIQw765yMAuMSG2gyZfGv0=",
"owner": "cachix",
"repo": "devenv",
"rev": "1e4ef61205b9aa20fe04bf1c468b6a316281c4f1",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "devenv",
"type": "github"
}
},
"devenv_2": {
"inputs": {
"flake-compat": [
"devenv",
"cachix",
"flake-compat"
],
"flake-compat": "flake-compat",
"git-hooks": "git-hooks",
"nix": "nix",
"nixpkgs": "nixpkgs",
"poetry2nix": "poetry2nix",
"pre-commit-hooks": [
"devenv",
"cachix",
"pre-commit-hooks"
]
"nixpkgs": "nixpkgs"
},
"locked": {
"lastModified": 1708704632,
"narHash": "sha256-w+dOIW60FKMaHI1q5714CSibk99JfYxm0CzTinYWr+Q=",
"lastModified": 1755257397,
"narHash": "sha256-VU+OHexL2y6y7yrpEc6bZvYYwoQg6aZK1b4YxT0yZCk=",
"owner": "cachix",
"repo": "devenv",
"rev": "2ee4450b0f4b95a1b90f2eb5ffea98b90e48c196",
"rev": "6f9c3d4722aa253631644329f7bda60b1d3d1b97",
"type": "github"
},
"original": {
"owner": "cachix",
"ref": "python-rewrite",
"repo": "devenv",
"type": "github"
}
@ -86,27 +57,11 @@
"flake-compat": {
"flake": false,
"locked": {
"lastModified": 1673956053,
"narHash": "sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm+504Ch3sNKLd8=",
"lastModified": 1747046372,
"narHash": "sha256-CIVLLkVgvHYbgI2UpXvIIBJ12HWgX+fjA8Xf8PUmqCY=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "35bb57c0c8d8b62bbfd284272c928ceb64ddbde9",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"flake-compat_2": {
"flake": false,
"locked": {
"lastModified": 1696426674,
"narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"rev": "9100a0f413b0c601e0533d1d94ffd501ce2e7885",
"type": "github"
},
"original": {
@ -116,15 +71,37 @@
}
},
"flake-parts": {
"inputs": {
"nixpkgs-lib": [
"devenv",
"nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1733312601,
"narHash": "sha256-4pDvzqnegAfRkPwO3wmwBhVi/Sye1mzps0zHWYnP88c=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "205b12d8b7cd4802fbcb8e8ef6a0f1408781a4f9",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "flake-parts",
"type": "github"
}
},
"flake-parts_2": {
"inputs": {
"nixpkgs-lib": "nixpkgs-lib"
},
"locked": {
"lastModified": 1722555600,
"narHash": "sha256-XOQkdLafnb/p9ij77byFQjDf5m5QYl9b2REiVClC+x4=",
"lastModified": 1754487366,
"narHash": "sha256-pHYj8gUBapuUzKV/kN/tR3Zvqc7o6gdFB9XKXIp1SQ8=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "8471fe90ad337a8074e957b69ca4d0089218391d",
"rev": "af66ad14b28a127c5c0f3bbb298218fc63528a18",
"type": "github"
},
"original": {
@ -133,39 +110,29 @@
"type": "github"
}
},
"flake-utils": {
"git-hooks": {
"inputs": {
"systems": "systems"
"flake-compat": [
"devenv",
"flake-compat"
],
"gitignore": "gitignore",
"nixpkgs": [
"devenv",
"nixpkgs"
]
},
"locked": {
"lastModified": 1689068808,
"narHash": "sha256-6ixXo3wt24N/melDWjq70UuHQLxGV8jZvooRanIHXw0=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "919d646de7be200f3bf08cb76ae1f09402b6f9b4",
"lastModified": 1750779888,
"narHash": "sha256-wibppH3g/E2lxU43ZQHC5yA/7kIKLGxVEnsnVK1BtRg=",
"owner": "cachix",
"repo": "git-hooks.nix",
"rev": "16ec914f6fb6f599ce988427d9d94efddf25fe6d",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_2": {
"inputs": {
"systems": "systems_2"
},
"locked": {
"lastModified": 1710146030,
"narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"owner": "cachix",
"repo": "git-hooks.nix",
"type": "github"
}
},
@ -173,7 +140,7 @@
"inputs": {
"nixpkgs": [
"devenv",
"pre-commit-hooks",
"git-hooks",
"nixpkgs"
]
},
@ -192,165 +159,49 @@
}
},
"nix": {
"inputs": {
"flake-compat": "flake-compat",
"nixpkgs": [
"devenv",
"cachix",
"devenv",
"nixpkgs"
],
"nixpkgs-regression": "nixpkgs-regression"
},
"locked": {
"lastModified": 1712911606,
"narHash": "sha256-BGvBhepCufsjcUkXnEEXhEVjwdJAwPglCC2+bInc794=",
"owner": "domenkozar",
"repo": "nix",
"rev": "b24a9318ea3f3600c1e24b4a00691ee912d4de12",
"type": "github"
},
"original": {
"owner": "domenkozar",
"ref": "devenv-2.21",
"repo": "nix",
"type": "github"
}
},
"nix-github-actions": {
"inputs": {
"nixpkgs": [
"devenv",
"cachix",
"devenv",
"poetry2nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1688870561,
"narHash": "sha256-4UYkifnPEw1nAzqqPOTL2MvWtm3sNGw1UTYTalkTcGY=",
"owner": "nix-community",
"repo": "nix-github-actions",
"rev": "165b1650b753316aa7f1787f3005a8d2da0f5301",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "nix-github-actions",
"type": "github"
}
},
"nix_2": {
"inputs": {
"flake-compat": [
"devenv",
"flake-compat"
],
"flake-parts": "flake-parts",
"git-hooks-nix": [
"devenv",
"git-hooks"
],
"nixpkgs": [
"devenv",
"nixpkgs"
],
"nixpkgs-regression": "nixpkgs-regression_2"
"nixpkgs-23-11": [
"devenv"
],
"nixpkgs-regression": [
"devenv"
]
},
"locked": {
"lastModified": 1712911606,
"narHash": "sha256-BGvBhepCufsjcUkXnEEXhEVjwdJAwPglCC2+bInc794=",
"owner": "domenkozar",
"lastModified": 1755029779,
"narHash": "sha256-3+GHIYGg4U9XKUN4rg473frIVNn8YD06bjwxKS1IPrU=",
"owner": "cachix",
"repo": "nix",
"rev": "b24a9318ea3f3600c1e24b4a00691ee912d4de12",
"rev": "b0972b0eee6726081d10b1199f54de6d2917f861",
"type": "github"
},
"original": {
"owner": "domenkozar",
"ref": "devenv-2.21",
"owner": "cachix",
"ref": "devenv-2.30",
"repo": "nix",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1692808169,
"narHash": "sha256-x9Opq06rIiwdwGeK2Ykj69dNc2IvUH1fY55Wm7atwrE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "9201b5ff357e781bf014d0330d18555695df7ba8",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs-lib": {
"locked": {
"lastModified": 1722555339,
"narHash": "sha256-uFf2QeW7eAHlYXuDktm9c25OxOyCoUOQmh5SZ9amE5Q=",
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz"
},
"original": {
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz"
}
},
"nixpkgs-regression": {
"locked": {
"lastModified": 1643052045,
"narHash": "sha256-uGJ0VXIhWKGXxkeNnq4TvV3CIOkUJ3PAoLZ3HMzNVMw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
}
},
"nixpkgs-regression_2": {
"locked": {
"lastModified": 1643052045,
"narHash": "sha256-uGJ0VXIhWKGXxkeNnq4TvV3CIOkUJ3PAoLZ3HMzNVMw=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
},
"original": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "215d4d0fd80ca5163643b03a33fde804a29cc1e2",
"type": "github"
}
},
"nixpkgs-stable": {
"locked": {
"lastModified": 1710695816,
"narHash": "sha256-3Eh7fhEID17pv9ZxrPwCLfqXnYP006RKzSs0JptsN84=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "614b4613980a522ba49f0d194531beddbb7220d3",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.11",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1713361204,
"narHash": "sha256-TA6EDunWTkc5FvDCqU3W2T3SFn0gRZqh6D/hJnM02MM=",
"lastModified": 1750441195,
"narHash": "sha256-yke+pm+MdgRb6c0dPt8MgDhv7fcBbdjmv1ZceNTyzKg=",
"owner": "cachix",
"repo": "devenv-nixpkgs",
"rev": "285676e87ad9f0ca23d8714a6ab61e7e027020c6",
"rev": "0ceffe312871b443929ff3006960d29b120dc627",
"type": "github"
},
"original": {
@ -360,13 +211,28 @@
"type": "github"
}
},
"nixpkgs_3": {
"nixpkgs-lib": {
"locked": {
"lastModified": 1724748588,
"narHash": "sha256-NlpGA4+AIf1dKNq76ps90rxowlFXUsV9x7vK/mN37JM=",
"lastModified": 1753579242,
"narHash": "sha256-zvaMGVn14/Zz8hnp4VWT9xVnhc8vuL3TStRqwk22biA=",
"owner": "nix-community",
"repo": "nixpkgs.lib",
"rev": "0f36c44e01a6129be94e3ade315a5883f0228a6e",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "nixpkgs.lib",
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1755268003,
"narHash": "sha256-nNaeJjo861wFR0tjHDyCnHs1rbRtrMgxAKMoig9Sj/w=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a6292e34000dc93d43bccf78338770c1c5ec8a99",
"rev": "32f313e49e42f715491e1ea7b306a87c16fe0388",
"type": "github"
},
"original": {
@ -376,94 +242,11 @@
"type": "github"
}
},
"poetry2nix": {
"inputs": {
"flake-utils": "flake-utils",
"nix-github-actions": "nix-github-actions",
"nixpkgs": [
"devenv",
"cachix",
"devenv",
"nixpkgs"
]
},
"locked": {
"lastModified": 1692876271,
"narHash": "sha256-IXfZEkI0Mal5y1jr6IRWMqK8GW2/f28xJenZIPQqkY0=",
"owner": "nix-community",
"repo": "poetry2nix",
"rev": "d5006be9c2c2417dafb2e2e5034d83fabd207ee3",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "poetry2nix",
"type": "github"
}
},
"pre-commit-hooks": {
"inputs": {
"flake-compat": [
"devenv",
"flake-compat"
],
"flake-utils": "flake-utils_2",
"gitignore": "gitignore",
"nixpkgs": [
"devenv",
"nixpkgs"
],
"nixpkgs-stable": "nixpkgs-stable"
},
"locked": {
"lastModified": 1713775815,
"narHash": "sha256-Wu9cdYTnGQQwtT20QQMg7jzkANKQjwBD9iccfGKkfls=",
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"rev": "2ac4dcbf55ed43f3be0bae15e181f08a57af24a4",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"type": "github"
}
},
"root": {
"inputs": {
"devenv": "devenv",
"flake-parts": "flake-parts",
"nixpkgs": "nixpkgs_3"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"systems_2": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
"flake-parts": "flake-parts_2",
"nixpkgs": "nixpkgs_2"
}
}
},

View file

@ -7,51 +7,55 @@
devenv.url = "github:cachix/devenv";
};
outputs = inputs@{ flake-parts, ... }:
outputs =
inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } {
imports = [
inputs.devenv.flakeModule
];
systems = [ "x86_64-linux" "x86_64-darwin" "aarch64-darwin" ];
systems = [
"x86_64-linux"
"x86_64-darwin"
"aarch64-darwin"
];
perSystem = { config, self', inputs', pkgs, system, ... }: rec {
devenv.shells = {
default = {
languages = {
go.enable = true;
go.package = pkgs.go_1_23;
};
perSystem =
{ pkgs, ... }:
{
devenv.shells = {
default = {
languages = {
go.enable = true;
};
pre-commit.hooks = {
nixpkgs-fmt.enable = true;
yamllint.enable = true;
};
git-hooks.hooks = {
nixpkgs-fmt.enable = true;
yamllint.enable = true;
};
packages = with pkgs; [
gnumake
packages = with pkgs; [
gnumake
golangci-lint
yamllint
];
golangci-lint
yamllint
];
scripts = {
versions.exec = ''
go version
golangci-lint version
scripts = {
versions.exec = ''
go version
golangci-lint version
'';
};
enterShell = ''
versions
'';
# https://github.com/cachix/devenv/issues/528#issuecomment-1556108767
containers = pkgs.lib.mkForce { };
};
enterShell = ''
versions
'';
# https://github.com/cachix/devenv/issues/528#issuecomment-1556108767
containers = pkgs.lib.mkForce { };
};
ci = devenv.shells.default;
};
};
};
}

View file

@ -1,6 +1,6 @@
package yaml
import "gopkg.in/yaml.v3"
import "go.yaml.in/yaml/v3"
// Codec implements the encoding.Encoder and encoding.Decoder interfaces for YAML encoding.
type Codec struct{}

View file

@ -219,7 +219,10 @@ func (v *Viper) watchKeyValueConfigOnChannel() error {
for {
b := <-rc
reader := bytes.NewReader(b.Value)
v.unmarshalReader(reader, v.kvstore)
err := v.unmarshalReader(reader, v.kvstore)
if err != nil {
v.logger.Error(fmt.Errorf("failed to unmarshal remote config: %w", err).Error())
}
}
}(respc)
return nil

View file

@ -174,10 +174,7 @@ func parseSizeInBytes(sizeStr string) uint {
}
}
size := cast.ToInt(sizeStr)
if size < 0 {
size = 0
}
size := max(cast.ToInt(sizeStr), 0)
return safeMul(uint(size), multiplier)
}

View file

@ -376,7 +376,12 @@ func (v *Viper) WatchConfig() {
}
}
}()
watcher.Add(configDir)
err = watcher.Add(configDir)
if err != nil {
v.logger.Error(fmt.Sprintf("failed to add watcher: %s", err))
initWG.Done()
return
}
initWG.Done() // done initializing the watch in this go routine, so the parent routine can move on...
eventsWG.Wait() // now, wait for event loop to end in this go-routine...
}()
@ -1181,11 +1186,26 @@ func (v *Viper) find(lcaseKey string, flagDefault bool) any {
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return res
case "boolSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToBoolSlice(res)
case "intSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToIntSlice(res)
case "uintSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToUintSlice(res)
case "float64Slice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToFloat64Slice(res)
case "durationSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
@ -1268,11 +1288,26 @@ func (v *Viper) find(lcaseKey string, flagDefault bool) any {
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return res
case "boolSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToBoolSlice(res)
case "intSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToIntSlice(res)
case "uintSlice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToUintSlice(res)
case "float64Slice":
s := strings.TrimPrefix(flag.ValueString(), "[")
s = strings.TrimSuffix(s, "]")
res, _ := readAsCSV(s)
return cast.ToFloat64Slice(res)
case "stringToString":
return stringToStringConv(flag.ValueString())
case "stringToInt":
@ -1670,7 +1705,10 @@ func (v *Viper) unmarshalReader(in io.Reader, c map[string]any) error {
}
buf := new(bytes.Buffer)
buf.ReadFrom(in)
_, err := buf.ReadFrom(in)
if err != nil {
return fmt.Errorf("failed to read configuration from input: %w", err)
}
// TODO: remove this once SupportedExts is deprecated/removed
if !slices.Contains(SupportedExts, format) {