* add warning message when wazero compiler not supported, update supported platforms in README
* whoops don't return a reason string for arm64, since it should always be supported
* peepeepoopoo
* test domain perm subs
* swagger
* envparsing
* dries your wets
* start on docs
* finish up docs
* copy paste errors
* rename actions package
* rename force -> skipCache
* move obfuscate parse nearer to where err is checked
* make higherPrios a simple slice
* don't use receiver for permsFrom funcs
* add more context to error logs
* defer finished log
* use switch for permType instead of if/else
* thanks linter, love you <3
* validate csv headers before full read
* use bufio scanner
This will be either an mp1, mp2 or mp3 file. In practice it'll probably
be mp3, but this handles mp1 too for good measure. We don't advertise
audio/mp1 as a supported media type since best I can tell that was never
a MIME type that's been used.
This also changes the returned MIME-type for mp2 and mp3 to audio/mpeg,
to match what's expected and supported by most things nowadays.
Fixes: #3531
* add support for extracting Updated field from Statusable implementers
* add support for status edits in the database, and update status dereferencer to handle them
* remove unused AdditionalInfo{}.CreatedAt
* remove unused AdditionalEmojiInfo{}.CreatedAt
* update new mention creation to use status.UpdatedAt
* remove mention.UpdatedAt, fixes related to NewULIDFromTime() change
* add migration to remove Mention{}.UpdatedAt field
* add migration to add the StatusEdit{} table
* start adding tests, add delete function for status edits
* add more of status edit migrations, fill in more of the necessary edit delete functionality
* remove unused function
* allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid`
* add StatusEdit{} test models
* fix new statusedits sql
* use model instead of table name
* actually remove the Mention.UpdatedAt field...
* fix tests now new models are added, add more status edit DB tests
* fix panic wording
* add test for deleting status edits
* don't automatically set `updated_at` field on updated statuses
* flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses
* remove media_attachments.updated_at column
* fix up more tests, further complete the dereferencer status edit tests
* update more status serialization tests not expecting 'updated' AS property
* gah!! json serialization tests!!
* undo some gtscontext wrapping changes
* more serialization test fixing 🥲
* more test fixing, ensure the edit.status_id field is actually set 🤦
* fix status edit test
* grrr linter
* add edited_at field to apimodel status
* remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...)
* ensure that status.updated_at always fits chronologically
* fix more serialization tests ...
* add more code comments
* fix envparsing
* update swagger file
* properly handle media description changes during status edits
* slight formatting tweak
* code comment
* determine mime-type to use during ffprobe evaluation stage, don't bother rechecking by file extension
* set mjpeg content-type
* fix up tests expecting differing default values
* [chore] Bump tooling versions, bump go -> v1.23.0
* undo silly change
* sign
* bump go version in go.mod
* allow overflow in imaging
* goreleaser deprecation notices
* [chore] Upgrade golangci-lint, ignore existing int overflow warnings
There is a new lint for unchecked int casts. Integer overflows are bad,
but the old code that triggers this lint seems to be perfectly fine.
Instead of disabling the lint entirely for new code as well, grandfather
in existing code.
* fix golangci-lint documentation link
* revert unrelated changes
* revert another unrelated change
* get rid of remaining nolint:gosec
* swagger updates
* apply review feedback
* fix wrong formatting specifier thing
* fix the linter for real
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* update go-storage dependency, for S3Storage manually call PutObject() so we can set content-type
* update calls to PutFile() to include the contentType
* use single-threaded image resizing in native code so we have more control over goroutines
* implement parallel-free versions of image transform functions also
* remove debug code
* use a single instance of wazero runtime and compiled modules
* remove test output 🤦
* undo process-{media,emoji} changes
* update test runner to include wazero compilation cache
* sign drone.yml
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* wrap thumbnailing code to handle generation natively where possible
* more code comments!
* add even more code comments!
* add code comments about blurhash generation
* maintain image rotation if contained in exif data
* move rotation before resizing
* ensure pix_fmt actually selected by ffprobe, check for alpha layer with gifs
* use linear instead of nearest-neighbour for resizing
* work with image "orientation" instead of "rotation". use default 75% quality for both webp and jpeg generation
* add header to new file
* use thumb extension when getting thumb mime type
* update test models and tests with new media processing
* add suggested code comments
* add note about thumbnail filter count reducing memory usage
* add test for emoji update image
* update emoji recache to set the instance account id
* don't refresh emoji if only not cached. in that case literally just recache
* code comment
* rename + move a few things
* add some more code comments, and rename some functions to make logic a bit clearer
* remove unnecessary nil check (the value can be nil)
* comment wording
* remove test data output
* handle the case of caching an emoji which has been refreshed then uncached
* allow overwriting on testrig storage as we do now on regular storage
* fix emoji category ID not getting updated
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* add back exif-terminator and use only for jpeg,png,webp
* fix arguments passed to terminateExif()
* pull in latest exif-terminator
* fix test
* update processed img
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* take into account rotation when generating thumbnail, simplify ffprobe output to show only fields we need
* only show rotation side data
* remove unnecessary comment
* fix code comments
* remove debug logging
* refactor file handling a tiny bit
* whoops
* make processing media / emoji defers a bit clear to see that it's the "on finished processing" path
* some wording
* add some debug logging
* add mutex locks for processing remote media
* try removing freshness check
* fix derefMedia not being allocated
* fix log format string
* handle case of empty file paths (i.e. not stored)
* remove media / emoji once finished processing from dereferencer maps
* whoops, fix the cached / force checks
* move url parsing outside of 'process___Safely()' funcs to prevalidate url
* use emoji.ShortcodeDomain()
* update RefreshEmoji() to also match RefreshMedia() changes
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* add flac support to the ffprobe format/stream -> filetype parser
* also add audio/flac for flac (not just x-flac)
* update tests
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* update to use webp for thumbnails
* bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting)
* update to use yuva colorspace, and use thumbnail=n=10 to select frame
* fix missing comma in ffmpeg args
* add links to appropriate ffmpeg docs
* update tests
* add file size tests for thumbnails
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* don't set emoji / media image paths on failed download, migrate FileType from string to integer
* fix incorrect uses of util.PtrOr, fix returned frontend media
* fix migration not setting arguments correctly in where clause
* fix not providing default with not null column
* whoops
* ensure a default gets set for media attachment file type
* remove the exclusive flag from writing files in disk storage
* rename PtrOr -> PtrOrZero, and rename PtrValueOr -> PtrOrValue to match
* slight wording changes
* use singular / plural word forms (no parentheses), is better for screen readers
* update testmodels with unknown media type to have unset file details, update attachment focus handling converting to frontend, update tests
* store first instance in ffmpeg wasm pool, fill remaining with closed instances
* add more supported file types to our media processor that ffmpeg supports, update supported mime type lists
* add code comments to the supported mime types slice
* don't check for zero value string, just parse
* remove some unneeded consts which make the code a bit harder to read
* fix test expected instance media mime types, use compact ffprobe json, simple media processing by type
* final tweaks to media processing code
* don't use safe divide where we don't need to
* [feature/frontend] Audio player for audio media types
* use video preview images for previews instead of video itself
* don't preload
* update tests for new zork status
* collapse media gallery into single row when small
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg
* specify the video codec to use when generating static image from emoji
* update go-storage library (fixes incompatibility after updating go-iotools)
* maintain image aspect ratio when generating a thumbnail for it
* update readme to show go-ffmpreg
* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility
* remove extra debug from error message
* fix up incorrect function signatures
* update PutFile to just use regular file copy, as changes are file is on separate partition
* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe
* update more tests, add more code comments
* add utilities to generate processed emoji / media outputs
* fix remaining tests
* add test for opus media file, add license header to utility cmds
* limit the number of concurrently available ffmpeg / ffprobe instances
* reduce number of instances
* further reduce number of instances
* fix envparsing test with configuration variables
* update docs and configuration with new media-{local,remote}-max-size variables
