after seeing a potential reported federating worker lockup i decided to start digging into the federatingdb code. this PR encompasses:
- removes one of our last unused interface types `federatingdb.DB{}`, replacing it with a struct type `*federatingdb.DB{}`
- in `transport.dereferenceLocal()` differentiates between an unsupported lookup type and ErrNoEntries to reduce unnecessary calls, and reduce potential lockups that may occur while trying to call our own endpoints that then call `federatingdb.Lock()`
- removes a bunch of the locks on follow state changes since the DB already synchronizes that
- removes the unnecessary `pub.Clock{}` struct field and type passed to the transport controller
frankly it would be great if we could remove the locking in `federatingdb.Lock()` and instead handle it ourselves as it gets very confusing trying to figure out what functions will have locks held. but i guess that's one for when we move further away from the go-fed/activity/pub package usage.
Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4178
Co-authored-by: kim <grufwub@gmail.com>
Co-committed-by: kim <grufwub@gmail.com>
# Description
> If this is a code change, please include a summary of what you've coded, and link to the issue(s) it closes/implements.
>
> If this is a documentation change, please briefly describe what you've changed and why.
Previously we were passing a *followers* URI into the federatingDB `Followers` function, which expects an *account* URI. This led to a nil body when doing internal dereferences (ie., while sending out messages). I think I probably wrote this bug when I refactored stuff a while ago.
This PR fixes that up by making the dereferenceLocal function more consistent, and falling back to an http dereference if nothing found. Clarifies behavior of the federatingdb.Get function as well, by commenting the shit out of it.
Also adds tests, precious tests.
Closes https://codeberg.org/superseriousbusiness/gotosocial/issues/4066
## Checklist
Please put an x inside each checkbox to indicate that you've read and followed it: `[ ]` -> `[x]`
If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want).
- [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md).
- [x] I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat.
- [x] I/we have not leveraged AI to create the proposed changes.
- [x] I/we have performed a self-review of added code.
- [x] I/we have written code that is legible and maintainable by others.
- [x] I/we have commented the added code, particularly in hard-to-understand areas.
- [x] I/we have made any necessary changes to documentation.
- [x] I/we have added tests that cover new code.
- [x] I/we have run tests and they pass locally with the changes.
- [x] I/we have run `go fmt ./...` and `golangci-lint run`.
Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4097
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
Co-committed-by: tobi <tobi.smethurst@protonmail.com>
* improved federatingdb logging in cases of unknown iri / types, add new log methods
* whoops; forgot to wrap log argument in serialize{} !
* use debug instead of warn level
* switch last entry to Debug
* [feature] Federate interaction policies + Accepts; enforce policies
* use Acceptable type
* fix index
* remove appendIRIStrs
* add GetAccept federatingdb function
* lock on object IRI
* update typeconverter to use state structure
* deinterface the typeutils.TypeConverter -> typeutils.Converter
* finish copying over old type converter code comments
* fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
* [chore] Remove years from all license headers
Years or year ranges aren't required in license headers. Many projects
have removed them in recent years and it avoids a bit of yearly toil.
In many cases our copyright claim was also a bit dodgy since we added
the 2021-2023 header to files created after 2021 but you can't claim
copyright into the past that way.
* [chore] Add license header check
This ensures a license header is always added to any new file. This
avoids maintainers/reviewers needing to remember to check for and ask
for it in case a contribution doesn't include it.
* [chore] Add missing license headers
* [chore] Further updates to license header
* Use the more common // indentend comment format
* Remove the hack we had for the linter now that we use the // format
* Add SPDX license identifier
This adds a lightweight form of tracing to GTS. Each incoming request is
assigned a Request ID which we then pass on and log in all our log
lines. Any function that gets called downstream from an HTTP handler
should now emit a requestID=value pair whenever it logs something.
Co-authored-by: kim <grufwub@gmail.com>
* first commit
Signed-off-by: kim <grufwub@gmail.com>
* replace logging with our own log library
Signed-off-by: kim <grufwub@gmail.com>
* fix imports
Signed-off-by: kim <grufwub@gmail.com>
* fix log imports
Signed-off-by: kim <grufwub@gmail.com>
* add license text
Signed-off-by: kim <grufwub@gmail.com>
* fix package import cycle between config and log package
Signed-off-by: kim <grufwub@gmail.com>
* fix empty kv.Fields{} being passed to WithFields()
Signed-off-by: kim <grufwub@gmail.com>
* fix uses of log.WithFields() with whitespace issues and empty slices
Signed-off-by: kim <grufwub@gmail.com>
* *linter related grumbling*
Signed-off-by: kim <grufwub@gmail.com>
* gofmt the codebase! also fix more log.WithFields() formatting issues
Signed-off-by: kim <grufwub@gmail.com>
* update testrig code to match new changes
Signed-off-by: kim <grufwub@gmail.com>
* fix error wrapping in non fmt.Errorf function
Signed-off-by: kim <grufwub@gmail.com>
* add benchmarking of log.Caller() vs non-cached
Signed-off-by: kim <grufwub@gmail.com>
* fix syslog tests, add standard build tags to test runner to ensure consistency
Signed-off-by: kim <grufwub@gmail.com>
* make syslog tests more robust
Signed-off-by: kim <grufwub@gmail.com>
* fix caller depth arithmatic (is that how you spell it?)
Signed-off-by: kim <grufwub@gmail.com>
* update to use unkeyed fields in kv.Field{} instances
Signed-off-by: kim <grufwub@gmail.com>
* update go-kv library
Signed-off-by: kim <grufwub@gmail.com>
* update libraries list
Signed-off-by: kim <grufwub@gmail.com>
* fuck you linter get nerfed
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
* reference logrus' global logger instead of passing and storing a logger reference everywhere
* always directly use global logrus logger instead of referencing an instance
* test suites should also directly use the global logrus logger
* rename gin logging function to clarify that it's middleware
* correct comments which erroneously referenced removed logger parameter
* setting log level for tests now uses logrus' exported type instead of the string value, to guarantee error isn't possible
* tidy up streaming
* cut down code duplication
* test get followers/following
* test streaming processor
* fix some test models
* add TimeMustParse
* fix uri / url typo
* make trace logging less verbose
* make logging more consistent
* disable quote on logging
* remove context.Background
* remove many extraneous mastodon references
* regenerate swagger
* don't log query on no rows result
* log latency first for easier reading
* start moving some database stuff around
* continue moving db stuff around
* more fiddling
* more updates
* and some more
* and yet more
* i broke SOMETHING but what, it's a mystery
* tidy up
* vendor ttlcache
* use ttlcache
* fix up some tests
* rename some stuff
* little reminder
* some more updates
