Commit graph

14 commits

Author SHA1 Message Date
kim
e3dfd88893 [performance] bump codeberg.org/gruf/go-kv to v2 (#4341)
updates our codeberg.org/gruf/go-kv log key-value formatting library to latest version, which comes with some maaaaaaajor speed boosts in the form of:
- very minimal reflect.Value{} usage
- caching prepared formatting functions per type

~~still a work-in-progress until i make a release tag on the go-kv repository, which itself is waiting on published benchmark results in the README and finishing writing some code comments~~

benchmarks so far show this to be ~3x faster than the "fmt" stdlib package on average, when run across a wide variety (106 different types) of test cases, while still creating more visually friendly log output and actually recursing down nested struct ptrs

Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4341
Co-authored-by: kim <grufwub@gmail.com>
Co-committed-by: kim <grufwub@gmail.com>
2025-07-29 09:23:20 +02:00
kim
c8a4ce9a88 [performance] use our own typed value context types for Value() key checking to improve performance (#4316)
Replaces our gtscontext package context.Context handling with our own typed contexts instead of `context.WithValue()`.

I wrote a quick benchmark consisting of (printlns to stop the compiler optimizing instructions away):
```golang
func BenchmarkContexts(b *testing.B) {
	var receiving *gtsmodel.Account
	var requesting *gtsmodel.Account
	var otherIRIs []*url.URL

	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			ctx := context.Background()

			ctx = gtscontext.SetBarebones(ctx)
			ctx = gtscontext.SetFastFail(ctx)
			ctx = gtscontext.SetDryRun(ctx)
			ctx = gtscontext.SetReceivingAccount(ctx, receiving)
			ctx = gtscontext.SetRequestingAccount(ctx, requesting)
			ctx = gtscontext.SetOtherIRIs(ctx, otherIRIs)

			if !gtscontext.Barebones(ctx) {
				println("oh no!")
			}

			if !gtscontext.IsFastfail(ctx) {
				println("oh no!")
			}

			if !gtscontext.DryRun(ctx) {
				println("oh no!")
			}

			if gtscontext.ReceivingAccount(ctx) != nil {
				println("oh no!")
			}

			if gtscontext.RequestingAccount(ctx) != nil {
				println("oh no!")
			}

			if len(gtscontext.OtherIRIs(ctx)) > 0 {
				println("oh no!")
			}
		}
	})
}
```

Before results:
```shell
kim @ ~/Projects/main/gts.4
--> go test -v -run=none -bench=.* -benchmem ./internal/gtscontext/ -count=5
goos: linux
goarch: amd64
pkg: code.superseriousbusiness.org/gotosocial/internal/gtscontext
cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
BenchmarkContexts
BenchmarkContexts-16            19050348                61.73 ns/op          288 B/op          6 allocs/op
BenchmarkContexts-16            18245772                61.71 ns/op          288 B/op          6 allocs/op
BenchmarkContexts-16            18853680                61.80 ns/op          288 B/op          6 allocs/op
BenchmarkContexts-16            18561621                62.67 ns/op          288 B/op          6 allocs/op
BenchmarkContexts-16            17819241                62.89 ns/op          288 B/op          6 allocs/op
PASS
ok      code.superseriousbusiness.org/gotosocial/internal/gtscontext    6.112s
```

After results:
```shell
kim @ ~/Projects/main/gts.4
--> go test -v -run=none -bench=.* -benchmem ./internal/gtscontext/ -count=5
goos: linux
goarch: amd64
pkg: code.superseriousbusiness.org/gotosocial/internal/gtscontext
cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
BenchmarkContexts
BenchmarkContexts-16            28038618                41.67 ns/op          144 B/op          6 allocs/op
BenchmarkContexts-16            26537552                42.50 ns/op          144 B/op          6 allocs/op
BenchmarkContexts-16            26720542                42.39 ns/op          144 B/op          6 allocs/op
BenchmarkContexts-16            27408031                43.15 ns/op          144 B/op          6 allocs/op
BenchmarkContexts-16            25597026                44.02 ns/op          144 B/op          6 allocs/op
PASS
ok      code.superseriousbusiness.org/gotosocial/internal/gtscontext    5.997s
```

Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4316
Co-authored-by: kim <grufwub@gmail.com>
Co-committed-by: kim <grufwub@gmail.com>
2025-07-09 01:38:33 +02:00
Daenney
ab10266bff [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
tobi
ffde1b150f
[chore] Move deps to code.superseriousbusiness.org (#4054) 2025-04-25 15:15:36 +02:00
tobi
5d0e3d9c35
[chore] github.com/superseriousbusiness/httpsig -> codeberg.org/superseriousbusiness/httpsig (#3854) 2025-03-02 13:28:38 +01:00
kim
d61d5c8a6a
[bugfix] httpclient not signing subsequent redirect requests (#2798)
* move http request signing to transport

* actually hook up the http roundtripper ...

* add code comments for the new gtscontext functions
2024-04-02 13:12:26 +02:00
Milas Bowman
af1a26a68f
[feature] Add Mastodon-compatible HTTP signature fallback (#2659)
On outgoing `GET` requests that are signed (e.g. authorized fetch),
if the initial request fails with `401`, try again, but _without_
the query parameters included in the HTTP signature.

This is primarily useful for compatibility with Mastodon; though
hopefully this can be removed in the not-too-distant future, as
they've started changing their behavior here.

Signed-off-by: Milas Bowman <devnull@milas.dev>
2024-02-19 11:18:17 +01:00
tobi
b614d33c40
[feature] Try HTTP signature validation with and without query params for incoming requests (#2591)
* [feature] Verify signatures both with + without query params

* Bump to tagged version
2024-01-31 14:15:28 +00:00
kim
e9c3663cce
[chore] ensure worker contexts have request ID (#2120) 2023-08-15 17:01:01 +01:00
kim
9a22102fa8
[bugfix/chore] oauth entropy fix + media cleanup tasks rewrite (#1853) 2023-06-22 20:46:36 +01:00
tobi
24fbdf2b0a
[chore] Refactor AP authentication, other small bits of tidying up (#1874) 2023-06-13 15:47:56 +01:00
Daenney
107237c8e8
[feature] Make client IP logging configurable (#1799) 2023-05-21 16:12:47 +01:00
kim
6a29c5ffd4
[performance] improved request batching (removes need for queueing) (#1687)
* revamp http client to not limit requests, instead use sender worker

Signed-off-by: kim <grufwub@gmail.com>

* remove separate sender worker pool, spawn 2*GOMAXPROCS batch senders each time, no need for transport cache sweeping

Signed-off-by: kim <grufwub@gmail.com>

* improve batch senders to keep popping recipients until remote URL found

Signed-off-by: kim <grufwub@gmail.com>

* fix recipient looping issue

Signed-off-by: kim <grufwub@gmail.com>

* fix missing mutex unlock

Signed-off-by: kim <grufwub@gmail.com>

* move request id ctx key to gtscontext, finish filling out more code comments, add basic support for not logging client IP

Signed-off-by: kim <grufwub@gmail.com>

* slight code reformatting

Signed-off-by: kim <grufwub@gmail.com>

* a whitespace

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code

Signed-off-by: kim <grufwub@gmail.com>

* add missing license headers

Signed-off-by: kim <grufwub@gmail.com>

* fix request backoff calculation

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-04-28 17:45:21 +02:00
kim
de6e3e5f2a
[performance] refactoring + add fave / follow / request / visibility caching (#1607)
* refactor visibility checking, add caching for visibility

* invalidate visibility cache items on account / status deletes

* fix requester ID passed to visibility cache nil ptr

* de-interface caches, fix home / public timeline caching + visibility

* finish adding code comments for visibility filter

* fix angry goconst linter warnings

* actually finish adding filter visibility code comments for timeline functions

* move home timeline status author check to after visibility

* remove now-unused code

* add more code comments

* add TODO code comment, update printed cache start names

* update printed cache names on stop

* start adding separate follow(request) delete db functions, add specific visibility cache tests

* add relationship type caching

* fix getting local account follows / followed-bys, other small codebase improvements

* simplify invalidation using cache hooks, add more GetAccountBy___() functions

* fix boosting to return 404 if not boostable but no error (to not leak status ID)

* remove dead code

* improved placement of cache invalidation

* update license headers

* add example follow, follow-request config entries

* add example visibility cache configuration to config file

* use specific PutFollowRequest() instead of just Put()

* add tests for all GetAccountBy()

* add GetBlockBy() tests

* update block to check primitive fields

* update and finish adding Get{Account,Block,Follow,FollowRequest}By() tests

* fix copy-pasted code

* update envparsing test

* whitespace

* fix bun struct tag

* add license header to gtscontext

* fix old license header

* improved error creation to not use fmt.Errorf() when not needed

* fix various rebase conflicts, fix account test

* remove commented-out code, fix-up mention caching

* fix mention select bun statement

* ensure mention target account populated, pass in context to customrenderer logging

* remove more uncommented code, fix typeutil test

* add statusfave database model caching

* add status fave cache configuration

* add status fave cache example config

* woops, catch missed error. nice catch linter!

* add back testrig panic on nil db

* update example configuration to match defaults, slight tweak to cache configuration defaults

* update envparsing test with new defaults

* fetch followingget to use the follow target account

* use accounnt.IsLocal() instead of empty domain check

* use constants for the cache visibility type check

* use bun.In() for notification type restriction in db query

* include replies when fetching PublicTimeline() (to account for single-author threads in Visibility{}.StatusPublicTimelineable())

* use bun query building for nested select statements to ensure working with postgres

* update public timeline future status checks to match visibility filter

* same as previous, for home timeline

* update public timeline tests to dynamically check for appropriate statuses

* migrate accounts to allow unique constraint on public_key

* provide minimal account with publicKey

---------

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-03-28 14:03:14 +01:00