[performance] processing media and scheduled jobs improvements (#1482)

* replace media workers with just runners.WorkerPool, move to state structure, use go-sched for global task scheduling

* improved code comment

* fix worker tryUntil function, update go-runners/go-sched

* make preprocess functions package public, use these where possible to stop doubled up processing

* remove separate emoji worker pool

* limit calls to time.Now() during media preprocessing

* use Processor{} to manage singular runtime of processing media

* ensure workers get started when media manager is used

* improved error setting in processing media, fix media test

* port changes from processingmedia to processing emoji

* finish code commenting

* finish code commenting and comment-out client API + federator worker pools until concurrency worker pools replaced

* linterrrrrrrrrrrrrrrr

---------

Signed-off-by: kim <grufwub@gmail.com>
This commit is contained in:
kim 2023-02-13 18:40:48 +00:00 committed by GitHub
commit acc95923da
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
54 changed files with 1853 additions and 2680 deletions

15
vendor/modules.txt vendored
View file

@ -1,8 +1,8 @@
# codeberg.org/gruf/go-atomics v1.1.0
## explicit; go 1.16
codeberg.org/gruf/go-atomics
# codeberg.org/gruf/go-bitutil v1.0.1
## explicit; go 1.16
# codeberg.org/gruf/go-bitutil v1.1.0
## explicit; go 1.19
codeberg.org/gruf/go-bitutil
# codeberg.org/gruf/go-bytes v1.0.2
## explicit; go 1.14
@ -18,7 +18,7 @@ codeberg.org/gruf/go-byteutil
codeberg.org/gruf/go-cache/v3
codeberg.org/gruf/go-cache/v3/result
codeberg.org/gruf/go-cache/v3/ttl
# codeberg.org/gruf/go-debug v1.2.0
# codeberg.org/gruf/go-debug v1.3.0
## explicit; go 1.16
codeberg.org/gruf/go-debug
# codeberg.org/gruf/go-errors/v2 v2.1.1
@ -58,10 +58,10 @@ codeberg.org/gruf/go-mutexes
# codeberg.org/gruf/go-pools v1.1.0
## explicit; go 1.16
codeberg.org/gruf/go-pools
# codeberg.org/gruf/go-runners v1.5.1
## explicit; go 1.14
# codeberg.org/gruf/go-runners v1.6.0
## explicit; go 1.19
codeberg.org/gruf/go-runners
# codeberg.org/gruf/go-sched v1.2.0
# codeberg.org/gruf/go-sched v1.2.3
## explicit; go 1.19
codeberg.org/gruf/go-sched
# codeberg.org/gruf/go-store/v2 v2.2.1
@ -379,9 +379,6 @@ github.com/quasoft/memstore
# github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0
## explicit; go 1.12
github.com/remyoudompheng/bigfft
# github.com/robfig/cron/v3 v3.0.1
## explicit; go 1.12
github.com/robfig/cron/v3
# github.com/rs/xid v1.4.0
## explicit; go 1.12
github.com/rs/xid