diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1a5d94c24..611bea97c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -439,7 +439,6 @@ By default the standalone testrig uses an in-memory SQLite database, which is fi - `GTS_DB_ADDRESS` - this is set to `:memory:` by default. You can change this to use an sqlite.db file somewhere, or set it to a Postgres address. - `GTS_DB_PORT`, `GTS_DB_USER`, `GTS_DB_PASSWORD`, `GTS_DB_DATABASE`, `GTS_DB_TLS_MODE`, `GTS_DB_TLS_CA_CERT` - you can set these if you change `GTS_DB_ADDRESS` to `postgres` and don't use `GTS_DB_POSTGRES_CONNECTION_STRING`. - `GTS_DB_POSTGRES_CONNECTION_STRING` - use this to provide a Postgres connection string if you don't want to set all the db env variables mentioned in the previous point. -- `GTS_ADVANCED_SCRAPER_DETERRENCE_ENABLED`, `GTS_ADVANCED_SCRAPER_DETERRENCE_DIFFICULTY` - set these if you want to try out the PoW scraper deterrence locally. Using these variables you can also (albeit awkwardly) test migrations from one schema to another. diff --git a/docs/admin/robots.md b/docs/admin/robots.md index 29a02db42..285e41bf9 100644 --- a/docs/admin/robots.md +++ b/docs/admin/robots.md @@ -10,6 +10,6 @@ You can allow or disallow crawlers from collecting stats about your instance fro The AI scrapers come from a [community maintained repository][airobots]. It's manually kept in sync for the time being. If you know of any missing robots, please send them a PR! -A number of AI scrapers are known to ignore entries in `robots.txt` even if it explicitly matches their User-Agent. This means the `robots.txt` file is not a foolproof way of ensuring AI scrapers don't grab your content. In addition to this you might want to look into blocking User-Agents via [requester header filtering](request_filtering_modes.md), and enabling a proof-of-work [scraper deterrence](../advanced/scraper_deterrence.md). +A number of AI scrapers are known to ignore entries in `robots.txt` even if it explicitly matches their User-Agent. This means the `robots.txt` file is not a foolproof way of ensuring AI scrapers don't grab your content. In addition to this you might want to look into blocking User-Agents via [requester header filtering](request_filtering_modes.md). [airobots]: https://github.com/ai-robots-txt/ai.robots.txt/ diff --git a/docs/advanced/scraper_deterrence.md b/docs/advanced/scraper_deterrence.md deleted file mode 100644 index e20e3f6c6..000000000 --- a/docs/advanced/scraper_deterrence.md +++ /dev/null @@ -1,19 +0,0 @@ -# Scraper Deterrence - -GoToSocial provides an optional proof-of-work based scraper and automated HTTP client deterrence that can be enabled on profile and status web views. The way -it works is that it generates a unique but deterministic challenge for each incoming HTTP request based on client information and current time, that-is a hex encoded SHA256 hash. It then asks the client to find an integer addition to a portion of this that will generate an expected encoded hash result. This is served to the client as a minimal holding page with a single JavaScript worker that computes a solution to this. - -The number of hash encode rounds the client is required to complete may be configured, where high values will take the client longer to find a solution and vice-versa. We also instill a certain amount of jitter to make it harder for scrapers to "game" the algorithm. If your challenges take too long to solve, you may deter users from accessing your web pages. And conversely, the longer it takes for a solution to be found, the more you'll be incurring costs for scrapers (and in some cases, causing their operation to time-out). That balance is up to you to configure, hence why this is an advanced feature. - -Once a solution to this challenge has been provided, by refreshing the page with the solution in the query parameter, GoToSocial will verify this solution and on success will return the expected profile / status page with a cookie that provides challenge-less access to the instance for up-to the next hour. - -The outcomes of this, (when enabled), is that it should make scraping of your instance's profile / status pages economically unviable for automated data gathering (e.g. by AI companies, search engines). The only negative, is that it places a requirement on JavaScript being enabled for people to access your profile / status web views. - -This was heavily inspired by the great project that is [anubis], but ultimately we determined we could implement it ourselves with only the features we require, minimal code, and more granularity with our existing authorization / authentication procedures. - -The GoToSocial implementation of this scraper deterrence is still incredibly minimal, so if you're looking for more features or fine-grained control over your deterrence measures then by all means keep ours disabled and stand-up a service like [anubis] in front of your instance! - -!!! warning - This proof-of-work scraper deterrence does not protect user profile RSS feeds due to the extra complexity involved. If you rely on your RSS feed being exposed, this is one such case where [anubis] may be a better fit! - -[anubis]: https://github.com/TecharoHQ/anubis \ No newline at end of file diff --git a/docs/configuration/advanced.md b/docs/configuration/advanced.md index 0b8b3183f..88f4aff67 100644 --- a/docs/configuration/advanced.md +++ b/docs/configuration/advanced.md @@ -182,23 +182,4 @@ advanced-csp-extra-uris: [] # Options: ["block", "allow", ""] # Default: "" advanced-header-filter-mode: "" - -# Bool. Enables a proof-of-work based deterrence against scrapers -# on profile and status web pages. This will generate a unique but -# deterministic challenge for each HTTP client to complete before -# accessing the above mentioned endpoints, on success being given -# a cookie that permits challenge-less access within a 1hr window. -# -# The outcome of this is that it should make scraping of these -# endpoints economically unfeasible, while having a negligible -# performance impact on your own instance. -# -# The downside is that it requires javascript to be enabled. -# -# For more details please check the documentation at: -# https://docs.gotosocial.org/en/latest/admin/scraper_deterrence -# -# Options: [true, false] -# Default: true -advanced-scraper-deterrence: false ``` diff --git a/docs/locales/zh/admin/robots.md b/docs/locales/zh/admin/robots.md index c0c5b81ca..06f395087 100644 --- a/docs/locales/zh/admin/robots.md +++ b/docs/locales/zh/admin/robots.md @@ -10,6 +10,6 @@ GoToSocial 在主域名上提供一个 `robots.txt` 文件。该文件包含试 AI 爬虫来自一个[社区维护的仓库][airobots]。目前是手动保持同步的。如果你知道有任何遗漏的爬虫，请给他们提交一个 PR！ -众所周知，很多 AI 爬虫在 `robots.txt` 不允许其 User-Agent 的情况下，仍然会忽略对应规则并继续抓去内容。这意味着 `robots.txt` 文件并不是确保 AI 爬虫不抓取你的内容的万无一失的方法。除此以外，你可能还需要考虑通过[请求标头过滤](request_filtering_modes.md)来阻止对应 User-Agent，以及启用基于工作证明的[爬虫防护](../advanced/scraper_deterrence.md)。 +众所周知，很多 AI 爬虫在 `robots.txt` 不允许其 User-Agent 的情况下，仍然会忽略对应规则并继续抓去内容。这意味着 `robots.txt` 文件并不是确保 AI 爬虫不抓取你的内容的万无一失的方法。除此以外，你可能还需要考虑通过[请求标头过滤](request_filtering_modes.md)来阻止对应 User-Agent。 [airobots]: https://github.com/ai-robots-txt/ai.robots.txt/ diff --git a/docs/locales/zh/advanced/scraper_deterrence.md b/docs/locales/zh/advanced/scraper_deterrence.md deleted file mode 100644 index 69ac8552f..000000000 --- a/docs/locales/zh/advanced/scraper_deterrence.md +++ /dev/null @@ -1,18 +0,0 @@ -# 爬虫防护 - -GoToSocial 提供一个可选的、基于工作量证明的爬虫和自动化 HTTP 客户端防护机制，可在账户页和贴文页的网页视图上启用。 - -它的工作原理是：针对每个传入的 HTTP 请求，系统会根据客户端信息和当前时间生成一个唯一质询（一个十六进制编码的 SHA256 哈希值）。然后，它要求客户端为该质询的一部分找到一个附加值，使（附加值+质询部分）组合计算出的新 SHA256 哈希值（同样为十六进制编码）至少包含 4 个前导 '0' 字符。这个质询会通过一个极简的等待页面呈现给客户端，该页面包含一个独立的 JavaScript worker 来计算解决方案。 - -一旦客户端提供了此质询的解，并通过在查询参数中携带该方案刷新页面，GoToSocial 将验证此方案。验证成功后，服务端会返回用户期望访问的账户或贴文页面，并设置一个 Cookie。该 Cookie 允许用户在接下来最多一小时内免验证访问该实例。 - -启用此功能的目的是让自动化数据收集（例如 AI 公司、搜索引擎）对你实例的账户和贴文页面进行爬取的行为，在经济上变得不可行。唯一的缺点是，用户需要启用 JavaScript 才能访问你的账户和贴文网页视图。 - -这个功能深受优秀的 [anubis] 项目的启发，但我们最终决定自己实现，只包含我们需要的功能，使用最少的代码，并能与我们现有的授权/认证流程实现更细粒度的结合。 - -GoToSocial 实现的这个爬虫防护功能仍然是极其精简的。因此，如果你需要更多功能或对防护措施进行更精细的控制，那么完全可以禁用我们的内置功能，并在你的实例前部署像 [anubis] 这样的服务！ - -!!! warning "警告" - 这个基于工作量证明的爬虫防护机制并不保护用户账户页的 RSS feed，因为这会带来额外的复杂性。如果你需要确保 RSS feed 可被访问，那么在这种情况下，[anubis] 可能是更合适的选择！ - -[anubis]: https://github.com/TecharoHQ/anubis \ No newline at end of file diff --git a/docs/locales/zh/configuration/advanced.md b/docs/locales/zh/configuration/advanced.md index d9ecab845..c41f3e47d 100644 --- a/docs/locales/zh/configuration/advanced.md +++ b/docs/locales/zh/configuration/advanced.md @@ -149,21 +149,4 @@ advanced-csp-extra-uris: [] # 选项: ["block", "allow", ""] # 默认: "" advanced-header-filter-mode: "" - -# 布尔值。启用基于工作量证明的爬虫威慑机制， -# 作用于账户页和贴文页面。这将为每个 HTTP 客户端生成一个唯一确定 -# 的质询，需要由客户端在访问上述端点时完成。 -# 完成后，客户端会获得一个 Cookie，允许其在 1 小时窗口内免验证访问。 -# -# 这样做的结果是，它理论上使得对这些端点的抓取在经济上变得不可行， -# 同时对你自己的实例的性能影响可以忽略不计。 -# -# 缺点是它要求客户端启用 JavaScript。 -# -# 更多详情请查阅文档： -# https://docs.gotosocial.org/zh-cn/latest/admin/scraper_deterrence -# -# 选项: [true, false] -# 默认值: true -advanced-scraper-deterrence: false ``` \ No newline at end of file diff --git a/docs/locales/zh/mkdocs.yml b/docs/locales/zh/mkdocs.yml index 5a011340d..94f8895ad 100644 --- a/docs/locales/zh/mkdocs.yml +++ b/docs/locales/zh/mkdocs.yml @@ -80,7 +80,6 @@ nav: - "advanced/tracing.md" - "advanced/metrics.md" - "advanced/replicating-sqlite.md" - - "advanced/scraper_deterrence.md" - "advanced/sqlite-networked-storage.md" - "适用进阶场景的构建": - "advanced/builds/nowasm.md" diff --git a/example/config.yaml b/example/config.yaml index a3b9ab5cd..3e7375627 100644 --- a/example/config.yaml +++ b/example/config.yaml @@ -1338,40 +1338,3 @@ advanced-csp-extra-uris: [] # Options: ["block", "allow", ""] # Default: "" advanced-header-filter-mode: "" - -# Bool. Enables a proof-of-work based deterrence against scrapers -# on profile and status web pages. This will generate a unique but -# deterministic challenge for each HTTP client to complete before -# accessing the above mentioned endpoints, on success being given -# a cookie that permits challenge-less access within a 1hr window. -# -# The outcome of this is that it should make scraping of these -# endpoints economically unfeasible, while having a negligible -# performance impact on your own instance. -# -# The downside is that it requires javascript to be enabled. -# -# For more details please check the documentation at: -# https://docs.gotosocial.org/en/latest/advanced/scraper_deterrence -# -# Options: [true, false] -# Default: true -advanced-scraper-deterrence-enabled: false - -# Uint. Allows tweaking the difficulty of the proof-of-work algorithm -# used in the scraper deterrence. This determines roughly how many hash -# encode rounds we require the client to complete to find a solution. -# Higher values will take longer to find solutions for, and vice-versa. -# -# The downside is that if your deterrence takes too long to solve, -# it may deter some users from viewing your web status / profile page. -# And conversely, the longer it takes for a solution to be found, the -# more you'll be incurring increased CPU usage for scrapers, and possibly -# even cause their operation to time out before completion. -# -# For more details please check the documentation at: -# https://docs.gotosocial.org/en/latest/advanced/scraper_deterrence -# -# Examples: [50000, 100000, 500000] -# Default: 100000 -advanced-scraper-deterrence-difficulty: 100000 diff --git a/go.mod b/go.mod index 2915d9847..115dde99e 100644 --- a/go.mod +++ b/go.mod @@ -6,7 +6,7 @@ go 1.24.6 replace github.com/go-swagger/go-swagger => codeberg.org/superseriousbusiness/go-swagger v0.32.3-gts-go1.23-fix // Replace modernc/sqlite with our version that fixes the concurrency INTERRUPT issue -replace modernc.org/sqlite => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.38.2-concurrency-workaround +replace modernc.org/sqlite => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.39.0-concurrency-workaround require ( code.superseriousbusiness.org/activity v1.17.0 @@ -85,13 +85,13 @@ require ( go.uber.org/automaxprocs v1.6.0 golang.org/x/crypto v0.42.0 golang.org/x/image v0.31.0 - golang.org/x/net v0.43.0 + golang.org/x/net v0.44.0 golang.org/x/oauth2 v0.31.0 golang.org/x/sys v0.36.0 golang.org/x/text v0.29.0 gopkg.in/mcuadros/go-syslog.v2 v2.3.0 gopkg.in/yaml.v3 v3.0.1 - modernc.org/sqlite v1.38.2 + modernc.org/sqlite v1.39.0 mvdan.cc/xurls/v2 v2.6.0 ) diff --git a/go.sum b/go.sum index de3bbb155..9efbe092d 100644 --- a/go.sum +++ b/go.sum @@ -510,8 +510,8 @@ github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82/go.mod h1:lgjkn3NuSvDf github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= github.com/yuin/goldmark v1.7.13 h1:GPddIs617DnBLFFVJFgpo1aBfe/4xcvMc3SB5t/D0pA= github.com/yuin/goldmark v1.7.13/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg= -gitlab.com/NyaaaWhatsUpDoc/sqlite v1.38.2-concurrency-workaround h1:v1DRiquV7HAI68FED0VopIYIWXtuaYYZxj3fL71Zz+s= -gitlab.com/NyaaaWhatsUpDoc/sqlite v1.38.2-concurrency-workaround/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E= +gitlab.com/NyaaaWhatsUpDoc/sqlite v1.39.0-concurrency-workaround h1:+e2m4Ycgsri3YaePPAuYcTawQxklz0q/3CbKEbqxhOM= +gitlab.com/NyaaaWhatsUpDoc/sqlite v1.39.0-concurrency-workaround/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E= go.mongodb.org/mongo-driver v1.17.3 h1:TQyXhnsWfWtgAhMtOgtYHMTkZIfBTpMTsMnd9ZBeHxQ= go.mongodb.org/mongo-driver v1.17.3/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ= go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= @@ -605,8 +605,8 @@ golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk= golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= -golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE= -golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg= +golang.org/x/net v0.44.0 h1:evd8IRDyfNBMBTTY5XRF1vaZlD+EmWx6x8PkhR04H/I= +golang.org/x/net v0.44.0/go.mod h1:ECOoLqd5U3Lhyeyo/QDCEVQ4sNgYsqvCZ722XogGieY= golang.org/x/oauth2 v0.31.0 h1:8Fq0yVZLh4j4YA47vHKFTa9Ew5XIrCP8LC6UeNZnLxo= golang.org/x/oauth2 v0.31.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= diff --git a/internal/config/config.go b/internal/config/config.go index 3cab53732..8768584fa 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -279,13 +279,12 @@ type CacheConfiguration struct { } type AdvancedConfig struct { - CookiesSamesite string `name:"cookies-samesite" usage:"'strict' or 'lax', see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite"` - SenderMultiplier int `name:"sender-multiplier" usage:"Multiplier to use per cpu for batching outgoing fedi messages. 0 or less turns batching off (not recommended)."` - CSPExtraURIs []string `name:"csp-extra-uris" usage:"Additional URIs to allow when building content-security-policy for media + images."` - HeaderFilterMode string `name:"header-filter-mode" usage:"Set incoming request header filtering mode."` - RateLimit RateLimitConfig `name:"rate-limit"` - Throttling ThrottlingConfig `name:"throttling"` - ScraperDeterrence ScraperDeterrenceConfig `name:"scraper-deterrence"` + CookiesSamesite string `name:"cookies-samesite" usage:"'strict' or 'lax', see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite"` + SenderMultiplier int `name:"sender-multiplier" usage:"Multiplier to use per cpu for batching outgoing fedi messages. 0 or less turns batching off (not recommended)."` + CSPExtraURIs []string `name:"csp-extra-uris" usage:"Additional URIs to allow when building content-security-policy for media + images."` + HeaderFilterMode string `name:"header-filter-mode" usage:"Set incoming request header filtering mode."` + RateLimit RateLimitConfig `name:"rate-limit"` + Throttling ThrottlingConfig `name:"throttling"` } type RateLimitConfig struct { @@ -297,8 +296,3 @@ type ThrottlingConfig struct { Multiplier int `name:"multiplier" usage:"Multiplier to use per cpu for http request throttling. 0 or less turns throttling off."` RetryAfter time.Duration `name:"retry-after" usage:"Retry-After duration response to send for throttled requests."` } - -type ScraperDeterrenceConfig struct { - Enabled bool `name:"enabled" usage:"Enable proof-of-work based scraper deterrence on profile / status pages"` - Difficulty uint32 `name:"difficulty" usage:"The proof-of-work difficulty, which determines roughly how many hash-encode rounds required of each client."` -} diff --git a/internal/config/config_test.go b/internal/config/config_test.go index a23fc20f4..f4e011b16 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -175,7 +175,6 @@ func TestCLIParsing(t *testing.T) { "--config-path", "testdata/test3.yaml", }, expected: expectedKV( - kv.Field{"advanced-scraper-deterrence-enabled", true}, kv.Field{"advanced-rate-limit-requests", 5000}, ), }, diff --git a/internal/config/defaults.go b/internal/config/defaults.go index f140d7877..e3e3c559c 100644 --- a/internal/config/defaults.go +++ b/internal/config/defaults.go @@ -154,11 +154,6 @@ var Defaults = Configuration{ Multiplier: 8, // 8 open requests per CPU RetryAfter: 30 * time.Second, }, - - ScraperDeterrence: ScraperDeterrenceConfig{ - Enabled: false, - Difficulty: 100000, - }, }, Cache: CacheConfiguration{ diff --git a/internal/config/helpers.gen.go b/internal/config/helpers.gen.go index 9f5d6f39c..d54d72b53 100644 --- a/internal/config/helpers.gen.go +++ b/internal/config/helpers.gen.go @@ -136,8 +136,6 @@ const ( AdvancedRateLimitExceptionsFlag = "advanced-rate-limit-exceptions" AdvancedThrottlingMultiplierFlag = "advanced-throttling-multiplier" AdvancedThrottlingRetryAfterFlag = "advanced-throttling-retry-after" - AdvancedScraperDeterrenceEnabledFlag = "advanced-scraper-deterrence-enabled" - AdvancedScraperDeterrenceDifficultyFlag = "advanced-scraper-deterrence-difficulty" HTTPClientAllowIPsFlag = "http-client-allow-ips" HTTPClientBlockIPsFlag = "http-client-block-ips" HTTPClientTimeoutFlag = "http-client-timeout" @@ -336,8 +334,6 @@ func (cfg *Configuration) RegisterFlags(flags *pflag.FlagSet) { flags.StringSlice("advanced-rate-limit-exceptions", cfg.Advanced.RateLimit.Exceptions.Strings(), "Slice of CIDRs to exclude from rate limit restrictions.") flags.Int("advanced-throttling-multiplier", cfg.Advanced.Throttling.Multiplier, "Multiplier to use per cpu for http request throttling. 0 or less turns throttling off.") flags.Duration("advanced-throttling-retry-after", cfg.Advanced.Throttling.RetryAfter, "Retry-After duration response to send for throttled requests.") - flags.Bool("advanced-scraper-deterrence-enabled", cfg.Advanced.ScraperDeterrence.Enabled, "Enable proof-of-work based scraper deterrence on profile / status pages") - flags.Uint32("advanced-scraper-deterrence-difficulty", cfg.Advanced.ScraperDeterrence.Difficulty, "The proof-of-work difficulty, which determines roughly how many hash-encode rounds required of each client.") flags.StringSlice("http-client-allow-ips", cfg.HTTPClient.AllowIPs, "") flags.StringSlice("http-client-block-ips", cfg.HTTPClient.BlockIPs, "") flags.Duration("http-client-timeout", cfg.HTTPClient.Timeout, "") @@ -420,7 +416,7 @@ func (cfg *Configuration) RegisterFlags(flags *pflag.FlagSet) { } func (cfg *Configuration) MarshalMap() map[string]any { - cfgmap := make(map[string]any, 197) + cfgmap := make(map[string]any, 195) cfgmap["log-level"] = cfg.LogLevel cfgmap["log-format"] = cfg.LogFormat cfgmap["log-timestamp-format"] = cfg.LogTimestampFormat @@ -528,8 +524,6 @@ func (cfg *Configuration) MarshalMap() map[string]any { cfgmap["advanced-rate-limit-exceptions"] = cfg.Advanced.RateLimit.Exceptions.Strings() cfgmap["advanced-throttling-multiplier"] = cfg.Advanced.Throttling.Multiplier cfgmap["advanced-throttling-retry-after"] = cfg.Advanced.Throttling.RetryAfter - cfgmap["advanced-scraper-deterrence-enabled"] = cfg.Advanced.ScraperDeterrence.Enabled - cfgmap["advanced-scraper-deterrence-difficulty"] = cfg.Advanced.ScraperDeterrence.Difficulty cfgmap["http-client-allow-ips"] = cfg.HTTPClient.AllowIPs cfgmap["http-client-block-ips"] = cfg.HTTPClient.BlockIPs cfgmap["http-client-timeout"] = cfg.HTTPClient.Timeout @@ -1496,22 +1490,6 @@ func (cfg *Configuration) UnmarshalMap(cfgmap map[string]any) error { } } - if ival, ok := cfgmap["advanced-scraper-deterrence-enabled"]; ok { - var err error - cfg.Advanced.ScraperDeterrence.Enabled, err = cast.ToBoolE(ival) - if err != nil { - return fmt.Errorf("error casting %#v -> bool for 'advanced-scraper-deterrence-enabled': %w", ival, err) - } - } - - if ival, ok := cfgmap["advanced-scraper-deterrence-difficulty"]; ok { - var err error - cfg.Advanced.ScraperDeterrence.Difficulty, err = cast.ToUint32E(ival) - if err != nil { - return fmt.Errorf("error casting %#v -> uint32 for 'advanced-scraper-deterrence-difficulty': %w", ival, err) - } - } - if ival, ok := cfgmap["http-client-allow-ips"]; ok { var err error cfg.HTTPClient.AllowIPs, err = toStringSlice(ival) @@ -2245,7 +2223,7 @@ func (st *ConfigState) GetLogLevel() (v string) { st.mutex.RLock() v = st.config.LogLevel st.mutex.RUnlock() - return + return v } // SetLogLevel safely sets the Configuration value for state's 'LogLevel' field @@ -2267,7 +2245,7 @@ func (st *ConfigState) GetLogFormat() (v string) { st.mutex.RLock() v = st.config.LogFormat st.mutex.RUnlock() - return + return v } // SetLogFormat safely sets the Configuration value for state's 'LogFormat' field @@ -2289,7 +2267,7 @@ func (st *ConfigState) GetLogTimestampFormat() (v string) { st.mutex.RLock() v = st.config.LogTimestampFormat st.mutex.RUnlock() - return + return v } // SetLogTimestampFormat safely sets the Configuration value for state's 'LogTimestampFormat' field @@ -2311,7 +2289,7 @@ func (st *ConfigState) GetLogDbQueries() (v bool) { st.mutex.RLock() v = st.config.LogDbQueries st.mutex.RUnlock() - return + return v } // SetLogDbQueries safely sets the Configuration value for state's 'LogDbQueries' field @@ -2333,7 +2311,7 @@ func (st *ConfigState) GetLogClientIP() (v bool) { st.mutex.RLock() v = st.config.LogClientIP st.mutex.RUnlock() - return + return v } // SetLogClientIP safely sets the Configuration value for state's 'LogClientIP' field @@ -2355,7 +2333,7 @@ func (st *ConfigState) GetRequestIDHeader() (v string) { st.mutex.RLock() v = st.config.RequestIDHeader st.mutex.RUnlock() - return + return v } // SetRequestIDHeader safely sets the Configuration value for state's 'RequestIDHeader' field @@ -2377,7 +2355,7 @@ func (st *ConfigState) GetConfigPath() (v string) { st.mutex.RLock() v = st.config.ConfigPath st.mutex.RUnlock() - return + return v } // SetConfigPath safely sets the Configuration value for state's 'ConfigPath' field @@ -2399,7 +2377,7 @@ func (st *ConfigState) GetApplicationName() (v string) { st.mutex.RLock() v = st.config.ApplicationName st.mutex.RUnlock() - return + return v } // SetApplicationName safely sets the Configuration value for state's 'ApplicationName' field @@ -2421,7 +2399,7 @@ func (st *ConfigState) GetLandingPageUser() (v string) { st.mutex.RLock() v = st.config.LandingPageUser st.mutex.RUnlock() - return + return v } // SetLandingPageUser safely sets the Configuration value for state's 'LandingPageUser' field @@ -2443,7 +2421,7 @@ func (st *ConfigState) GetHost() (v string) { st.mutex.RLock() v = st.config.Host st.mutex.RUnlock() - return + return v } // SetHost safely sets the Configuration value for state's 'Host' field @@ -2465,7 +2443,7 @@ func (st *ConfigState) GetAccountDomain() (v string) { st.mutex.RLock() v = st.config.AccountDomain st.mutex.RUnlock() - return + return v } // SetAccountDomain safely sets the Configuration value for state's 'AccountDomain' field @@ -2487,7 +2465,7 @@ func (st *ConfigState) GetProtocol() (v string) { st.mutex.RLock() v = st.config.Protocol st.mutex.RUnlock() - return + return v } // SetProtocol safely sets the Configuration value for state's 'Protocol' field @@ -2509,7 +2487,7 @@ func (st *ConfigState) GetBindAddress() (v string) { st.mutex.RLock() v = st.config.BindAddress st.mutex.RUnlock() - return + return v } // SetBindAddress safely sets the Configuration value for state's 'BindAddress' field @@ -2531,7 +2509,7 @@ func (st *ConfigState) GetPort() (v int) { st.mutex.RLock() v = st.config.Port st.mutex.RUnlock() - return + return v } // SetPort safely sets the Configuration value for state's 'Port' field @@ -2553,7 +2531,7 @@ func (st *ConfigState) GetTrustedProxies() (v []string) { st.mutex.RLock() v = st.config.TrustedProxies st.mutex.RUnlock() - return + return v } // SetTrustedProxies safely sets the Configuration value for state's 'TrustedProxies' field @@ -2575,7 +2553,7 @@ func (st *ConfigState) GetSoftwareVersion() (v string) { st.mutex.RLock() v = st.config.SoftwareVersion st.mutex.RUnlock() - return + return v } // SetSoftwareVersion safely sets the Configuration value for state's 'SoftwareVersion' field @@ -2597,7 +2575,7 @@ func (st *ConfigState) GetDbType() (v string) { st.mutex.RLock() v = st.config.DbType st.mutex.RUnlock() - return + return v } // SetDbType safely sets the Configuration value for state's 'DbType' field @@ -2619,7 +2597,7 @@ func (st *ConfigState) GetDbAddress() (v string) { st.mutex.RLock() v = st.config.DbAddress st.mutex.RUnlock() - return + return v } // SetDbAddress safely sets the Configuration value for state's 'DbAddress' field @@ -2641,7 +2619,7 @@ func (st *ConfigState) GetDbPort() (v int) { st.mutex.RLock() v = st.config.DbPort st.mutex.RUnlock() - return + return v } // SetDbPort safely sets the Configuration value for state's 'DbPort' field @@ -2663,7 +2641,7 @@ func (st *ConfigState) GetDbUser() (v string) { st.mutex.RLock() v = st.config.DbUser st.mutex.RUnlock() - return + return v } // SetDbUser safely sets the Configuration value for state's 'DbUser' field @@ -2685,7 +2663,7 @@ func (st *ConfigState) GetDbPassword() (v string) { st.mutex.RLock() v = st.config.DbPassword st.mutex.RUnlock() - return + return v } // SetDbPassword safely sets the Configuration value for state's 'DbPassword' field @@ -2707,7 +2685,7 @@ func (st *ConfigState) GetDbDatabase() (v string) { st.mutex.RLock() v = st.config.DbDatabase st.mutex.RUnlock() - return + return v } // SetDbDatabase safely sets the Configuration value for state's 'DbDatabase' field @@ -2729,7 +2707,7 @@ func (st *ConfigState) GetDbTLSMode() (v string) { st.mutex.RLock() v = st.config.DbTLSMode st.mutex.RUnlock() - return + return v } // SetDbTLSMode safely sets the Configuration value for state's 'DbTLSMode' field @@ -2751,7 +2729,7 @@ func (st *ConfigState) GetDbTLSCACert() (v string) { st.mutex.RLock() v = st.config.DbTLSCACert st.mutex.RUnlock() - return + return v } // SetDbTLSCACert safely sets the Configuration value for state's 'DbTLSCACert' field @@ -2773,7 +2751,7 @@ func (st *ConfigState) GetDbMaxOpenConnsMultiplier() (v int) { st.mutex.RLock() v = st.config.DbMaxOpenConnsMultiplier st.mutex.RUnlock() - return + return v } // SetDbMaxOpenConnsMultiplier safely sets the Configuration value for state's 'DbMaxOpenConnsMultiplier' field @@ -2795,7 +2773,7 @@ func (st *ConfigState) GetDbSqliteJournalMode() (v string) { st.mutex.RLock() v = st.config.DbSqliteJournalMode st.mutex.RUnlock() - return + return v } // SetDbSqliteJournalMode safely sets the Configuration value for state's 'DbSqliteJournalMode' field @@ -2817,7 +2795,7 @@ func (st *ConfigState) GetDbSqliteSynchronous() (v string) { st.mutex.RLock() v = st.config.DbSqliteSynchronous st.mutex.RUnlock() - return + return v } // SetDbSqliteSynchronous safely sets the Configuration value for state's 'DbSqliteSynchronous' field @@ -2839,7 +2817,7 @@ func (st *ConfigState) GetDbSqliteCacheSize() (v bytesize.Size) { st.mutex.RLock() v = st.config.DbSqliteCacheSize st.mutex.RUnlock() - return + return v } // SetDbSqliteCacheSize safely sets the Configuration value for state's 'DbSqliteCacheSize' field @@ -2861,7 +2839,7 @@ func (st *ConfigState) GetDbSqliteBusyTimeout() (v time.Duration) { st.mutex.RLock() v = st.config.DbSqliteBusyTimeout st.mutex.RUnlock() - return + return v } // SetDbSqliteBusyTimeout safely sets the Configuration value for state's 'DbSqliteBusyTimeout' field @@ -2883,7 +2861,7 @@ func (st *ConfigState) GetDbPostgresConnectionString() (v string) { st.mutex.RLock() v = st.config.DbPostgresConnectionString st.mutex.RUnlock() - return + return v } // SetDbPostgresConnectionString safely sets the Configuration value for state's 'DbPostgresConnectionString' field @@ -2905,7 +2883,7 @@ func (st *ConfigState) GetWebTemplateBaseDir() (v string) { st.mutex.RLock() v = st.config.WebTemplateBaseDir st.mutex.RUnlock() - return + return v } // SetWebTemplateBaseDir safely sets the Configuration value for state's 'WebTemplateBaseDir' field @@ -2927,7 +2905,7 @@ func (st *ConfigState) GetWebAssetBaseDir() (v string) { st.mutex.RLock() v = st.config.WebAssetBaseDir st.mutex.RUnlock() - return + return v } // SetWebAssetBaseDir safely sets the Configuration value for state's 'WebAssetBaseDir' field @@ -2949,7 +2927,7 @@ func (st *ConfigState) GetInstanceFederationMode() (v string) { st.mutex.RLock() v = st.config.InstanceFederationMode st.mutex.RUnlock() - return + return v } // SetInstanceFederationMode safely sets the Configuration value for state's 'InstanceFederationMode' field @@ -2971,7 +2949,7 @@ func (st *ConfigState) GetInstanceFederationSpamFilter() (v bool) { st.mutex.RLock() v = st.config.InstanceFederationSpamFilter st.mutex.RUnlock() - return + return v } // SetInstanceFederationSpamFilter safely sets the Configuration value for state's 'InstanceFederationSpamFilter' field @@ -2993,7 +2971,7 @@ func (st *ConfigState) GetInstanceExposePeers() (v bool) { st.mutex.RLock() v = st.config.InstanceExposePeers st.mutex.RUnlock() - return + return v } // SetInstanceExposePeers safely sets the Configuration value for state's 'InstanceExposePeers' field @@ -3015,7 +2993,7 @@ func (st *ConfigState) GetInstanceExposeBlocklist() (v bool) { st.mutex.RLock() v = st.config.InstanceExposeBlocklist st.mutex.RUnlock() - return + return v } // SetInstanceExposeBlocklist safely sets the Configuration value for state's 'InstanceExposeBlocklist' field @@ -3037,7 +3015,7 @@ func (st *ConfigState) GetInstanceExposeBlocklistWeb() (v bool) { st.mutex.RLock() v = st.config.InstanceExposeBlocklistWeb st.mutex.RUnlock() - return + return v } // SetInstanceExposeBlocklistWeb safely sets the Configuration value for state's 'InstanceExposeBlocklistWeb' field @@ -3059,7 +3037,7 @@ func (st *ConfigState) GetInstanceExposeAllowlist() (v bool) { st.mutex.RLock() v = st.config.InstanceExposeAllowlist st.mutex.RUnlock() - return + return v } // SetInstanceExposeAllowlist safely sets the Configuration value for state's 'InstanceExposeAllowlist' field @@ -3081,7 +3059,7 @@ func (st *ConfigState) GetInstanceExposeAllowlistWeb() (v bool) { st.mutex.RLock() v = st.config.InstanceExposeAllowlistWeb st.mutex.RUnlock() - return + return v } // SetInstanceExposeAllowlistWeb safely sets the Configuration value for state's 'InstanceExposeAllowlistWeb' field @@ -3103,7 +3081,7 @@ func (st *ConfigState) GetInstanceExposePublicTimeline() (v bool) { st.mutex.RLock() v = st.config.InstanceExposePublicTimeline st.mutex.RUnlock() - return + return v } // SetInstanceExposePublicTimeline safely sets the Configuration value for state's 'InstanceExposePublicTimeline' field @@ -3125,7 +3103,7 @@ func (st *ConfigState) GetInstanceExposeCustomEmojis() (v bool) { st.mutex.RLock() v = st.config.InstanceExposeCustomEmojis st.mutex.RUnlock() - return + return v } // SetInstanceExposeCustomEmojis safely sets the Configuration value for state's 'InstanceExposeCustomEmojis' field @@ -3147,7 +3125,7 @@ func (st *ConfigState) GetInstanceDeliverToSharedInboxes() (v bool) { st.mutex.RLock() v = st.config.InstanceDeliverToSharedInboxes st.mutex.RUnlock() - return + return v } // SetInstanceDeliverToSharedInboxes safely sets the Configuration value for state's 'InstanceDeliverToSharedInboxes' field @@ -3169,7 +3147,7 @@ func (st *ConfigState) GetInstanceInjectMastodonVersion() (v bool) { st.mutex.RLock() v = st.config.InstanceInjectMastodonVersion st.mutex.RUnlock() - return + return v } // SetInstanceInjectMastodonVersion safely sets the Configuration value for state's 'InstanceInjectMastodonVersion' field @@ -3191,7 +3169,7 @@ func (st *ConfigState) GetInstanceLanguages() (v language.Languages) { st.mutex.RLock() v = st.config.InstanceLanguages st.mutex.RUnlock() - return + return v } // SetInstanceLanguages safely sets the Configuration value for state's 'InstanceLanguages' field @@ -3213,7 +3191,7 @@ func (st *ConfigState) GetInstanceSubscriptionsProcessFrom() (v string) { st.mutex.RLock() v = st.config.InstanceSubscriptionsProcessFrom st.mutex.RUnlock() - return + return v } // SetInstanceSubscriptionsProcessFrom safely sets the Configuration value for state's 'InstanceSubscriptionsProcessFrom' field @@ -3237,7 +3215,7 @@ func (st *ConfigState) GetInstanceSubscriptionsProcessEvery() (v time.Duration) st.mutex.RLock() v = st.config.InstanceSubscriptionsProcessEvery st.mutex.RUnlock() - return + return v } // SetInstanceSubscriptionsProcessEvery safely sets the Configuration value for state's 'InstanceSubscriptionsProcessEvery' field @@ -3263,7 +3241,7 @@ func (st *ConfigState) GetInstanceStatsMode() (v string) { st.mutex.RLock() v = st.config.InstanceStatsMode st.mutex.RUnlock() - return + return v } // SetInstanceStatsMode safely sets the Configuration value for state's 'InstanceStatsMode' field @@ -3285,7 +3263,7 @@ func (st *ConfigState) GetInstanceAllowBackdatingStatuses() (v bool) { st.mutex.RLock() v = st.config.InstanceAllowBackdatingStatuses st.mutex.RUnlock() - return + return v } // SetInstanceAllowBackdatingStatuses safely sets the Configuration value for state's 'InstanceAllowBackdatingStatuses' field @@ -3307,7 +3285,7 @@ func (st *ConfigState) GetAccountsRegistrationOpen() (v bool) { st.mutex.RLock() v = st.config.AccountsRegistrationOpen st.mutex.RUnlock() - return + return v } // SetAccountsRegistrationOpen safely sets the Configuration value for state's 'AccountsRegistrationOpen' field @@ -3329,7 +3307,7 @@ func (st *ConfigState) GetAccountsReasonRequired() (v bool) { st.mutex.RLock() v = st.config.AccountsReasonRequired st.mutex.RUnlock() - return + return v } // SetAccountsReasonRequired safely sets the Configuration value for state's 'AccountsReasonRequired' field @@ -3351,7 +3329,7 @@ func (st *ConfigState) GetAccountsRegistrationDailyLimit() (v int) { st.mutex.RLock() v = st.config.AccountsRegistrationDailyLimit st.mutex.RUnlock() - return + return v } // SetAccountsRegistrationDailyLimit safely sets the Configuration value for state's 'AccountsRegistrationDailyLimit' field @@ -3373,7 +3351,7 @@ func (st *ConfigState) GetAccountsRegistrationBacklogLimit() (v int) { st.mutex.RLock() v = st.config.AccountsRegistrationBacklogLimit st.mutex.RUnlock() - return + return v } // SetAccountsRegistrationBacklogLimit safely sets the Configuration value for state's 'AccountsRegistrationBacklogLimit' field @@ -3395,7 +3373,7 @@ func (st *ConfigState) GetAccountsAllowCustomCSS() (v bool) { st.mutex.RLock() v = st.config.AccountsAllowCustomCSS st.mutex.RUnlock() - return + return v } // SetAccountsAllowCustomCSS safely sets the Configuration value for state's 'AccountsAllowCustomCSS' field @@ -3417,7 +3395,7 @@ func (st *ConfigState) GetAccountsCustomCSSLength() (v int) { st.mutex.RLock() v = st.config.AccountsCustomCSSLength st.mutex.RUnlock() - return + return v } // SetAccountsCustomCSSLength safely sets the Configuration value for state's 'AccountsCustomCSSLength' field @@ -3439,7 +3417,7 @@ func (st *ConfigState) GetAccountsMaxProfileFields() (v int) { st.mutex.RLock() v = st.config.AccountsMaxProfileFields st.mutex.RUnlock() - return + return v } // SetAccountsMaxProfileFields safely sets the Configuration value for state's 'AccountsMaxProfileFields' field @@ -3461,7 +3439,7 @@ func (st *ConfigState) GetStorageBackend() (v string) { st.mutex.RLock() v = st.config.StorageBackend st.mutex.RUnlock() - return + return v } // SetStorageBackend safely sets the Configuration value for state's 'StorageBackend' field @@ -3483,7 +3461,7 @@ func (st *ConfigState) GetStorageLocalBasePath() (v string) { st.mutex.RLock() v = st.config.StorageLocalBasePath st.mutex.RUnlock() - return + return v } // SetStorageLocalBasePath safely sets the Configuration value for state's 'StorageLocalBasePath' field @@ -3505,7 +3483,7 @@ func (st *ConfigState) GetStorageS3Endpoint() (v string) { st.mutex.RLock() v = st.config.StorageS3Endpoint st.mutex.RUnlock() - return + return v } // SetStorageS3Endpoint safely sets the Configuration value for state's 'StorageS3Endpoint' field @@ -3527,7 +3505,7 @@ func (st *ConfigState) GetStorageS3AccessKey() (v string) { st.mutex.RLock() v = st.config.StorageS3AccessKey st.mutex.RUnlock() - return + return v } // SetStorageS3AccessKey safely sets the Configuration value for state's 'StorageS3AccessKey' field @@ -3549,7 +3527,7 @@ func (st *ConfigState) GetStorageS3SecretKey() (v string) { st.mutex.RLock() v = st.config.StorageS3SecretKey st.mutex.RUnlock() - return + return v } // SetStorageS3SecretKey safely sets the Configuration value for state's 'StorageS3SecretKey' field @@ -3571,7 +3549,7 @@ func (st *ConfigState) GetStorageS3UseSSL() (v bool) { st.mutex.RLock() v = st.config.StorageS3UseSSL st.mutex.RUnlock() - return + return v } // SetStorageS3UseSSL safely sets the Configuration value for state's 'StorageS3UseSSL' field @@ -3593,7 +3571,7 @@ func (st *ConfigState) GetStorageS3BucketName() (v string) { st.mutex.RLock() v = st.config.StorageS3BucketName st.mutex.RUnlock() - return + return v } // SetStorageS3BucketName safely sets the Configuration value for state's 'StorageS3BucketName' field @@ -3615,7 +3593,7 @@ func (st *ConfigState) GetStorageS3Proxy() (v bool) { st.mutex.RLock() v = st.config.StorageS3Proxy st.mutex.RUnlock() - return + return v } // SetStorageS3Proxy safely sets the Configuration value for state's 'StorageS3Proxy' field @@ -3637,7 +3615,7 @@ func (st *ConfigState) GetStorageS3RedirectURL() (v string) { st.mutex.RLock() v = st.config.StorageS3RedirectURL st.mutex.RUnlock() - return + return v } // SetStorageS3RedirectURL safely sets the Configuration value for state's 'StorageS3RedirectURL' field @@ -3659,7 +3637,7 @@ func (st *ConfigState) GetStorageS3BucketLookup() (v string) { st.mutex.RLock() v = st.config.StorageS3BucketLookup st.mutex.RUnlock() - return + return v } // SetStorageS3BucketLookup safely sets the Configuration value for state's 'StorageS3BucketLookup' field @@ -3681,7 +3659,7 @@ func (st *ConfigState) GetStorageS3KeyPrefix() (v string) { st.mutex.RLock() v = st.config.StorageS3KeyPrefix st.mutex.RUnlock() - return + return v } // SetStorageS3KeyPrefix safely sets the Configuration value for state's 'StorageS3KeyPrefix' field @@ -3703,7 +3681,7 @@ func (st *ConfigState) GetStatusesMaxChars() (v int) { st.mutex.RLock() v = st.config.StatusesMaxChars st.mutex.RUnlock() - return + return v } // SetStatusesMaxChars safely sets the Configuration value for state's 'StatusesMaxChars' field @@ -3725,7 +3703,7 @@ func (st *ConfigState) GetStatusesPollMaxOptions() (v int) { st.mutex.RLock() v = st.config.StatusesPollMaxOptions st.mutex.RUnlock() - return + return v } // SetStatusesPollMaxOptions safely sets the Configuration value for state's 'StatusesPollMaxOptions' field @@ -3747,7 +3725,7 @@ func (st *ConfigState) GetStatusesPollOptionMaxChars() (v int) { st.mutex.RLock() v = st.config.StatusesPollOptionMaxChars st.mutex.RUnlock() - return + return v } // SetStatusesPollOptionMaxChars safely sets the Configuration value for state's 'StatusesPollOptionMaxChars' field @@ -3769,7 +3747,7 @@ func (st *ConfigState) GetStatusesMediaMaxFiles() (v int) { st.mutex.RLock() v = st.config.StatusesMediaMaxFiles st.mutex.RUnlock() - return + return v } // SetStatusesMediaMaxFiles safely sets the Configuration value for state's 'StatusesMediaMaxFiles' field @@ -3791,7 +3769,7 @@ func (st *ConfigState) GetScheduledStatusesMaxTotal() (v int) { st.mutex.RLock() v = st.config.ScheduledStatusesMaxTotal st.mutex.RUnlock() - return + return v } // SetScheduledStatusesMaxTotal safely sets the Configuration value for state's 'ScheduledStatusesMaxTotal' field @@ -3813,7 +3791,7 @@ func (st *ConfigState) GetScheduledStatusesMaxDaily() (v int) { st.mutex.RLock() v = st.config.ScheduledStatusesMaxDaily st.mutex.RUnlock() - return + return v } // SetScheduledStatusesMaxDaily safely sets the Configuration value for state's 'ScheduledStatusesMaxDaily' field @@ -3835,7 +3813,7 @@ func (st *ConfigState) GetLetsEncryptEnabled() (v bool) { st.mutex.RLock() v = st.config.LetsEncryptEnabled st.mutex.RUnlock() - return + return v } // SetLetsEncryptEnabled safely sets the Configuration value for state's 'LetsEncryptEnabled' field @@ -3857,7 +3835,7 @@ func (st *ConfigState) GetLetsEncryptPort() (v int) { st.mutex.RLock() v = st.config.LetsEncryptPort st.mutex.RUnlock() - return + return v } // SetLetsEncryptPort safely sets the Configuration value for state's 'LetsEncryptPort' field @@ -3879,7 +3857,7 @@ func (st *ConfigState) GetLetsEncryptCertDir() (v string) { st.mutex.RLock() v = st.config.LetsEncryptCertDir st.mutex.RUnlock() - return + return v } // SetLetsEncryptCertDir safely sets the Configuration value for state's 'LetsEncryptCertDir' field @@ -3901,7 +3879,7 @@ func (st *ConfigState) GetLetsEncryptEmailAddress() (v string) { st.mutex.RLock() v = st.config.LetsEncryptEmailAddress st.mutex.RUnlock() - return + return v } // SetLetsEncryptEmailAddress safely sets the Configuration value for state's 'LetsEncryptEmailAddress' field @@ -3923,7 +3901,7 @@ func (st *ConfigState) GetTLSCertificateChain() (v string) { st.mutex.RLock() v = st.config.TLSCertificateChain st.mutex.RUnlock() - return + return v } // SetTLSCertificateChain safely sets the Configuration value for state's 'TLSCertificateChain' field @@ -3945,7 +3923,7 @@ func (st *ConfigState) GetTLSCertificateKey() (v string) { st.mutex.RLock() v = st.config.TLSCertificateKey st.mutex.RUnlock() - return + return v } // SetTLSCertificateKey safely sets the Configuration value for state's 'TLSCertificateKey' field @@ -3967,7 +3945,7 @@ func (st *ConfigState) GetOIDCEnabled() (v bool) { st.mutex.RLock() v = st.config.OIDCEnabled st.mutex.RUnlock() - return + return v } // SetOIDCEnabled safely sets the Configuration value for state's 'OIDCEnabled' field @@ -3989,7 +3967,7 @@ func (st *ConfigState) GetOIDCIdpName() (v string) { st.mutex.RLock() v = st.config.OIDCIdpName st.mutex.RUnlock() - return + return v } // SetOIDCIdpName safely sets the Configuration value for state's 'OIDCIdpName' field @@ -4011,7 +3989,7 @@ func (st *ConfigState) GetOIDCSkipVerification() (v bool) { st.mutex.RLock() v = st.config.OIDCSkipVerification st.mutex.RUnlock() - return + return v } // SetOIDCSkipVerification safely sets the Configuration value for state's 'OIDCSkipVerification' field @@ -4033,7 +4011,7 @@ func (st *ConfigState) GetOIDCIssuer() (v string) { st.mutex.RLock() v = st.config.OIDCIssuer st.mutex.RUnlock() - return + return v } // SetOIDCIssuer safely sets the Configuration value for state's 'OIDCIssuer' field @@ -4055,7 +4033,7 @@ func (st *ConfigState) GetOIDCClientID() (v string) { st.mutex.RLock() v = st.config.OIDCClientID st.mutex.RUnlock() - return + return v } // SetOIDCClientID safely sets the Configuration value for state's 'OIDCClientID' field @@ -4077,7 +4055,7 @@ func (st *ConfigState) GetOIDCClientSecret() (v string) { st.mutex.RLock() v = st.config.OIDCClientSecret st.mutex.RUnlock() - return + return v } // SetOIDCClientSecret safely sets the Configuration value for state's 'OIDCClientSecret' field @@ -4099,7 +4077,7 @@ func (st *ConfigState) GetOIDCScopes() (v []string) { st.mutex.RLock() v = st.config.OIDCScopes st.mutex.RUnlock() - return + return v } // SetOIDCScopes safely sets the Configuration value for state's 'OIDCScopes' field @@ -4121,7 +4099,7 @@ func (st *ConfigState) GetOIDCLinkExisting() (v bool) { st.mutex.RLock() v = st.config.OIDCLinkExisting st.mutex.RUnlock() - return + return v } // SetOIDCLinkExisting safely sets the Configuration value for state's 'OIDCLinkExisting' field @@ -4143,7 +4121,7 @@ func (st *ConfigState) GetOIDCAllowedGroups() (v []string) { st.mutex.RLock() v = st.config.OIDCAllowedGroups st.mutex.RUnlock() - return + return v } // SetOIDCAllowedGroups safely sets the Configuration value for state's 'OIDCAllowedGroups' field @@ -4165,7 +4143,7 @@ func (st *ConfigState) GetOIDCAdminGroups() (v []string) { st.mutex.RLock() v = st.config.OIDCAdminGroups st.mutex.RUnlock() - return + return v } // SetOIDCAdminGroups safely sets the Configuration value for state's 'OIDCAdminGroups' field @@ -4187,7 +4165,7 @@ func (st *ConfigState) GetTracingEnabled() (v bool) { st.mutex.RLock() v = st.config.TracingEnabled st.mutex.RUnlock() - return + return v } // SetTracingEnabled safely sets the Configuration value for state's 'TracingEnabled' field @@ -4209,7 +4187,7 @@ func (st *ConfigState) GetMetricsEnabled() (v bool) { st.mutex.RLock() v = st.config.MetricsEnabled st.mutex.RUnlock() - return + return v } // SetMetricsEnabled safely sets the Configuration value for state's 'MetricsEnabled' field @@ -4231,7 +4209,7 @@ func (st *ConfigState) GetSMTPHost() (v string) { st.mutex.RLock() v = st.config.SMTPHost st.mutex.RUnlock() - return + return v } // SetSMTPHost safely sets the Configuration value for state's 'SMTPHost' field @@ -4253,7 +4231,7 @@ func (st *ConfigState) GetSMTPPort() (v int) { st.mutex.RLock() v = st.config.SMTPPort st.mutex.RUnlock() - return + return v } // SetSMTPPort safely sets the Configuration value for state's 'SMTPPort' field @@ -4275,7 +4253,7 @@ func (st *ConfigState) GetSMTPUsername() (v string) { st.mutex.RLock() v = st.config.SMTPUsername st.mutex.RUnlock() - return + return v } // SetSMTPUsername safely sets the Configuration value for state's 'SMTPUsername' field @@ -4297,7 +4275,7 @@ func (st *ConfigState) GetSMTPPassword() (v string) { st.mutex.RLock() v = st.config.SMTPPassword st.mutex.RUnlock() - return + return v } // SetSMTPPassword safely sets the Configuration value for state's 'SMTPPassword' field @@ -4319,7 +4297,7 @@ func (st *ConfigState) GetSMTPFrom() (v string) { st.mutex.RLock() v = st.config.SMTPFrom st.mutex.RUnlock() - return + return v } // SetSMTPFrom safely sets the Configuration value for state's 'SMTPFrom' field @@ -4341,7 +4319,7 @@ func (st *ConfigState) GetSMTPDiscloseRecipients() (v bool) { st.mutex.RLock() v = st.config.SMTPDiscloseRecipients st.mutex.RUnlock() - return + return v } // SetSMTPDiscloseRecipients safely sets the Configuration value for state's 'SMTPDiscloseRecipients' field @@ -4363,7 +4341,7 @@ func (st *ConfigState) GetSyslogEnabled() (v bool) { st.mutex.RLock() v = st.config.SyslogEnabled st.mutex.RUnlock() - return + return v } // SetSyslogEnabled safely sets the Configuration value for state's 'SyslogEnabled' field @@ -4385,7 +4363,7 @@ func (st *ConfigState) GetSyslogProtocol() (v string) { st.mutex.RLock() v = st.config.SyslogProtocol st.mutex.RUnlock() - return + return v } // SetSyslogProtocol safely sets the Configuration value for state's 'SyslogProtocol' field @@ -4407,7 +4385,7 @@ func (st *ConfigState) GetSyslogAddress() (v string) { st.mutex.RLock() v = st.config.SyslogAddress st.mutex.RUnlock() - return + return v } // SetSyslogAddress safely sets the Configuration value for state's 'SyslogAddress' field @@ -4429,7 +4407,7 @@ func (st *ConfigState) GetAdvancedCookiesSamesite() (v string) { st.mutex.RLock() v = st.config.Advanced.CookiesSamesite st.mutex.RUnlock() - return + return v } // SetAdvancedCookiesSamesite safely sets the Configuration value for state's 'Advanced.CookiesSamesite' field @@ -4451,7 +4429,7 @@ func (st *ConfigState) GetAdvancedSenderMultiplier() (v int) { st.mutex.RLock() v = st.config.Advanced.SenderMultiplier st.mutex.RUnlock() - return + return v } // SetAdvancedSenderMultiplier safely sets the Configuration value for state's 'Advanced.SenderMultiplier' field @@ -4473,7 +4451,7 @@ func (st *ConfigState) GetAdvancedCSPExtraURIs() (v []string) { st.mutex.RLock() v = st.config.Advanced.CSPExtraURIs st.mutex.RUnlock() - return + return v } // SetAdvancedCSPExtraURIs safely sets the Configuration value for state's 'Advanced.CSPExtraURIs' field @@ -4495,7 +4473,7 @@ func (st *ConfigState) GetAdvancedHeaderFilterMode() (v string) { st.mutex.RLock() v = st.config.Advanced.HeaderFilterMode st.mutex.RUnlock() - return + return v } // SetAdvancedHeaderFilterMode safely sets the Configuration value for state's 'Advanced.HeaderFilterMode' field @@ -4517,7 +4495,7 @@ func (st *ConfigState) GetAdvancedRateLimitRequests() (v int) { st.mutex.RLock() v = st.config.Advanced.RateLimit.Requests st.mutex.RUnlock() - return + return v } // SetAdvancedRateLimitRequests safely sets the Configuration value for state's 'Advanced.RateLimit.Requests' field @@ -4539,7 +4517,7 @@ func (st *ConfigState) GetAdvancedRateLimitExceptions() (v IPPrefixes) { st.mutex.RLock() v = st.config.Advanced.RateLimit.Exceptions st.mutex.RUnlock() - return + return v } // SetAdvancedRateLimitExceptions safely sets the Configuration value for state's 'Advanced.RateLimit.Exceptions' field @@ -4561,7 +4539,7 @@ func (st *ConfigState) GetAdvancedThrottlingMultiplier() (v int) { st.mutex.RLock() v = st.config.Advanced.Throttling.Multiplier st.mutex.RUnlock() - return + return v } // SetAdvancedThrottlingMultiplier safely sets the Configuration value for state's 'Advanced.Throttling.Multiplier' field @@ -4583,7 +4561,7 @@ func (st *ConfigState) GetAdvancedThrottlingRetryAfter() (v time.Duration) { st.mutex.RLock() v = st.config.Advanced.Throttling.RetryAfter st.mutex.RUnlock() - return + return v } // SetAdvancedThrottlingRetryAfter safely sets the Configuration value for state's 'Advanced.Throttling.RetryAfter' field @@ -4600,60 +4578,12 @@ func GetAdvancedThrottlingRetryAfter() time.Duration { return global.GetAdvanced // SetAdvancedThrottlingRetryAfter safely sets the value for global configuration 'Advanced.Throttling.RetryAfter' field func SetAdvancedThrottlingRetryAfter(v time.Duration) { global.SetAdvancedThrottlingRetryAfter(v) } -// GetAdvancedScraperDeterrenceEnabled safely fetches the Configuration value for state's 'Advanced.ScraperDeterrence.Enabled' field -func (st *ConfigState) GetAdvancedScraperDeterrenceEnabled() (v bool) { - st.mutex.RLock() - v = st.config.Advanced.ScraperDeterrence.Enabled - st.mutex.RUnlock() - return -} - -// SetAdvancedScraperDeterrenceEnabled safely sets the Configuration value for state's 'Advanced.ScraperDeterrence.Enabled' field -func (st *ConfigState) SetAdvancedScraperDeterrenceEnabled(v bool) { - st.mutex.Lock() - defer st.mutex.Unlock() - st.config.Advanced.ScraperDeterrence.Enabled = v - st.reloadToViper() -} - -// GetAdvancedScraperDeterrenceEnabled safely fetches the value for global configuration 'Advanced.ScraperDeterrence.Enabled' field -func GetAdvancedScraperDeterrenceEnabled() bool { return global.GetAdvancedScraperDeterrenceEnabled() } - -// SetAdvancedScraperDeterrenceEnabled safely sets the value for global configuration 'Advanced.ScraperDeterrence.Enabled' field -func SetAdvancedScraperDeterrenceEnabled(v bool) { global.SetAdvancedScraperDeterrenceEnabled(v) } - -// GetAdvancedScraperDeterrenceDifficulty safely fetches the Configuration value for state's 'Advanced.ScraperDeterrence.Difficulty' field -func (st *ConfigState) GetAdvancedScraperDeterrenceDifficulty() (v uint32) { - st.mutex.RLock() - v = st.config.Advanced.ScraperDeterrence.Difficulty - st.mutex.RUnlock() - return -} - -// SetAdvancedScraperDeterrenceDifficulty safely sets the Configuration value for state's 'Advanced.ScraperDeterrence.Difficulty' field -func (st *ConfigState) SetAdvancedScraperDeterrenceDifficulty(v uint32) { - st.mutex.Lock() - defer st.mutex.Unlock() - st.config.Advanced.ScraperDeterrence.Difficulty = v - st.reloadToViper() -} - -// GetAdvancedScraperDeterrenceDifficulty safely fetches the value for global configuration 'Advanced.ScraperDeterrence.Difficulty' field -func GetAdvancedScraperDeterrenceDifficulty() uint32 { - return global.GetAdvancedScraperDeterrenceDifficulty() -} - -// SetAdvancedScraperDeterrenceDifficulty safely sets the value for global configuration 'Advanced.ScraperDeterrence.Difficulty' field -func SetAdvancedScraperDeterrenceDifficulty(v uint32) { - global.SetAdvancedScraperDeterrenceDifficulty(v) -} - // GetHTTPClientAllowIPs safely fetches the Configuration value for state's 'HTTPClient.AllowIPs' field func (st *ConfigState) GetHTTPClientAllowIPs() (v []string) { st.mutex.RLock() v = st.config.HTTPClient.AllowIPs st.mutex.RUnlock() - return + return v } // SetHTTPClientAllowIPs safely sets the Configuration value for state's 'HTTPClient.AllowIPs' field @@ -4675,7 +4605,7 @@ func (st *ConfigState) GetHTTPClientBlockIPs() (v []string) { st.mutex.RLock() v = st.config.HTTPClient.BlockIPs st.mutex.RUnlock() - return + return v } // SetHTTPClientBlockIPs safely sets the Configuration value for state's 'HTTPClient.BlockIPs' field @@ -4697,7 +4627,7 @@ func (st *ConfigState) GetHTTPClientTimeout() (v time.Duration) { st.mutex.RLock() v = st.config.HTTPClient.Timeout st.mutex.RUnlock() - return + return v } // SetHTTPClientTimeout safely sets the Configuration value for state's 'HTTPClient.Timeout' field @@ -4719,7 +4649,7 @@ func (st *ConfigState) GetHTTPClientTLSInsecureSkipVerify() (v bool) { st.mutex.RLock() v = st.config.HTTPClient.TLSInsecureSkipVerify st.mutex.RUnlock() - return + return v } // SetHTTPClientTLSInsecureSkipVerify safely sets the Configuration value for state's 'HTTPClient.TLSInsecureSkipVerify' field @@ -4741,7 +4671,7 @@ func (st *ConfigState) GetHTTPClientInsecureOutgoing() (v bool) { st.mutex.RLock() v = st.config.HTTPClient.InsecureOutgoing st.mutex.RUnlock() - return + return v } // SetHTTPClientInsecureOutgoing safely sets the Configuration value for state's 'HTTPClient.InsecureOutgoing' field @@ -4763,7 +4693,7 @@ func (st *ConfigState) GetMediaDescriptionMinChars() (v int) { st.mutex.RLock() v = st.config.Media.DescriptionMinChars st.mutex.RUnlock() - return + return v } // SetMediaDescriptionMinChars safely sets the Configuration value for state's 'Media.DescriptionMinChars' field @@ -4785,7 +4715,7 @@ func (st *ConfigState) GetMediaDescriptionMaxChars() (v int) { st.mutex.RLock() v = st.config.Media.DescriptionMaxChars st.mutex.RUnlock() - return + return v } // SetMediaDescriptionMaxChars safely sets the Configuration value for state's 'Media.DescriptionMaxChars' field @@ -4807,7 +4737,7 @@ func (st *ConfigState) GetMediaRemoteCacheDays() (v int) { st.mutex.RLock() v = st.config.Media.RemoteCacheDays st.mutex.RUnlock() - return + return v } // SetMediaRemoteCacheDays safely sets the Configuration value for state's 'Media.RemoteCacheDays' field @@ -4829,7 +4759,7 @@ func (st *ConfigState) GetMediaEmojiLocalMaxSize() (v bytesize.Size) { st.mutex.RLock() v = st.config.Media.EmojiLocalMaxSize st.mutex.RUnlock() - return + return v } // SetMediaEmojiLocalMaxSize safely sets the Configuration value for state's 'Media.EmojiLocalMaxSize' field @@ -4851,7 +4781,7 @@ func (st *ConfigState) GetMediaEmojiRemoteMaxSize() (v bytesize.Size) { st.mutex.RLock() v = st.config.Media.EmojiRemoteMaxSize st.mutex.RUnlock() - return + return v } // SetMediaEmojiRemoteMaxSize safely sets the Configuration value for state's 'Media.EmojiRemoteMaxSize' field @@ -4873,7 +4803,7 @@ func (st *ConfigState) GetMediaImageSizeHint() (v bytesize.Size) { st.mutex.RLock() v = st.config.Media.ImageSizeHint st.mutex.RUnlock() - return + return v } // SetMediaImageSizeHint safely sets the Configuration value for state's 'Media.ImageSizeHint' field @@ -4895,7 +4825,7 @@ func (st *ConfigState) GetMediaVideoSizeHint() (v bytesize.Size) { st.mutex.RLock() v = st.config.Media.VideoSizeHint st.mutex.RUnlock() - return + return v } // SetMediaVideoSizeHint safely sets the Configuration value for state's 'Media.VideoSizeHint' field @@ -4917,7 +4847,7 @@ func (st *ConfigState) GetMediaLocalMaxSize() (v bytesize.Size) { st.mutex.RLock() v = st.config.Media.LocalMaxSize st.mutex.RUnlock() - return + return v } // SetMediaLocalMaxSize safely sets the Configuration value for state's 'Media.LocalMaxSize' field @@ -4939,7 +4869,7 @@ func (st *ConfigState) GetMediaRemoteMaxSize() (v bytesize.Size) { st.mutex.RLock() v = st.config.Media.RemoteMaxSize st.mutex.RUnlock() - return + return v } // SetMediaRemoteMaxSize safely sets the Configuration value for state's 'Media.RemoteMaxSize' field @@ -4961,7 +4891,7 @@ func (st *ConfigState) GetMediaCleanupFrom() (v string) { st.mutex.RLock() v = st.config.Media.CleanupFrom st.mutex.RUnlock() - return + return v } // SetMediaCleanupFrom safely sets the Configuration value for state's 'Media.CleanupFrom' field @@ -4983,7 +4913,7 @@ func (st *ConfigState) GetMediaCleanupEvery() (v time.Duration) { st.mutex.RLock() v = st.config.Media.CleanupEvery st.mutex.RUnlock() - return + return v } // SetMediaCleanupEvery safely sets the Configuration value for state's 'Media.CleanupEvery' field @@ -5005,7 +4935,7 @@ func (st *ConfigState) GetMediaFfmpegPoolSize() (v int) { st.mutex.RLock() v = st.config.Media.FfmpegPoolSize st.mutex.RUnlock() - return + return v } // SetMediaFfmpegPoolSize safely sets the Configuration value for state's 'Media.FfmpegPoolSize' field @@ -5027,7 +4957,7 @@ func (st *ConfigState) GetMediaThumbMaxPixels() (v int) { st.mutex.RLock() v = st.config.Media.ThumbMaxPixels st.mutex.RUnlock() - return + return v } // SetMediaThumbMaxPixels safely sets the Configuration value for state's 'Media.ThumbMaxPixels' field @@ -5049,7 +4979,7 @@ func (st *ConfigState) GetCacheMemoryTarget() (v bytesize.Size) { st.mutex.RLock() v = st.config.Cache.MemoryTarget st.mutex.RUnlock() - return + return v } // SetCacheMemoryTarget safely sets the Configuration value for state's 'Cache.MemoryTarget' field @@ -5071,7 +5001,7 @@ func (st *ConfigState) GetCacheAccountMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.AccountMemRatio st.mutex.RUnlock() - return + return v } // SetCacheAccountMemRatio safely sets the Configuration value for state's 'Cache.AccountMemRatio' field @@ -5093,7 +5023,7 @@ func (st *ConfigState) GetCacheAccountNoteMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.AccountNoteMemRatio st.mutex.RUnlock() - return + return v } // SetCacheAccountNoteMemRatio safely sets the Configuration value for state's 'Cache.AccountNoteMemRatio' field @@ -5115,7 +5045,7 @@ func (st *ConfigState) GetCacheAccountSettingsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.AccountSettingsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheAccountSettingsMemRatio safely sets the Configuration value for state's 'Cache.AccountSettingsMemRatio' field @@ -5137,7 +5067,7 @@ func (st *ConfigState) GetCacheAccountStatsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.AccountStatsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheAccountStatsMemRatio safely sets the Configuration value for state's 'Cache.AccountStatsMemRatio' field @@ -5159,7 +5089,7 @@ func (st *ConfigState) GetCacheApplicationMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ApplicationMemRatio st.mutex.RUnlock() - return + return v } // SetCacheApplicationMemRatio safely sets the Configuration value for state's 'Cache.ApplicationMemRatio' field @@ -5181,7 +5111,7 @@ func (st *ConfigState) GetCacheBlockMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.BlockMemRatio st.mutex.RUnlock() - return + return v } // SetCacheBlockMemRatio safely sets the Configuration value for state's 'Cache.BlockMemRatio' field @@ -5203,7 +5133,7 @@ func (st *ConfigState) GetCacheBlockIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.BlockIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheBlockIDsMemRatio safely sets the Configuration value for state's 'Cache.BlockIDsMemRatio' field @@ -5225,7 +5155,7 @@ func (st *ConfigState) GetCacheBoostOfIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.BoostOfIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheBoostOfIDsMemRatio safely sets the Configuration value for state's 'Cache.BoostOfIDsMemRatio' field @@ -5247,7 +5177,7 @@ func (st *ConfigState) GetCacheClientMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ClientMemRatio st.mutex.RUnlock() - return + return v } // SetCacheClientMemRatio safely sets the Configuration value for state's 'Cache.ClientMemRatio' field @@ -5269,7 +5199,7 @@ func (st *ConfigState) GetCacheConversationMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ConversationMemRatio st.mutex.RUnlock() - return + return v } // SetCacheConversationMemRatio safely sets the Configuration value for state's 'Cache.ConversationMemRatio' field @@ -5291,7 +5221,7 @@ func (st *ConfigState) GetCacheConversationLastStatusIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ConversationLastStatusIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheConversationLastStatusIDsMemRatio safely sets the Configuration value for state's 'Cache.ConversationLastStatusIDsMemRatio' field @@ -5317,7 +5247,7 @@ func (st *ConfigState) GetCacheDomainPermissionDraftMemRation() (v float64) { st.mutex.RLock() v = st.config.Cache.DomainPermissionDraftMemRation st.mutex.RUnlock() - return + return v } // SetCacheDomainPermissionDraftMemRation safely sets the Configuration value for state's 'Cache.DomainPermissionDraftMemRation' field @@ -5343,7 +5273,7 @@ func (st *ConfigState) GetCacheDomainPermissionSubscriptionMemRation() (v float6 st.mutex.RLock() v = st.config.Cache.DomainPermissionSubscriptionMemRation st.mutex.RUnlock() - return + return v } // SetCacheDomainPermissionSubscriptionMemRation safely sets the Configuration value for state's 'Cache.DomainPermissionSubscriptionMemRation' field @@ -5369,7 +5299,7 @@ func (st *ConfigState) GetCacheEmojiMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.EmojiMemRatio st.mutex.RUnlock() - return + return v } // SetCacheEmojiMemRatio safely sets the Configuration value for state's 'Cache.EmojiMemRatio' field @@ -5391,7 +5321,7 @@ func (st *ConfigState) GetCacheEmojiCategoryMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.EmojiCategoryMemRatio st.mutex.RUnlock() - return + return v } // SetCacheEmojiCategoryMemRatio safely sets the Configuration value for state's 'Cache.EmojiCategoryMemRatio' field @@ -5413,7 +5343,7 @@ func (st *ConfigState) GetCacheFilterMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FilterMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFilterMemRatio safely sets the Configuration value for state's 'Cache.FilterMemRatio' field @@ -5435,7 +5365,7 @@ func (st *ConfigState) GetCacheFilterIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FilterIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFilterIDsMemRatio safely sets the Configuration value for state's 'Cache.FilterIDsMemRatio' field @@ -5457,7 +5387,7 @@ func (st *ConfigState) GetCacheFilterKeywordMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FilterKeywordMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFilterKeywordMemRatio safely sets the Configuration value for state's 'Cache.FilterKeywordMemRatio' field @@ -5479,7 +5409,7 @@ func (st *ConfigState) GetCacheFilterStatusMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FilterStatusMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFilterStatusMemRatio safely sets the Configuration value for state's 'Cache.FilterStatusMemRatio' field @@ -5501,7 +5431,7 @@ func (st *ConfigState) GetCacheFollowMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FollowMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFollowMemRatio safely sets the Configuration value for state's 'Cache.FollowMemRatio' field @@ -5523,7 +5453,7 @@ func (st *ConfigState) GetCacheFollowIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FollowIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFollowIDsMemRatio safely sets the Configuration value for state's 'Cache.FollowIDsMemRatio' field @@ -5545,7 +5475,7 @@ func (st *ConfigState) GetCacheFollowRequestMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FollowRequestMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFollowRequestMemRatio safely sets the Configuration value for state's 'Cache.FollowRequestMemRatio' field @@ -5567,7 +5497,7 @@ func (st *ConfigState) GetCacheFollowRequestIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FollowRequestIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFollowRequestIDsMemRatio safely sets the Configuration value for state's 'Cache.FollowRequestIDsMemRatio' field @@ -5589,7 +5519,7 @@ func (st *ConfigState) GetCacheFollowingTagIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.FollowingTagIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheFollowingTagIDsMemRatio safely sets the Configuration value for state's 'Cache.FollowingTagIDsMemRatio' field @@ -5611,7 +5541,7 @@ func (st *ConfigState) GetCacheInReplyToIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.InReplyToIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheInReplyToIDsMemRatio safely sets the Configuration value for state's 'Cache.InReplyToIDsMemRatio' field @@ -5633,7 +5563,7 @@ func (st *ConfigState) GetCacheInstanceMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.InstanceMemRatio st.mutex.RUnlock() - return + return v } // SetCacheInstanceMemRatio safely sets the Configuration value for state's 'Cache.InstanceMemRatio' field @@ -5655,7 +5585,7 @@ func (st *ConfigState) GetCacheInteractionRequestMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.InteractionRequestMemRatio st.mutex.RUnlock() - return + return v } // SetCacheInteractionRequestMemRatio safely sets the Configuration value for state's 'Cache.InteractionRequestMemRatio' field @@ -5677,7 +5607,7 @@ func (st *ConfigState) GetCacheListMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ListMemRatio st.mutex.RUnlock() - return + return v } // SetCacheListMemRatio safely sets the Configuration value for state's 'Cache.ListMemRatio' field @@ -5699,7 +5629,7 @@ func (st *ConfigState) GetCacheListIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ListIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheListIDsMemRatio safely sets the Configuration value for state's 'Cache.ListIDsMemRatio' field @@ -5721,7 +5651,7 @@ func (st *ConfigState) GetCacheListedIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ListedIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheListedIDsMemRatio safely sets the Configuration value for state's 'Cache.ListedIDsMemRatio' field @@ -5743,7 +5673,7 @@ func (st *ConfigState) GetCacheMarkerMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.MarkerMemRatio st.mutex.RUnlock() - return + return v } // SetCacheMarkerMemRatio safely sets the Configuration value for state's 'Cache.MarkerMemRatio' field @@ -5765,7 +5695,7 @@ func (st *ConfigState) GetCacheMediaMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.MediaMemRatio st.mutex.RUnlock() - return + return v } // SetCacheMediaMemRatio safely sets the Configuration value for state's 'Cache.MediaMemRatio' field @@ -5787,7 +5717,7 @@ func (st *ConfigState) GetCacheMentionMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.MentionMemRatio st.mutex.RUnlock() - return + return v } // SetCacheMentionMemRatio safely sets the Configuration value for state's 'Cache.MentionMemRatio' field @@ -5809,7 +5739,7 @@ func (st *ConfigState) GetCacheMoveMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.MoveMemRatio st.mutex.RUnlock() - return + return v } // SetCacheMoveMemRatio safely sets the Configuration value for state's 'Cache.MoveMemRatio' field @@ -5831,7 +5761,7 @@ func (st *ConfigState) GetCacheNotificationMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.NotificationMemRatio st.mutex.RUnlock() - return + return v } // SetCacheNotificationMemRatio safely sets the Configuration value for state's 'Cache.NotificationMemRatio' field @@ -5853,7 +5783,7 @@ func (st *ConfigState) GetCachePollMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.PollMemRatio st.mutex.RUnlock() - return + return v } // SetCachePollMemRatio safely sets the Configuration value for state's 'Cache.PollMemRatio' field @@ -5875,7 +5805,7 @@ func (st *ConfigState) GetCachePollVoteMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.PollVoteMemRatio st.mutex.RUnlock() - return + return v } // SetCachePollVoteMemRatio safely sets the Configuration value for state's 'Cache.PollVoteMemRatio' field @@ -5897,7 +5827,7 @@ func (st *ConfigState) GetCachePollVoteIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.PollVoteIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCachePollVoteIDsMemRatio safely sets the Configuration value for state's 'Cache.PollVoteIDsMemRatio' field @@ -5919,7 +5849,7 @@ func (st *ConfigState) GetCacheReportMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ReportMemRatio st.mutex.RUnlock() - return + return v } // SetCacheReportMemRatio safely sets the Configuration value for state's 'Cache.ReportMemRatio' field @@ -5941,7 +5871,7 @@ func (st *ConfigState) GetCacheScheduledStatusMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ScheduledStatusMemRatio st.mutex.RUnlock() - return + return v } // SetCacheScheduledStatusMemRatio safely sets the Configuration value for state's 'Cache.ScheduledStatusMemRatio' field @@ -5963,7 +5893,7 @@ func (st *ConfigState) GetCacheSinBinStatusMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.SinBinStatusMemRatio st.mutex.RUnlock() - return + return v } // SetCacheSinBinStatusMemRatio safely sets the Configuration value for state's 'Cache.SinBinStatusMemRatio' field @@ -5985,7 +5915,7 @@ func (st *ConfigState) GetCacheStatusMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusMemRatio safely sets the Configuration value for state's 'Cache.StatusMemRatio' field @@ -6007,7 +5937,7 @@ func (st *ConfigState) GetCacheStatusBookmarkMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusBookmarkMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusBookmarkMemRatio safely sets the Configuration value for state's 'Cache.StatusBookmarkMemRatio' field @@ -6029,7 +5959,7 @@ func (st *ConfigState) GetCacheStatusBookmarkIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusBookmarkIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusBookmarkIDsMemRatio safely sets the Configuration value for state's 'Cache.StatusBookmarkIDsMemRatio' field @@ -6051,7 +5981,7 @@ func (st *ConfigState) GetCacheStatusEditMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusEditMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusEditMemRatio safely sets the Configuration value for state's 'Cache.StatusEditMemRatio' field @@ -6073,7 +6003,7 @@ func (st *ConfigState) GetCacheStatusFaveMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusFaveMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusFaveMemRatio safely sets the Configuration value for state's 'Cache.StatusFaveMemRatio' field @@ -6095,7 +6025,7 @@ func (st *ConfigState) GetCacheStatusFaveIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusFaveIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusFaveIDsMemRatio safely sets the Configuration value for state's 'Cache.StatusFaveIDsMemRatio' field @@ -6117,7 +6047,7 @@ func (st *ConfigState) GetCacheTagMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.TagMemRatio st.mutex.RUnlock() - return + return v } // SetCacheTagMemRatio safely sets the Configuration value for state's 'Cache.TagMemRatio' field @@ -6139,7 +6069,7 @@ func (st *ConfigState) GetCacheThreadMuteMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.ThreadMuteMemRatio st.mutex.RUnlock() - return + return v } // SetCacheThreadMuteMemRatio safely sets the Configuration value for state's 'Cache.ThreadMuteMemRatio' field @@ -6161,7 +6091,7 @@ func (st *ConfigState) GetCacheTokenMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.TokenMemRatio st.mutex.RUnlock() - return + return v } // SetCacheTokenMemRatio safely sets the Configuration value for state's 'Cache.TokenMemRatio' field @@ -6183,7 +6113,7 @@ func (st *ConfigState) GetCacheTombstoneMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.TombstoneMemRatio st.mutex.RUnlock() - return + return v } // SetCacheTombstoneMemRatio safely sets the Configuration value for state's 'Cache.TombstoneMemRatio' field @@ -6205,7 +6135,7 @@ func (st *ConfigState) GetCacheUserMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.UserMemRatio st.mutex.RUnlock() - return + return v } // SetCacheUserMemRatio safely sets the Configuration value for state's 'Cache.UserMemRatio' field @@ -6227,7 +6157,7 @@ func (st *ConfigState) GetCacheUserMuteMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.UserMuteMemRatio st.mutex.RUnlock() - return + return v } // SetCacheUserMuteMemRatio safely sets the Configuration value for state's 'Cache.UserMuteMemRatio' field @@ -6249,7 +6179,7 @@ func (st *ConfigState) GetCacheUserMuteIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.UserMuteIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheUserMuteIDsMemRatio safely sets the Configuration value for state's 'Cache.UserMuteIDsMemRatio' field @@ -6271,7 +6201,7 @@ func (st *ConfigState) GetCacheWebfingerMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.WebfingerMemRatio st.mutex.RUnlock() - return + return v } // SetCacheWebfingerMemRatio safely sets the Configuration value for state's 'Cache.WebfingerMemRatio' field @@ -6293,7 +6223,7 @@ func (st *ConfigState) GetCacheWebPushSubscriptionMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.WebPushSubscriptionMemRatio st.mutex.RUnlock() - return + return v } // SetCacheWebPushSubscriptionMemRatio safely sets the Configuration value for state's 'Cache.WebPushSubscriptionMemRatio' field @@ -6317,7 +6247,7 @@ func (st *ConfigState) GetCacheWebPushSubscriptionIDsMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.WebPushSubscriptionIDsMemRatio st.mutex.RUnlock() - return + return v } // SetCacheWebPushSubscriptionIDsMemRatio safely sets the Configuration value for state's 'Cache.WebPushSubscriptionIDsMemRatio' field @@ -6343,7 +6273,7 @@ func (st *ConfigState) GetCacheMutesMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.MutesMemRatio st.mutex.RUnlock() - return + return v } // SetCacheMutesMemRatio safely sets the Configuration value for state's 'Cache.MutesMemRatio' field @@ -6365,7 +6295,7 @@ func (st *ConfigState) GetCacheStatusFilterMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.StatusFilterMemRatio st.mutex.RUnlock() - return + return v } // SetCacheStatusFilterMemRatio safely sets the Configuration value for state's 'Cache.StatusFilterMemRatio' field @@ -6387,7 +6317,7 @@ func (st *ConfigState) GetCacheVisibilityMemRatio() (v float64) { st.mutex.RLock() v = st.config.Cache.VisibilityMemRatio st.mutex.RUnlock() - return + return v } // SetCacheVisibilityMemRatio safely sets the Configuration value for state's 'Cache.VisibilityMemRatio' field @@ -6409,7 +6339,7 @@ func (st *ConfigState) GetAdminAccountUsername() (v string) { st.mutex.RLock() v = st.config.AdminAccountUsername st.mutex.RUnlock() - return + return v } // SetAdminAccountUsername safely sets the Configuration value for state's 'AdminAccountUsername' field @@ -6431,7 +6361,7 @@ func (st *ConfigState) GetAdminAccountEmail() (v string) { st.mutex.RLock() v = st.config.AdminAccountEmail st.mutex.RUnlock() - return + return v } // SetAdminAccountEmail safely sets the Configuration value for state's 'AdminAccountEmail' field @@ -6453,7 +6383,7 @@ func (st *ConfigState) GetAdminAccountPassword() (v string) { st.mutex.RLock() v = st.config.AdminAccountPassword st.mutex.RUnlock() - return + return v } // SetAdminAccountPassword safely sets the Configuration value for state's 'AdminAccountPassword' field @@ -6475,7 +6405,7 @@ func (st *ConfigState) GetAdminTransPath() (v string) { st.mutex.RLock() v = st.config.AdminTransPath st.mutex.RUnlock() - return + return v } // SetAdminTransPath safely sets the Configuration value for state's 'AdminTransPath' field @@ -6497,7 +6427,7 @@ func (st *ConfigState) GetAdminMediaPruneDryRun() (v bool) { st.mutex.RLock() v = st.config.AdminMediaPruneDryRun st.mutex.RUnlock() - return + return v } // SetAdminMediaPruneDryRun safely sets the Configuration value for state's 'AdminMediaPruneDryRun' field @@ -6519,7 +6449,7 @@ func (st *ConfigState) GetAdminMediaListLocalOnly() (v bool) { st.mutex.RLock() v = st.config.AdminMediaListLocalOnly st.mutex.RUnlock() - return + return v } // SetAdminMediaListLocalOnly safely sets the Configuration value for state's 'AdminMediaListLocalOnly' field @@ -6541,7 +6471,7 @@ func (st *ConfigState) GetAdminMediaListRemoteOnly() (v bool) { st.mutex.RLock() v = st.config.AdminMediaListRemoteOnly st.mutex.RUnlock() - return + return v } // SetAdminMediaListRemoteOnly safely sets the Configuration value for state's 'AdminMediaListRemoteOnly' field @@ -6563,7 +6493,7 @@ func (st *ConfigState) GetTestrigSkipDBSetup() (v bool) { st.mutex.RLock() v = st.config.TestrigSkipDBSetup st.mutex.RUnlock() - return + return v } // SetTestrigSkipDBSetup safely sets the Configuration value for state's 'TestrigSkipDBSetup' field @@ -6585,7 +6515,7 @@ func (st *ConfigState) GetTestrigSkipDBTeardown() (v bool) { st.mutex.RLock() v = st.config.TestrigSkipDBTeardown st.mutex.RUnlock() - return + return v } // SetTestrigSkipDBTeardown safely sets the Configuration value for state's 'TestrigSkipDBTeardown' field @@ -6666,7 +6596,7 @@ func (st *ConfigState) GetTotalOfMemRatios() (total float64) { total += st.config.Cache.StatusFilterMemRatio total += st.config.Cache.VisibilityMemRatio st.mutex.RUnlock() - return + return total } // GetTotalOfMemRatios safely fetches the combined value for all the global state's mem ratio fields @@ -6766,30 +6696,6 @@ func flattenConfigMap(cfgmap map[string]any) { } } - for _, key := range [][]string{ - {"advanced-scraper-deterrence", "enabled"}, - {"advanced", "scraper-deterrence", "enabled"}, - } { - ival, ok := mapGet(cfgmap, key...) - if ok { - cfgmap["advanced-scraper-deterrence-enabled"] = ival - nestedKeys[key[0]] = struct{}{} - break - } - } - - for _, key := range [][]string{ - {"advanced-scraper-deterrence", "difficulty"}, - {"advanced", "scraper-deterrence", "difficulty"}, - } { - ival, ok := mapGet(cfgmap, key...) - if ok { - cfgmap["advanced-scraper-deterrence-difficulty"] = ival - nestedKeys[key[0]] = struct{}{} - break - } - } - for _, key := range [][]string{ {"http-client", "allow-ips"}, } { diff --git a/internal/config/testdata/test3.yaml b/internal/config/testdata/test3.yaml index d383fdd80..b0d526380 100644 --- a/internal/config/testdata/test3.yaml +++ b/internal/config/testdata/test3.yaml @@ -1,5 +1,3 @@ advanced: - scraper-deterrence: - enabled: true rate-limit: requests: 5000 diff --git a/internal/middleware/nollamas.go b/internal/middleware/nollamas.go deleted file mode 100644 index 6191ebe3b..000000000 --- a/internal/middleware/nollamas.go +++ /dev/null @@ -1,385 +0,0 @@ -// GoToSocial -// Copyright (C) GoToSocial Authors admin@gotosocial.org -// SPDX-License-Identifier: AGPL-3.0-or-later -// -// This program is free software: you can redistribute it and/or modify -// it under the terms of the GNU Affero General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU Affero General Public License for more details. -// -// You should have received a copy of the GNU Affero General Public License -// along with this program. If not, see . - -package middleware - -import ( - "context" - "crypto/rand" - "crypto/sha256" - "crypto/subtle" - "encoding/hex" - "hash" - "io" - "net/http" - "strconv" - "time" - - apimodel "code.superseriousbusiness.org/gotosocial/internal/api/model" - apiutil "code.superseriousbusiness.org/gotosocial/internal/api/util" - "code.superseriousbusiness.org/gotosocial/internal/config" - "code.superseriousbusiness.org/gotosocial/internal/gtscontext" - "code.superseriousbusiness.org/gotosocial/internal/gtserror" - "code.superseriousbusiness.org/gotosocial/internal/log" - "code.superseriousbusiness.org/gotosocial/internal/oauth" - "codeberg.org/gruf/go-bitutil" - "codeberg.org/gruf/go-byteutil" - "github.com/gin-gonic/gin" -) - -// NoLLaMas returns a piece of HTTP middleware that provides a deterrence -// on routes it is applied to, against bots and scrapers. It generates a -// unique but deterministic challenge for each HTTP client within an hour -// TTL that requires a proof-of-work solution to pass onto the next handler. -// On successful solution, the client is provided a cookie that allows them -// to bypass this check within that hour TTL. The outcome of this is that it -// should make scraping of these endpoints economically unfeasible, when enabled, -// and with an absurdly minimal performance impact. The downside is that it -// requires javascript to be enabled on the client to pass the middleware check. -// -// Heavily inspired by: https://github.com/TecharoHQ/anubis -func NoLLaMas( - cookiePolicy apiutil.CookiePolicy, - getInstanceV1 func(context.Context) (*apimodel.InstanceV1, gtserror.WithCode), -) gin.HandlerFunc { - - if !config.GetAdvancedScraperDeterrenceEnabled() { - // NoLLaMas middleware disabled. - return func(*gin.Context) {} - } - - var seed [32]byte - - // Read random data for the token seed. - _, err := io.ReadFull(rand.Reader, seed[:]) - if err != nil { - panic(err) - } - - // Configure nollamas. - var nollamas nollamas - nollamas.entropy = seed - nollamas.ttl = time.Hour - nollamas.rounds = config.GetAdvancedScraperDeterrenceDifficulty() - nollamas.getInstanceV1 = getInstanceV1 - nollamas.policy = cookiePolicy - return nollamas.Serve -} - -// i.e. hash slice length. -const hashLen = sha256.Size - -// i.e. hex.EncodedLen(hashLen). -const encodedHashLen = 2 * hashLen - -// hashWithBufs encompasses a hash along -// with the necessary buffers to generate -// a hashsum and then encode that sum. -type hashWithBufs struct { - hash hash.Hash - hbuf [hashLen]byte - ebuf [encodedHashLen]byte -} - -// write is a passthrough to hash.Hash{}.Write(). -func (h *hashWithBufs) write(b []byte) { - _, _ = h.hash.Write(b) -} - -// writeString is a passthrough to hash.Hash{}.Write([]byte(s)). -func (h *hashWithBufs) writeString(s string) { - _, _ = h.hash.Write(byteutil.S2B(s)) -} - -// EncodedSum returns the hex encoded sum of hash.Sum(). -func (h *hashWithBufs) EncodedSum() string { - _ = h.hash.Sum(h.hbuf[:0]) - hex.Encode(h.ebuf[:], h.hbuf[:]) - return string(h.ebuf[:]) -} - -// Reset will reset hash and buffers. -func (h *hashWithBufs) Reset() { - h.ebuf = [encodedHashLen]byte{} - h.hbuf = [hashLen]byte{} - h.hash.Reset() -} - -type nollamas struct { - // our instance cookie policy. - policy apiutil.CookiePolicy - - // unique entropy - // to prevent hashes - // being guessable - entropy [32]byte - - // success cookie TTL - ttl time.Duration - - // rounds determines roughly how - // many hash-encode rounds each - // client is required to complete. - rounds uint32 - - // extra fields required for - // our template rendering. - getInstanceV1 func(ctx context.Context) (*apimodel.InstanceV1, gtserror.WithCode) -} - -func (m *nollamas) Serve(c *gin.Context) { - if c.Request.Method != http.MethodGet { - // Only interested in protecting - // crawlable 'GET' endpoints. - c.Next() - return - } - - // Extract request context. - ctx := c.Request.Context() - - if ctx.Value(oauth.SessionAuthorizedToken) != nil { - // Don't guard against requests - // providing valid OAuth tokens. - c.Next() - return - } - - if gtscontext.HTTPSignature(ctx) != "" { - // Don't guard against requests - // providing HTTP signatures. - c.Next() - return - } - - // Prepare new hash with buffers. - hash := hashWithBufs{hash: sha256.New()} - - // Extract client fingerprint data. - userAgent := c.GetHeader("User-Agent") - clientIP := c.ClientIP() - - // Generate a unique token for this request, - // only valid for a period of now +- m.ttl. - token := m.getToken(&hash, userAgent, clientIP) - - // Check for a provided success token. - cookie, _ := c.Cookie("gts-nollamas") - - // Check whether passed cookie - // is the expected success token. - if subtle.ConstantTimeCompare( - byteutil.S2B(cookie), - byteutil.S2B(token), - ) == 1 { - - // They passed us a valid, expected - // token. They already passed checks. - c.Next() - return - } - - // From here-on out, all - // possibilities are handled - // by us. Prevent further http - // handlers from being called. - c.Abort() - - // Generate challenge for this unique (yet deterministic) token, - // returning seed, wanted 'challenge' result and expected solution. - seed, challenge, solution := m.getChallenge(&hash, token) - - // Prepare new log entry. - l := log.WithContext(ctx). - WithField("userAgent", userAgent). - WithField("seed", seed). - WithField("rounds", solution) - - // Extract and parse query. - query := c.Request.URL.Query() - - // Check query to see if an in-progress - // challenge solution has been provided. - nonce := query.Get("nollamas_solution") - if nonce == "" { - - // No solution given, likely new client! - // Simply present them with challenge. - m.renderChallenge(c, seed, challenge) - l.Info("posing new challenge") - return - } - - // Check nonce matches expected. - if subtle.ConstantTimeCompare( - byteutil.S2B(solution), - byteutil.S2B(nonce), - ) != 1 { - - // Their nonce failed, re-challenge them. - m.renderChallenge(c, challenge, solution) - l.Infof("invalid solution provided: %s", nonce) - return - } - - l.Info("challenge passed") - - // Drop solution query and encode. - query.Del("nollamas_solution") - c.Request.URL.RawQuery = query.Encode() - - // They passed the challenge! Set success token - // cookie and allow them to continue to next handlers. - m.policy.SetCookie(c, "gts-nollamas", token, int(m.ttl/time.Second), "/") - c.Redirect(http.StatusTemporaryRedirect, c.Request.URL.RequestURI()) -} - -func (m *nollamas) renderChallenge(c *gin.Context, seed, challenge string) { - // Fetch current instance information for templating vars. - instance, errWithCode := m.getInstanceV1(c.Request.Context()) - if errWithCode != nil { - apiutil.ErrorHandler(c, errWithCode, m.getInstanceV1) - return - } - - // Write templated challenge response to client. - apiutil.TemplateWebPage(c, apiutil.WebPage{ - Template: "nollamas.tmpl", - Instance: instance, - Stylesheets: []string{ - "/assets/dist/nollamas.css", - // Include fork-awesome stylesheet - // to get nice loading spinner. - "/assets/Fork-Awesome/css/fork-awesome.min.css", - }, - Extra: map[string]any{ - "seed": seed, - "challenge": challenge, - }, - Javascript: []apiutil.JavascriptEntry{ - { - Src: "/assets/dist/nollamas.js", - Defer: true, - }, - }, - }) -} - -// getToken generates a unique yet deterministic token for given HTTP request -// details, seeded by runtime generated entropy data and ttl rounded timestamp. -func (m *nollamas) getToken(hash *hashWithBufs, userAgent, clientIP string) string { - - // Reset before - // using hash. - hash.Reset() - - // Use our unique entropy to seed hash, - // to ensure we have cryptographically - // unique, yet deterministic, tokens - // generated for a given http client. - hash.write(m.entropy[:]) - - // Also seed the generated input with - // current time rounded to TTL, so our - // single comparison handles expiries. - now := time.Now().Round(m.ttl).Unix() - hash.write([]byte{ - byte(now >> 56), - byte(now >> 48), - byte(now >> 40), - byte(now >> 32), - byte(now >> 24), - byte(now >> 16), - byte(now >> 8), - byte(now), - }) - - // Append client request data. - hash.writeString(userAgent) - hash.writeString(clientIP) - - // Return hex encoded hash. - return hash.EncodedSum() -} - -// getChallenge prepares a new challenge given the deterministic input token for this request. -// it will return an input seed string, a challenge string which is the end result the client -// should be looking for, and the solution for this such that challenge = hex(sha256(seed + solution)). -// the solution will always be a string-encoded 64bit integer calculated from m.rounds + random jitter. -func (m *nollamas) getChallenge(hash *hashWithBufs, token string) (seed, challenge, solution string) { - - // For their unique seed string just use a - // single portion of their 'success' token. - // SHA256 is not yet cracked, this is not an - // application of a hash requiring serious - // cryptographic security and it rotates on - // a TTL basis, so it should be fine. - seed = token[:len(token)/4] - - // BEFORE resetting the hash, get the last - // two bytes of NON-hex-encoded data from - // token generation to use for random jitter. - // This is taken from the end of the hash as - // this is the "unseen" end part of token. - // - // (if we used hex-encoded data it would - // only ever be '0-9' or 'a-z' ASCII chars). - // - // Security-wise, same applies as-above. - jitter := int16(hash.hbuf[len(hash.hbuf)-2]) | - int16(hash.hbuf[len(hash.hbuf)-1])<<8 - - var rounds int64 - switch { - // For some small percentage of - // clients we purposely low-ball - // their rounds required, to make - // it so gaming it with a starting - // nonce value may suddenly fail. - case jitter%37 == 0: - rounds = int64(m.rounds/10) + int64(jitter/10) - case jitter%31 == 0: - rounds = int64(m.rounds/5) + int64(jitter/5) - case jitter%29 == 0: - rounds = int64(m.rounds/3) + int64(jitter/3) - case jitter%13 == 0: - rounds = int64(m.rounds/2) + int64(jitter/2) - - // Determine an appropriate number of hash rounds - // we want the client to perform on input seed. This - // is determined as configured m.rounds +- jitter. - // This will be the 'solution' to create 'challenge'. - default: - rounds = int64(m.rounds) + int64(jitter) //nolint:gosec - } - - // Encode (positive) determined hash rounds as string. - solution = strconv.FormatInt(bitutil.Abs64(rounds), 10) - - // Reset before - // using hash. - hash.Reset() - - // Calculate the expected result - // of hex(sha256(seed + solution)), - // i.e. the proposed 'challenge'. - hash.writeString(seed) - hash.writeString(solution) - challenge = hash.EncodedSum() - - return -} diff --git a/internal/middleware/nollamas_test.go b/internal/middleware/nollamas_test.go deleted file mode 100644 index f6b8e0e02..000000000 --- a/internal/middleware/nollamas_test.go +++ /dev/null @@ -1,178 +0,0 @@ -// GoToSocial -// Copyright (C) GoToSocial Authors admin@gotosocial.org -// SPDX-License-Identifier: AGPL-3.0-or-later -// -// This program is free software: you can redistribute it and/or modify -// it under the terms of the GNU Affero General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU Affero General Public License for more details. -// -// You should have received a copy of the GNU Affero General Public License -// along with this program. If not, see . - -package middleware_test - -import ( - "context" - "crypto/sha256" - "encoding/hex" - "io" - "net/http" - "net/http/httptest" - "slices" - "strconv" - "strings" - "testing" - - "code.superseriousbusiness.org/gotosocial/internal/api/model" - apiutil "code.superseriousbusiness.org/gotosocial/internal/api/util" - "code.superseriousbusiness.org/gotosocial/internal/config" - "code.superseriousbusiness.org/gotosocial/internal/gtserror" - "code.superseriousbusiness.org/gotosocial/internal/middleware" - "code.superseriousbusiness.org/gotosocial/internal/router" - "codeberg.org/gruf/go-byteutil" - "github.com/gin-gonic/gin" - "github.com/stretchr/testify/assert" -) - -func TestNoLLaMasMiddleware(t *testing.T) { - // Gin test engine. - e := gin.New() - - // Setup necessary configuration variables. - config.SetAdvancedScraperDeterrenceEnabled(true) - config.SetWebTemplateBaseDir("../../web/template") - - // Load templates into engine. - err := router.LoadTemplates(e) - assert.NoError(t, err) - - // Add middleware to the gin engine handler stack. - middleware := middleware.NoLLaMas(apiutil.CookiePolicy{}, getInstanceV1) - e.Use(middleware) - - // Set test handler we can - // easily check if was used. - e.Handle("GET", "/", testHandler) - - // Test with differing user-agents. - for _, userAgent := range []string{ - "CURL", - "Mozilla FireSox", - "Google Gnome", - } { - testNoLLaMasMiddleware(t, e, userAgent) - } -} - -func testNoLLaMasMiddleware(t *testing.T, e *gin.Engine, userAgent string) { - // Prepare a test request for gin engine. - r := httptest.NewRequest("GET", "/", nil) - r.Header.Set("User-Agent", userAgent) - rw := httptest.NewRecorder() - - // Pass req through - // engine handler. - e.ServeHTTP(rw, r) - - // Get http result. - res := rw.Result() - - // It should have been stopped - // by middleware and NOT used - // the expected test handler. - ok := usedTestHandler(res) - assert.False(t, ok) - - // Read entire response body. - b, err := io.ReadAll(res.Body) - if err != nil { - panic(err) - } - - var seed string - var challenge string - - // Parse output body and find the challenge / difficulty. - for _, line := range strings.Split(string(b), "\n") { - line = strings.TrimSpace(line) - switch { - case strings.HasPrefix(line, "data-nollamas-seed=\""): - line = line[20:] - line = line[:len(line)-1] - seed = line - case strings.HasPrefix(line, "data-nollamas-challenge=\""): - line = line[25:] - line = line[:len(line)-1] - challenge = line - } - } - - // Ensure valid posed challenge. - assert.NotEmpty(t, challenge) - assert.NotEmpty(t, seed) - - // Prepare a test request for gin engine. - r = httptest.NewRequest("GET", "/", nil) - r.Header.Set("User-Agent", userAgent) - rw = httptest.NewRecorder() - - t.Logf("seed=%s", seed) - t.Logf("challenge=%s", challenge) - - // Now compute and set solution query paramater. - solution := computeSolution(seed, challenge) - r.URL.RawQuery = "nollamas_solution=" + solution - t.Logf("solution=%s", solution) - - // Pass req through - // engine handler. - e.ServeHTTP(rw, r) - - // Get http result. - res = rw.Result() - - // Should have received redirect. - uri, err := res.Location() - assert.NoError(t, err) - assert.Equal(t, uri.String(), "/") - - // Ensure our expected solution cookie (to bypass challenge) was set. - ok = slices.ContainsFunc(res.Cookies(), func(c *http.Cookie) bool { - return c.Name == "gts-nollamas" - }) - assert.True(t, ok) -} - -// computeSolution does the functional equivalent of our nollamas workerTask.js. -func computeSolution(seed, challenge string) string { - for i := 0; ; i++ { - solution := strconv.Itoa(i) - combined := seed + solution - hash := sha256.Sum256(byteutil.S2B(combined)) - encoded := hex.EncodeToString(hash[:]) - if encoded != challenge { - continue - } - return solution - } -} - -// usedTestHandler returns whether testHandler() was used. -func usedTestHandler(res *http.Response) bool { - return res.Header.Get("test-handler") == "ok" -} - -func testHandler(c *gin.Context) { - c.Writer.Header().Set("test-handler", "ok") - c.Writer.WriteHeader(http.StatusOK) -} - -func getInstanceV1(context.Context) (*model.InstanceV1, gtserror.WithCode) { - return &model.InstanceV1{}, nil -} diff --git a/internal/web/web.go b/internal/web/web.go index c7b7c9f25..3468ef63b 100644 --- a/internal/web/web.go +++ b/internal/web/web.go @@ -101,16 +101,12 @@ func (m *Module) Route(r *router.Router, mi ...gin.HandlerFunc) { // Handlers that serve profiles and statuses should use // the SignatureCheck middleware, so that requests with - // content-type application/activity+json can be served, - // and (if enabled) the nollamas middleware, to protect - // against scraping by shitty LLM bullshit. + // content-type application/activity+json can be served. profileGroup := r.AttachGroup(profileGroupPath) profileGroup.Use(mi...) profileGroup.Use(middleware.SignatureCheck(m.isURIBlocked), middleware.CacheControl(middleware.CacheControlConfig{ Directives: []string{"no-store"}, })) - nollamas := middleware.NoLLaMas(m.cookiePolicy, m.processor.InstanceGetV1) - profileGroup.Use(nollamas) profileGroup.Handle(http.MethodGet, "", m.profileGETHandler) // use empty path here since it's the base of the group profileGroup.Handle(http.MethodGet, statusPath, m.threadGETHandler) diff --git a/mkdocs.yml b/mkdocs.yml index 255ae8f7b..02dadf4cf 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -131,7 +131,6 @@ nav: - "advanced/tracing.md" - "advanced/metrics.md" - "advanced/replicating-sqlite.md" - - "advanced/scraper_deterrence.md" - "advanced/sqlite-networked-storage.md" - "Advanced builds": - "advanced/builds/nowasm.md" diff --git a/scripts/bundle_licenses.sh b/scripts/bundle_licenses.sh index ae94837d6..0886f897e 100755 --- a/scripts/bundle_licenses.sh +++ b/scripts/bundle_licenses.sh @@ -33,8 +33,7 @@ for file in $(find ./web/source | license_filter); do done # Copy over misc other licenses -for file in ./LICENSE \ - ./web/source/nollamasworker/sha256.js; do +for file in ./LICENSE; do echo "----------------------------------------------------------" >> "$OUTPUT" echo >> "$OUTPUT" echo "${file}:" >> "$OUTPUT" diff --git a/test/envparsing.sh b/test/envparsing.sh index d61a1a728..36fd847c5 100755 --- a/test/envparsing.sh +++ b/test/envparsing.sh @@ -20,8 +20,6 @@ EXPECT=$(cat << "EOF" "127.0.0.1/32" ], "advanced-rate-limit-requests": 6969, - "advanced-scraper-deterrence-difficulty": 500000, - "advanced-scraper-deterrence-enabled": true, "advanced-sender-multiplier": -1, "advanced-throttling-multiplier": -1, "advanced-throttling-retry-after": 10000000000, @@ -325,8 +323,6 @@ GTS_SYSLOG_ADDRESS='127.0.0.1:6969' \ GTS_ADVANCED_COOKIES_SAMESITE='strict' \ GTS_ADVANCED_RATE_LIMIT_EXCEPTIONS="192.0.2.0/24,127.0.0.1/32" \ GTS_ADVANCED_RATE_LIMIT_REQUESTS=6969 \ -GTS_ADVANCED_SCRAPER_DETERRENCE_DIFFICULTY=500000 \ -GTS_ADVANCED_SCRAPER_DETERRENCE_ENABLED=true \ GTS_ADVANCED_SENDER_MULTIPLIER=-1 \ GTS_ADVANCED_THROTTLING_MULTIPLIER=-1 \ GTS_ADVANCED_THROTTLING_RETRY_AFTER='10s' \ diff --git a/testrig/config.go b/testrig/config.go index 991382bd2..f3e3e6fe0 100644 --- a/testrig/config.go +++ b/testrig/config.go @@ -184,11 +184,6 @@ func testDefaults() config.Configuration { Throttling: config.ThrottlingConfig{ Multiplier: 0, // disabled }, - - ScraperDeterrence: config.ScraperDeterrenceConfig{ - Enabled: envBool("GTS_ADVANCED_SCRAPER_DETERRENCE_ENABLED", false), - Difficulty: uint32(envInt("GTS_ADVANCED_SCRAPER_DETERRENCE_DIFFICULTY", 100000)), //nolint - }, }, SoftwareVersion: "0.0.0-testrig", diff --git a/vendor/golang.org/x/net/context/context.go b/vendor/golang.org/x/net/context/context.go index db1c95fab..d3cb95175 100644 --- a/vendor/golang.org/x/net/context/context.go +++ b/vendor/golang.org/x/net/context/context.go @@ -6,7 +6,7 @@ // cancellation signals, and other request-scoped values across API boundaries // and between processes. // As of Go 1.7 this package is available in the standard library under the -// name [context], and migrating to it can be done automatically with [go fix]. +// name [context]. // // Incoming requests to a server should create a [Context], and outgoing // calls to servers should accept a Context. The chain of function @@ -38,8 +38,6 @@ // // See https://go.dev/blog/context for example code for a server that uses // Contexts. -// -// [go fix]: https://go.dev/cmd/go#hdr-Update_packages_to_use_new_APIs package context import ( @@ -51,36 +49,37 @@ import ( // API boundaries. // // Context's methods may be called by multiple goroutines simultaneously. +// +//go:fix inline type Context = context.Context // Canceled is the error returned by [Context.Err] when the context is canceled // for some reason other than its deadline passing. +// +//go:fix inline var Canceled = context.Canceled // DeadlineExceeded is the error returned by [Context.Err] when the context is canceled // due to its deadline passing. +// +//go:fix inline var DeadlineExceeded = context.DeadlineExceeded // Background returns a non-nil, empty Context. It is never canceled, has no // values, and has no deadline. It is typically used by the main function, // initialization, and tests, and as the top-level Context for incoming // requests. -func Background() Context { - return background -} +// +//go:fix inline +func Background() Context { return context.Background() } // TODO returns a non-nil, empty Context. Code should use context.TODO when // it's unclear which Context to use or it is not yet available (because the // surrounding function has not yet been extended to accept a Context // parameter). -func TODO() Context { - return todo -} - -var ( - background = context.Background() - todo = context.TODO() -) +// +//go:fix inline +func TODO() Context { return context.TODO() } // A CancelFunc tells an operation to abandon its work. // A CancelFunc does not wait for the work to stop. @@ -95,6 +94,8 @@ type CancelFunc = context.CancelFunc // // Canceling this context releases resources associated with it, so code should // call cancel as soon as the operations running in this [Context] complete. +// +//go:fix inline func WithCancel(parent Context) (ctx Context, cancel CancelFunc) { return context.WithCancel(parent) } @@ -108,6 +109,8 @@ func WithCancel(parent Context) (ctx Context, cancel CancelFunc) { // // Canceling this context releases resources associated with it, so code should // call cancel as soon as the operations running in this [Context] complete. +// +//go:fix inline func WithDeadline(parent Context, d time.Time) (Context, CancelFunc) { return context.WithDeadline(parent, d) } @@ -122,6 +125,8 @@ func WithDeadline(parent Context, d time.Time) (Context, CancelFunc) { // defer cancel() // releases resources if slowOperation completes before timeout elapses // return slowOperation(ctx) // } +// +//go:fix inline func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) { return context.WithTimeout(parent, timeout) } @@ -139,6 +144,8 @@ func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) { // interface{}, context keys often have concrete type // struct{}. Alternatively, exported context key variables' static // type should be a pointer or interface. +// +//go:fix inline func WithValue(parent Context, key, val interface{}) Context { return context.WithValue(parent, key, val) } diff --git a/vendor/golang.org/x/net/http2/config.go b/vendor/golang.org/x/net/http2/config.go index ca645d9a1..02fe0c2d4 100644 --- a/vendor/golang.org/x/net/http2/config.go +++ b/vendor/golang.org/x/net/http2/config.go @@ -55,7 +55,7 @@ func configFromServer(h1 *http.Server, h2 *Server) http2Config { PermitProhibitedCipherSuites: h2.PermitProhibitedCipherSuites, CountError: h2.CountError, } - fillNetHTTPServerConfig(&conf, h1) + fillNetHTTPConfig(&conf, h1.HTTP2) setConfigDefaults(&conf, true) return conf } @@ -81,7 +81,7 @@ func configFromTransport(h2 *Transport) http2Config { } if h2.t1 != nil { - fillNetHTTPTransportConfig(&conf, h2.t1) + fillNetHTTPConfig(&conf, h2.t1.HTTP2) } setConfigDefaults(&conf, false) return conf @@ -120,3 +120,45 @@ func adjustHTTP1MaxHeaderSize(n int64) int64 { const typicalHeaders = 10 // conservative return n + typicalHeaders*perFieldOverhead } + +func fillNetHTTPConfig(conf *http2Config, h2 *http.HTTP2Config) { + if h2 == nil { + return + } + if h2.MaxConcurrentStreams != 0 { + conf.MaxConcurrentStreams = uint32(h2.MaxConcurrentStreams) + } + if h2.MaxEncoderHeaderTableSize != 0 { + conf.MaxEncoderHeaderTableSize = uint32(h2.MaxEncoderHeaderTableSize) + } + if h2.MaxDecoderHeaderTableSize != 0 { + conf.MaxDecoderHeaderTableSize = uint32(h2.MaxDecoderHeaderTableSize) + } + if h2.MaxConcurrentStreams != 0 { + conf.MaxConcurrentStreams = uint32(h2.MaxConcurrentStreams) + } + if h2.MaxReadFrameSize != 0 { + conf.MaxReadFrameSize = uint32(h2.MaxReadFrameSize) + } + if h2.MaxReceiveBufferPerConnection != 0 { + conf.MaxUploadBufferPerConnection = int32(h2.MaxReceiveBufferPerConnection) + } + if h2.MaxReceiveBufferPerStream != 0 { + conf.MaxUploadBufferPerStream = int32(h2.MaxReceiveBufferPerStream) + } + if h2.SendPingTimeout != 0 { + conf.SendPingTimeout = h2.SendPingTimeout + } + if h2.PingTimeout != 0 { + conf.PingTimeout = h2.PingTimeout + } + if h2.WriteByteTimeout != 0 { + conf.WriteByteTimeout = h2.WriteByteTimeout + } + if h2.PermitProhibitedCipherSuites { + conf.PermitProhibitedCipherSuites = true + } + if h2.CountError != nil { + conf.CountError = h2.CountError + } +} diff --git a/vendor/golang.org/x/net/http2/config_go124.go b/vendor/golang.org/x/net/http2/config_go124.go deleted file mode 100644 index 5b516c55f..000000000 --- a/vendor/golang.org/x/net/http2/config_go124.go +++ /dev/null @@ -1,61 +0,0 @@ -// Copyright 2024 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build go1.24 - -package http2 - -import "net/http" - -// fillNetHTTPServerConfig sets fields in conf from srv.HTTP2. -func fillNetHTTPServerConfig(conf *http2Config, srv *http.Server) { - fillNetHTTPConfig(conf, srv.HTTP2) -} - -// fillNetHTTPTransportConfig sets fields in conf from tr.HTTP2. -func fillNetHTTPTransportConfig(conf *http2Config, tr *http.Transport) { - fillNetHTTPConfig(conf, tr.HTTP2) -} - -func fillNetHTTPConfig(conf *http2Config, h2 *http.HTTP2Config) { - if h2 == nil { - return - } - if h2.MaxConcurrentStreams != 0 { - conf.MaxConcurrentStreams = uint32(h2.MaxConcurrentStreams) - } - if h2.MaxEncoderHeaderTableSize != 0 { - conf.MaxEncoderHeaderTableSize = uint32(h2.MaxEncoderHeaderTableSize) - } - if h2.MaxDecoderHeaderTableSize != 0 { - conf.MaxDecoderHeaderTableSize = uint32(h2.MaxDecoderHeaderTableSize) - } - if h2.MaxConcurrentStreams != 0 { - conf.MaxConcurrentStreams = uint32(h2.MaxConcurrentStreams) - } - if h2.MaxReadFrameSize != 0 { - conf.MaxReadFrameSize = uint32(h2.MaxReadFrameSize) - } - if h2.MaxReceiveBufferPerConnection != 0 { - conf.MaxUploadBufferPerConnection = int32(h2.MaxReceiveBufferPerConnection) - } - if h2.MaxReceiveBufferPerStream != 0 { - conf.MaxUploadBufferPerStream = int32(h2.MaxReceiveBufferPerStream) - } - if h2.SendPingTimeout != 0 { - conf.SendPingTimeout = h2.SendPingTimeout - } - if h2.PingTimeout != 0 { - conf.PingTimeout = h2.PingTimeout - } - if h2.WriteByteTimeout != 0 { - conf.WriteByteTimeout = h2.WriteByteTimeout - } - if h2.PermitProhibitedCipherSuites { - conf.PermitProhibitedCipherSuites = true - } - if h2.CountError != nil { - conf.CountError = h2.CountError - } -} diff --git a/vendor/golang.org/x/net/http2/config_pre_go124.go b/vendor/golang.org/x/net/http2/config_pre_go124.go deleted file mode 100644 index 060fd6c64..000000000 --- a/vendor/golang.org/x/net/http2/config_pre_go124.go +++ /dev/null @@ -1,16 +0,0 @@ -// Copyright 2024 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. - -//go:build !go1.24 - -package http2 - -import "net/http" - -// Pre-Go 1.24 fallback. -// The Server.HTTP2 and Transport.HTTP2 config fields were added in Go 1.24. - -func fillNetHTTPServerConfig(conf *http2Config, srv *http.Server) {} - -func fillNetHTTPTransportConfig(conf *http2Config, tr *http.Transport) {} diff --git a/vendor/golang.org/x/net/http2/gotrack.go b/vendor/golang.org/x/net/http2/gotrack.go index 9933c9f8c..9921ca096 100644 --- a/vendor/golang.org/x/net/http2/gotrack.go +++ b/vendor/golang.org/x/net/http2/gotrack.go @@ -15,21 +15,32 @@ import ( "runtime" "strconv" "sync" + "sync/atomic" ) var DebugGoroutines = os.Getenv("DEBUG_HTTP2_GOROUTINES") == "1" +// Setting DebugGoroutines to false during a test to disable goroutine debugging +// results in race detector complaints when a test leaves goroutines running before +// returning. Tests shouldn't do this, of course, but when they do it generally shows +// up as infrequent, hard-to-debug flakes. (See #66519.) +// +// Disable goroutine debugging during individual tests with an atomic bool. +// (Note that it's safe to enable/disable debugging mid-test, so the actual race condition +// here is harmless.) +var disableDebugGoroutines atomic.Bool + type goroutineLock uint64 func newGoroutineLock() goroutineLock { - if !DebugGoroutines { + if !DebugGoroutines || disableDebugGoroutines.Load() { return 0 } return goroutineLock(curGoroutineID()) } func (g goroutineLock) check() { - if !DebugGoroutines { + if !DebugGoroutines || disableDebugGoroutines.Load() { return } if curGoroutineID() != uint64(g) { @@ -38,7 +49,7 @@ func (g goroutineLock) check() { } func (g goroutineLock) checkNotOn() { - if !DebugGoroutines { + if !DebugGoroutines || disableDebugGoroutines.Load() { return } if curGoroutineID() == uint64(g) { diff --git a/vendor/golang.org/x/net/http2/http2.go b/vendor/golang.org/x/net/http2/http2.go index ea5ae629f..6878f8ecc 100644 --- a/vendor/golang.org/x/net/http2/http2.go +++ b/vendor/golang.org/x/net/http2/http2.go @@ -15,7 +15,6 @@ package http2 // import "golang.org/x/net/http2" import ( "bufio" - "context" "crypto/tls" "errors" "fmt" @@ -255,15 +254,13 @@ func (cw closeWaiter) Wait() { // idle memory usage with many connections. type bufferedWriter struct { _ incomparable - group synctestGroupInterface // immutable - conn net.Conn // immutable - bw *bufio.Writer // non-nil when data is buffered - byteTimeout time.Duration // immutable, WriteByteTimeout + conn net.Conn // immutable + bw *bufio.Writer // non-nil when data is buffered + byteTimeout time.Duration // immutable, WriteByteTimeout } -func newBufferedWriter(group synctestGroupInterface, conn net.Conn, timeout time.Duration) *bufferedWriter { +func newBufferedWriter(conn net.Conn, timeout time.Duration) *bufferedWriter { return &bufferedWriter{ - group: group, conn: conn, byteTimeout: timeout, } @@ -314,24 +311,18 @@ func (w *bufferedWriter) Flush() error { type bufferedWriterTimeoutWriter bufferedWriter func (w *bufferedWriterTimeoutWriter) Write(p []byte) (n int, err error) { - return writeWithByteTimeout(w.group, w.conn, w.byteTimeout, p) + return writeWithByteTimeout(w.conn, w.byteTimeout, p) } // writeWithByteTimeout writes to conn. // If more than timeout passes without any bytes being written to the connection, // the write fails. -func writeWithByteTimeout(group synctestGroupInterface, conn net.Conn, timeout time.Duration, p []byte) (n int, err error) { +func writeWithByteTimeout(conn net.Conn, timeout time.Duration, p []byte) (n int, err error) { if timeout <= 0 { return conn.Write(p) } for { - var now time.Time - if group == nil { - now = time.Now() - } else { - now = group.Now() - } - conn.SetWriteDeadline(now.Add(timeout)) + conn.SetWriteDeadline(time.Now().Add(timeout)) nn, err := conn.Write(p[n:]) n += nn if n == len(p) || nn == 0 || !errors.Is(err, os.ErrDeadlineExceeded) { @@ -417,14 +408,3 @@ func (s *sorter) SortStrings(ss []string) { // makes that struct also non-comparable, and generally doesn't add // any size (as long as it's first). type incomparable [0]func() - -// synctestGroupInterface is the methods of synctestGroup used by Server and Transport. -// It's defined as an interface here to let us keep synctestGroup entirely test-only -// and not a part of non-test builds. -type synctestGroupInterface interface { - Join() - Now() time.Time - NewTimer(d time.Duration) timer - AfterFunc(d time.Duration, f func()) timer - ContextWithTimeout(ctx context.Context, d time.Duration) (context.Context, context.CancelFunc) -} diff --git a/vendor/golang.org/x/net/http2/server.go b/vendor/golang.org/x/net/http2/server.go index 51fca38f6..64085f6e1 100644 --- a/vendor/golang.org/x/net/http2/server.go +++ b/vendor/golang.org/x/net/http2/server.go @@ -176,39 +176,6 @@ type Server struct { // so that we don't embed a Mutex in this struct, which will make the // struct non-copyable, which might break some callers. state *serverInternalState - - // Synchronization group used for testing. - // Outside of tests, this is nil. - group synctestGroupInterface -} - -func (s *Server) markNewGoroutine() { - if s.group != nil { - s.group.Join() - } -} - -func (s *Server) now() time.Time { - if s.group != nil { - return s.group.Now() - } - return time.Now() -} - -// newTimer creates a new time.Timer, or a synthetic timer in tests. -func (s *Server) newTimer(d time.Duration) timer { - if s.group != nil { - return s.group.NewTimer(d) - } - return timeTimer{time.NewTimer(d)} -} - -// afterFunc creates a new time.AfterFunc timer, or a synthetic timer in tests. -func (s *Server) afterFunc(d time.Duration, f func()) timer { - if s.group != nil { - return s.group.AfterFunc(d, f) - } - return timeTimer{time.AfterFunc(d, f)} } type serverInternalState struct { @@ -423,6 +390,9 @@ func (o *ServeConnOpts) handler() http.Handler { // // The opts parameter is optional. If nil, default values are used. func (s *Server) ServeConn(c net.Conn, opts *ServeConnOpts) { + if opts == nil { + opts = &ServeConnOpts{} + } s.serveConn(c, opts, nil) } @@ -438,7 +408,7 @@ func (s *Server) serveConn(c net.Conn, opts *ServeConnOpts, newf func(*serverCon conn: c, baseCtx: baseCtx, remoteAddrStr: c.RemoteAddr().String(), - bw: newBufferedWriter(s.group, c, conf.WriteByteTimeout), + bw: newBufferedWriter(c, conf.WriteByteTimeout), handler: opts.handler(), streams: make(map[uint32]*stream), readFrameCh: make(chan readFrameResult), @@ -638,11 +608,11 @@ type serverConn struct { pingSent bool sentPingData [8]byte goAwayCode ErrCode - shutdownTimer timer // nil until used - idleTimer timer // nil if unused + shutdownTimer *time.Timer // nil until used + idleTimer *time.Timer // nil if unused readIdleTimeout time.Duration pingTimeout time.Duration - readIdleTimer timer // nil if unused + readIdleTimer *time.Timer // nil if unused // Owned by the writeFrameAsync goroutine: headerWriteBuf bytes.Buffer @@ -687,12 +657,12 @@ type stream struct { flow outflow // limits writing from Handler to client inflow inflow // what the client is allowed to POST/etc to us state streamState - resetQueued bool // RST_STREAM queued for write; set by sc.resetStream - gotTrailerHeader bool // HEADER frame for trailers was seen - wroteHeaders bool // whether we wrote headers (not status 100) - readDeadline timer // nil if unused - writeDeadline timer // nil if unused - closeErr error // set before cw is closed + resetQueued bool // RST_STREAM queued for write; set by sc.resetStream + gotTrailerHeader bool // HEADER frame for trailers was seen + wroteHeaders bool // whether we wrote headers (not status 100) + readDeadline *time.Timer // nil if unused + writeDeadline *time.Timer // nil if unused + closeErr error // set before cw is closed trailer http.Header // accumulated trailers reqTrailer http.Header // handler's Request.Trailer @@ -848,7 +818,6 @@ type readFrameResult struct { // consumer is done with the frame. // It's run on its own goroutine. func (sc *serverConn) readFrames() { - sc.srv.markNewGoroutine() gate := make(chan struct{}) gateDone := func() { gate <- struct{}{} } for { @@ -881,7 +850,6 @@ type frameWriteResult struct { // At most one goroutine can be running writeFrameAsync at a time per // serverConn. func (sc *serverConn) writeFrameAsync(wr FrameWriteRequest, wd *writeData) { - sc.srv.markNewGoroutine() var err error if wd == nil { err = wr.write.writeFrame(sc) @@ -965,22 +933,22 @@ func (sc *serverConn) serve(conf http2Config) { sc.setConnState(http.StateIdle) if sc.srv.IdleTimeout > 0 { - sc.idleTimer = sc.srv.afterFunc(sc.srv.IdleTimeout, sc.onIdleTimer) + sc.idleTimer = time.AfterFunc(sc.srv.IdleTimeout, sc.onIdleTimer) defer sc.idleTimer.Stop() } if conf.SendPingTimeout > 0 { sc.readIdleTimeout = conf.SendPingTimeout - sc.readIdleTimer = sc.srv.afterFunc(conf.SendPingTimeout, sc.onReadIdleTimer) + sc.readIdleTimer = time.AfterFunc(conf.SendPingTimeout, sc.onReadIdleTimer) defer sc.readIdleTimer.Stop() } go sc.readFrames() // closed by defer sc.conn.Close above - settingsTimer := sc.srv.afterFunc(firstSettingsTimeout, sc.onSettingsTimer) + settingsTimer := time.AfterFunc(firstSettingsTimeout, sc.onSettingsTimer) defer settingsTimer.Stop() - lastFrameTime := sc.srv.now() + lastFrameTime := time.Now() loopNum := 0 for { loopNum++ @@ -994,7 +962,7 @@ func (sc *serverConn) serve(conf http2Config) { case res := <-sc.wroteFrameCh: sc.wroteFrame(res) case res := <-sc.readFrameCh: - lastFrameTime = sc.srv.now() + lastFrameTime = time.Now() // Process any written frames before reading new frames from the client since a // written frame could have triggered a new stream to be started. if sc.writingFrameAsync { @@ -1077,7 +1045,7 @@ func (sc *serverConn) handlePingTimer(lastFrameReadTime time.Time) { } pingAt := lastFrameReadTime.Add(sc.readIdleTimeout) - now := sc.srv.now() + now := time.Now() if pingAt.After(now) { // We received frames since arming the ping timer. // Reset it for the next possible timeout. @@ -1141,10 +1109,10 @@ func (sc *serverConn) readPreface() error { errc <- nil } }() - timer := sc.srv.newTimer(prefaceTimeout) // TODO: configurable on *Server? + timer := time.NewTimer(prefaceTimeout) // TODO: configurable on *Server? defer timer.Stop() select { - case <-timer.C(): + case <-timer.C: return errPrefaceTimeout case err := <-errc: if err == nil { @@ -1160,6 +1128,21 @@ var errChanPool = sync.Pool{ New: func() interface{} { return make(chan error, 1) }, } +func getErrChan() chan error { + if inTests { + // Channels cannot be reused across synctest tests. + return make(chan error, 1) + } else { + return errChanPool.Get().(chan error) + } +} + +func putErrChan(ch chan error) { + if !inTests { + errChanPool.Put(ch) + } +} + var writeDataPool = sync.Pool{ New: func() interface{} { return new(writeData) }, } @@ -1167,7 +1150,7 @@ var writeDataPool = sync.Pool{ // writeDataFromHandler writes DATA response frames from a handler on // the given stream. func (sc *serverConn) writeDataFromHandler(stream *stream, data []byte, endStream bool) error { - ch := errChanPool.Get().(chan error) + ch := getErrChan() writeArg := writeDataPool.Get().(*writeData) *writeArg = writeData{stream.id, data, endStream} err := sc.writeFrameFromHandler(FrameWriteRequest{ @@ -1199,7 +1182,7 @@ func (sc *serverConn) writeDataFromHandler(stream *stream, data []byte, endStrea return errStreamClosed } } - errChanPool.Put(ch) + putErrChan(ch) if frameWriteDone { writeDataPool.Put(writeArg) } @@ -1513,7 +1496,7 @@ func (sc *serverConn) goAway(code ErrCode) { func (sc *serverConn) shutDownIn(d time.Duration) { sc.serveG.check() - sc.shutdownTimer = sc.srv.afterFunc(d, sc.onShutdownTimer) + sc.shutdownTimer = time.AfterFunc(d, sc.onShutdownTimer) } func (sc *serverConn) resetStream(se StreamError) { @@ -2118,7 +2101,7 @@ func (sc *serverConn) processHeaders(f *MetaHeadersFrame) error { // (in Go 1.8), though. That's a more sane option anyway. if sc.hs.ReadTimeout > 0 { sc.conn.SetReadDeadline(time.Time{}) - st.readDeadline = sc.srv.afterFunc(sc.hs.ReadTimeout, st.onReadTimeout) + st.readDeadline = time.AfterFunc(sc.hs.ReadTimeout, st.onReadTimeout) } return sc.scheduleHandler(id, rw, req, handler) @@ -2216,7 +2199,7 @@ func (sc *serverConn) newStream(id, pusherID uint32, state streamState) *stream st.flow.add(sc.initialStreamSendWindowSize) st.inflow.init(sc.initialStreamRecvWindowSize) if sc.hs.WriteTimeout > 0 { - st.writeDeadline = sc.srv.afterFunc(sc.hs.WriteTimeout, st.onWriteTimeout) + st.writeDeadline = time.AfterFunc(sc.hs.WriteTimeout, st.onWriteTimeout) } sc.streams[id] = st @@ -2405,7 +2388,6 @@ func (sc *serverConn) handlerDone() { // Run on its own goroutine. func (sc *serverConn) runHandler(rw *responseWriter, req *http.Request, handler func(http.ResponseWriter, *http.Request)) { - sc.srv.markNewGoroutine() defer sc.sendServeMsg(handlerDoneMsg) didPanic := true defer func() { @@ -2454,7 +2436,7 @@ func (sc *serverConn) writeHeaders(st *stream, headerData *writeResHeaders) erro // waiting for this frame to be written, so an http.Flush mid-handler // writes out the correct value of keys, before a handler later potentially // mutates it. - errc = errChanPool.Get().(chan error) + errc = getErrChan() } if err := sc.writeFrameFromHandler(FrameWriteRequest{ write: headerData, @@ -2466,7 +2448,7 @@ func (sc *serverConn) writeHeaders(st *stream, headerData *writeResHeaders) erro if errc != nil { select { case err := <-errc: - errChanPool.Put(errc) + putErrChan(errc) return err case <-sc.doneServing: return errClientDisconnected @@ -2573,7 +2555,7 @@ func (b *requestBody) Read(p []byte) (n int, err error) { if err == io.EOF { b.sawEOF = true } - if b.conn == nil && inTests { + if b.conn == nil { return } b.conn.noteBodyReadFromHandler(b.stream, n, err) @@ -2702,7 +2684,7 @@ func (rws *responseWriterState) writeChunk(p []byte) (n int, err error) { var date string if _, ok := rws.snapHeader["Date"]; !ok { // TODO(bradfitz): be faster here, like net/http? measure. - date = rws.conn.srv.now().UTC().Format(http.TimeFormat) + date = time.Now().UTC().Format(http.TimeFormat) } for _, v := range rws.snapHeader["Trailer"] { @@ -2824,7 +2806,7 @@ func (rws *responseWriterState) promoteUndeclaredTrailers() { func (w *responseWriter) SetReadDeadline(deadline time.Time) error { st := w.rws.stream - if !deadline.IsZero() && deadline.Before(w.rws.conn.srv.now()) { + if !deadline.IsZero() && deadline.Before(time.Now()) { // If we're setting a deadline in the past, reset the stream immediately // so writes after SetWriteDeadline returns will fail. st.onReadTimeout() @@ -2840,9 +2822,9 @@ func (w *responseWriter) SetReadDeadline(deadline time.Time) error { if deadline.IsZero() { st.readDeadline = nil } else if st.readDeadline == nil { - st.readDeadline = sc.srv.afterFunc(deadline.Sub(sc.srv.now()), st.onReadTimeout) + st.readDeadline = time.AfterFunc(deadline.Sub(time.Now()), st.onReadTimeout) } else { - st.readDeadline.Reset(deadline.Sub(sc.srv.now())) + st.readDeadline.Reset(deadline.Sub(time.Now())) } }) return nil @@ -2850,7 +2832,7 @@ func (w *responseWriter) SetReadDeadline(deadline time.Time) error { func (w *responseWriter) SetWriteDeadline(deadline time.Time) error { st := w.rws.stream - if !deadline.IsZero() && deadline.Before(w.rws.conn.srv.now()) { + if !deadline.IsZero() && deadline.Before(time.Now()) { // If we're setting a deadline in the past, reset the stream immediately // so writes after SetWriteDeadline returns will fail. st.onWriteTimeout() @@ -2866,9 +2848,9 @@ func (w *responseWriter) SetWriteDeadline(deadline time.Time) error { if deadline.IsZero() { st.writeDeadline = nil } else if st.writeDeadline == nil { - st.writeDeadline = sc.srv.afterFunc(deadline.Sub(sc.srv.now()), st.onWriteTimeout) + st.writeDeadline = time.AfterFunc(deadline.Sub(time.Now()), st.onWriteTimeout) } else { - st.writeDeadline.Reset(deadline.Sub(sc.srv.now())) + st.writeDeadline.Reset(deadline.Sub(time.Now())) } }) return nil @@ -3147,7 +3129,7 @@ func (w *responseWriter) Push(target string, opts *http.PushOptions) error { method: opts.Method, url: u, header: cloneHeader(opts.Header), - done: errChanPool.Get().(chan error), + done: getErrChan(), } select { @@ -3164,7 +3146,7 @@ func (w *responseWriter) Push(target string, opts *http.PushOptions) error { case <-st.cw: return errStreamClosed case err := <-msg.done: - errChanPool.Put(msg.done) + putErrChan(msg.done) return err } } diff --git a/vendor/golang.org/x/net/http2/timer.go b/vendor/golang.org/x/net/http2/timer.go deleted file mode 100644 index 0b1c17b81..000000000 --- a/vendor/golang.org/x/net/http2/timer.go +++ /dev/null @@ -1,20 +0,0 @@ -// Copyright 2024 The Go Authors. All rights reserved. -// Use of this source code is governed by a BSD-style -// license that can be found in the LICENSE file. -package http2 - -import "time" - -// A timer is a time.Timer, as an interface which can be replaced in tests. -type timer = interface { - C() <-chan time.Time - Reset(d time.Duration) bool - Stop() bool -} - -// timeTimer adapts a time.Timer to the timer interface. -type timeTimer struct { - *time.Timer -} - -func (t timeTimer) C() <-chan time.Time { return t.Timer.C } diff --git a/vendor/golang.org/x/net/http2/transport.go b/vendor/golang.org/x/net/http2/transport.go index f26356b9c..35e390251 100644 --- a/vendor/golang.org/x/net/http2/transport.go +++ b/vendor/golang.org/x/net/http2/transport.go @@ -193,50 +193,6 @@ type Transport struct { type transportTestHooks struct { newclientconn func(*ClientConn) - group synctestGroupInterface -} - -func (t *Transport) markNewGoroutine() { - if t != nil && t.transportTestHooks != nil { - t.transportTestHooks.group.Join() - } -} - -func (t *Transport) now() time.Time { - if t != nil && t.transportTestHooks != nil { - return t.transportTestHooks.group.Now() - } - return time.Now() -} - -func (t *Transport) timeSince(when time.Time) time.Duration { - if t != nil && t.transportTestHooks != nil { - return t.now().Sub(when) - } - return time.Since(when) -} - -// newTimer creates a new time.Timer, or a synthetic timer in tests. -func (t *Transport) newTimer(d time.Duration) timer { - if t.transportTestHooks != nil { - return t.transportTestHooks.group.NewTimer(d) - } - return timeTimer{time.NewTimer(d)} -} - -// afterFunc creates a new time.AfterFunc timer, or a synthetic timer in tests. -func (t *Transport) afterFunc(d time.Duration, f func()) timer { - if t.transportTestHooks != nil { - return t.transportTestHooks.group.AfterFunc(d, f) - } - return timeTimer{time.AfterFunc(d, f)} -} - -func (t *Transport) contextWithTimeout(ctx context.Context, d time.Duration) (context.Context, context.CancelFunc) { - if t.transportTestHooks != nil { - return t.transportTestHooks.group.ContextWithTimeout(ctx, d) - } - return context.WithTimeout(ctx, d) } func (t *Transport) maxHeaderListSize() uint32 { @@ -366,7 +322,7 @@ type ClientConn struct { readerErr error // set before readerDone is closed idleTimeout time.Duration // or 0 for never - idleTimer timer + idleTimer *time.Timer mu sync.Mutex // guards following cond *sync.Cond // hold mu; broadcast on flow/closed changes @@ -534,14 +490,12 @@ func (cs *clientStream) closeReqBodyLocked() { cs.reqBodyClosed = make(chan struct{}) reqBodyClosed := cs.reqBodyClosed go func() { - cs.cc.t.markNewGoroutine() cs.reqBody.Close() close(reqBodyClosed) }() } type stickyErrWriter struct { - group synctestGroupInterface conn net.Conn timeout time.Duration err *error @@ -551,7 +505,7 @@ func (sew stickyErrWriter) Write(p []byte) (n int, err error) { if *sew.err != nil { return 0, *sew.err } - n, err = writeWithByteTimeout(sew.group, sew.conn, sew.timeout, p) + n, err = writeWithByteTimeout(sew.conn, sew.timeout, p) *sew.err = err return n, err } @@ -650,9 +604,9 @@ func (t *Transport) RoundTripOpt(req *http.Request, opt RoundTripOpt) (*http.Res backoff := float64(uint(1) << (uint(retry) - 1)) backoff += backoff * (0.1 * mathrand.Float64()) d := time.Second * time.Duration(backoff) - tm := t.newTimer(d) + tm := time.NewTimer(d) select { - case <-tm.C(): + case <-tm.C: t.vlogf("RoundTrip retrying after failure: %v", roundTripErr) continue case <-req.Context().Done(): @@ -699,6 +653,7 @@ var ( errClientConnUnusable = errors.New("http2: client conn not usable") errClientConnNotEstablished = errors.New("http2: client conn could not be established") errClientConnGotGoAway = errors.New("http2: Transport received Server's graceful shutdown GOAWAY") + errClientConnForceClosed = errors.New("http2: client connection force closed via ClientConn.Close") ) // shouldRetryRequest is called by RoundTrip when a request fails to get @@ -838,14 +793,11 @@ func (t *Transport) newClientConn(c net.Conn, singleUse bool) (*ClientConn, erro pingTimeout: conf.PingTimeout, pings: make(map[[8]byte]chan struct{}), reqHeaderMu: make(chan struct{}, 1), - lastActive: t.now(), + lastActive: time.Now(), } - var group synctestGroupInterface if t.transportTestHooks != nil { - t.markNewGoroutine() t.transportTestHooks.newclientconn(cc) c = cc.tconn - group = t.group } if VerboseLogs { t.vlogf("http2: Transport creating client conn %p to %v", cc, c.RemoteAddr()) @@ -857,7 +809,6 @@ func (t *Transport) newClientConn(c net.Conn, singleUse bool) (*ClientConn, erro // TODO: adjust this writer size to account for frame size + // MTU + crypto/tls record padding. cc.bw = bufio.NewWriter(stickyErrWriter{ - group: group, conn: c, timeout: conf.WriteByteTimeout, err: &cc.werr, @@ -906,7 +857,7 @@ func (t *Transport) newClientConn(c net.Conn, singleUse bool) (*ClientConn, erro // Start the idle timer after the connection is fully initialized. if d := t.idleConnTimeout(); d != 0 { cc.idleTimeout = d - cc.idleTimer = t.afterFunc(d, cc.onIdleTimeout) + cc.idleTimer = time.AfterFunc(d, cc.onIdleTimeout) } go cc.readLoop() @@ -917,7 +868,7 @@ func (cc *ClientConn) healthCheck() { pingTimeout := cc.pingTimeout // We don't need to periodically ping in the health check, because the readLoop of ClientConn will // trigger the healthCheck again if there is no frame received. - ctx, cancel := cc.t.contextWithTimeout(context.Background(), pingTimeout) + ctx, cancel := context.WithTimeout(context.Background(), pingTimeout) defer cancel() cc.vlogf("http2: Transport sending health check") err := cc.Ping(ctx) @@ -1120,7 +1071,7 @@ func (cc *ClientConn) tooIdleLocked() bool { // times are compared based on their wall time. We don't want // to reuse a connection that's been sitting idle during // VM/laptop suspend if monotonic time was also frozen. - return cc.idleTimeout != 0 && !cc.lastIdle.IsZero() && cc.t.timeSince(cc.lastIdle.Round(0)) > cc.idleTimeout + return cc.idleTimeout != 0 && !cc.lastIdle.IsZero() && time.Since(cc.lastIdle.Round(0)) > cc.idleTimeout } // onIdleTimeout is called from a time.AfterFunc goroutine. It will @@ -1186,7 +1137,6 @@ func (cc *ClientConn) Shutdown(ctx context.Context) error { done := make(chan struct{}) cancelled := false // guarded by cc.mu go func() { - cc.t.markNewGoroutine() cc.mu.Lock() defer cc.mu.Unlock() for { @@ -1257,8 +1207,7 @@ func (cc *ClientConn) closeForError(err error) { // // In-flight requests are interrupted. For a graceful shutdown, use Shutdown instead. func (cc *ClientConn) Close() error { - err := errors.New("http2: client connection force closed via ClientConn.Close") - cc.closeForError(err) + cc.closeForError(errClientConnForceClosed) return nil } @@ -1427,7 +1376,6 @@ func (cc *ClientConn) roundTrip(req *http.Request, streamf func(*clientStream)) // // It sends the request and performs post-request cleanup (closing Request.Body, etc.). func (cs *clientStream) doRequest(req *http.Request, streamf func(*clientStream)) { - cs.cc.t.markNewGoroutine() err := cs.writeRequest(req, streamf) cs.cleanupWriteRequest(err) } @@ -1558,9 +1506,9 @@ func (cs *clientStream) writeRequest(req *http.Request, streamf func(*clientStre var respHeaderTimer <-chan time.Time var respHeaderRecv chan struct{} if d := cc.responseHeaderTimeout(); d != 0 { - timer := cc.t.newTimer(d) + timer := time.NewTimer(d) defer timer.Stop() - respHeaderTimer = timer.C() + respHeaderTimer = timer.C respHeaderRecv = cs.respHeaderRecv } // Wait until the peer half-closes its end of the stream, @@ -1753,7 +1701,7 @@ func (cc *ClientConn) awaitOpenSlotForStreamLocked(cs *clientStream) error { // Return a fatal error which aborts the retry loop. return errClientConnNotEstablished } - cc.lastActive = cc.t.now() + cc.lastActive = time.Now() if cc.closed || !cc.canTakeNewRequestLocked() { return errClientConnUnusable } @@ -2092,10 +2040,10 @@ func (cc *ClientConn) forgetStreamID(id uint32) { if len(cc.streams) != slen-1 { panic("forgetting unknown stream id") } - cc.lastActive = cc.t.now() + cc.lastActive = time.Now() if len(cc.streams) == 0 && cc.idleTimer != nil { cc.idleTimer.Reset(cc.idleTimeout) - cc.lastIdle = cc.t.now() + cc.lastIdle = time.Now() } // Wake up writeRequestBody via clientStream.awaitFlowControl and // wake up RoundTrip if there is a pending request. @@ -2121,7 +2069,6 @@ type clientConnReadLoop struct { // readLoop runs in its own goroutine and reads and dispatches frames. func (cc *ClientConn) readLoop() { - cc.t.markNewGoroutine() rl := &clientConnReadLoop{cc: cc} defer rl.cleanup() cc.readerErr = rl.run() @@ -2188,9 +2135,9 @@ func (rl *clientConnReadLoop) cleanup() { if cc.idleTimeout > 0 && unusedWaitTime > cc.idleTimeout { unusedWaitTime = cc.idleTimeout } - idleTime := cc.t.now().Sub(cc.lastActive) + idleTime := time.Now().Sub(cc.lastActive) if atomic.LoadUint32(&cc.atomicReused) == 0 && idleTime < unusedWaitTime && !cc.closedOnIdle { - cc.idleTimer = cc.t.afterFunc(unusedWaitTime-idleTime, func() { + cc.idleTimer = time.AfterFunc(unusedWaitTime-idleTime, func() { cc.t.connPool().MarkDead(cc) }) } else { @@ -2250,9 +2197,9 @@ func (rl *clientConnReadLoop) run() error { cc := rl.cc gotSettings := false readIdleTimeout := cc.readIdleTimeout - var t timer + var t *time.Timer if readIdleTimeout != 0 { - t = cc.t.afterFunc(readIdleTimeout, cc.healthCheck) + t = time.AfterFunc(readIdleTimeout, cc.healthCheck) } for { f, err := cc.fr.ReadFrame() @@ -2998,7 +2945,6 @@ func (cc *ClientConn) Ping(ctx context.Context) error { var pingError error errc := make(chan struct{}) go func() { - cc.t.markNewGoroutine() cc.wmu.Lock() defer cc.wmu.Unlock() if pingError = cc.fr.WritePing(false, p); pingError != nil { @@ -3228,7 +3174,7 @@ func traceGotConn(req *http.Request, cc *ClientConn, reused bool) { cc.mu.Lock() ci.WasIdle = len(cc.streams) == 0 && reused if ci.WasIdle && !cc.lastActive.IsZero() { - ci.IdleTime = cc.t.timeSince(cc.lastActive) + ci.IdleTime = time.Since(cc.lastActive) } cc.mu.Unlock() diff --git a/vendor/modernc.org/sqlite/AUTHORS b/vendor/modernc.org/sqlite/AUTHORS index 61af343a1..268498a3b 100644 --- a/vendor/modernc.org/sqlite/AUTHORS +++ b/vendor/modernc.org/sqlite/AUTHORS @@ -15,6 +15,7 @@ Dan Peterson David Walton Davsk Ltd Co FerretDB Inc. +Harald Albrecht Jaap Aarts Jan Mercl <0xjnml@gmail.com> Josh Bleecher Snyder diff --git a/vendor/modernc.org/sqlite/CONTRIBUTORS b/vendor/modernc.org/sqlite/CONTRIBUTORS index e9ea3aff5..625b7dcd2 100644 --- a/vendor/modernc.org/sqlite/CONTRIBUTORS +++ b/vendor/modernc.org/sqlite/CONTRIBUTORS @@ -17,6 +17,8 @@ David Walton Elle Mouton FlyingOnion <731677080@qq.com> Gleb Sakhnov +Guénaël Muller +Harald Albrecht Jaap Aarts Jan Mercl <0xjnml@gmail.com> Josh Bleecher Snyder diff --git a/vendor/modernc.org/sqlite/sqlite.go b/vendor/modernc.org/sqlite/sqlite.go index 39675a68b..a0b6ca4eb 100644 --- a/vendor/modernc.org/sqlite/sqlite.go +++ b/vendor/modernc.org/sqlite/sqlite.go @@ -254,7 +254,30 @@ func (r *rows) Next(dest []driver.Value) (err error) { return err } - dest[i] = v + if !r.c.intToTime { + dest[i] = v + } else { + // Inspired by mattn/go-sqlite3: + // https://github.com/mattn/go-sqlite3/blob/f76bae4b0044cbba8fb2c72b8e4559e8fbcffd86/sqlite3.go#L2254-L2262 + // but we put make this compatibility optional behind a DSN + // query parameter, because this changes API behavior, so an + // opt-in is needed. + switch r.ColumnTypeDatabaseTypeName(i) { + case "DATE", "DATETIME", "TIMESTAMP": + // Is it a seconds timestamp or a milliseconds + // timestamp? + if v > 1e12 || v < -1e12 { + // time.Unix expects nanoseconds, but this is a + // milliseconds timestamp, so convert ms->ns. + v *= int64(time.Millisecond) + dest[i] = time.Unix(0, v).UTC() + } else { + dest[i] = time.Unix(v, 0) + } + default: + dest[i] = v + } + } case sqlite3.SQLITE_FLOAT: v, err := r.c.columnDouble(r.pstmt, i) if err != nil { @@ -752,8 +775,10 @@ type conn struct { db uintptr // *sqlite3.Xsqlite3 tls *libc.TLS - writeTimeFormat string - beginMode string + writeTimeFormat string + beginMode string + intToTime bool + integerTimeFormat string } func newConn(dsn string) (*conn, error) { @@ -858,6 +883,17 @@ func applyQueryParams(c *conn, query string) error { } c.writeTimeFormat = f } + if v := q.Get("_time_integer_format"); v != "" { + switch v { + case "unix": + case "unix_milli": + case "unix_micro": + case "unix_nano": + default: + return fmt.Errorf("unknown _time_integer_format %q", v) + } + c.integerTimeFormat = v + } if v := q.Get("_txlock"); v != "" { lower := strings.ToLower(v) @@ -867,6 +903,15 @@ func applyQueryParams(c *conn, query string) error { c.beginMode = v } + if v := q.Get("_inttotime"); v != "" { + onoff, err := strconv.ParseBool(v) + if err != nil { + return fmt.Errorf("unknown _inttotime %q, must be 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False", + v) + } + c.intToTime = onoff + } + return nil } @@ -1117,9 +1162,29 @@ func (c *conn) bind(pstmt uintptr, n int, args []driver.NamedValue) (allocs []ui return allocs, err } case time.Time: - if p, err = c.bindText(pstmt, i, c.formatTime(x)); err != nil { - return allocs, err + switch c.integerTimeFormat { + case "unix": + if err := c.bindInt64(pstmt, i, x.Unix()); err != nil { + return allocs, err + } + case "unix_milli": + if err := c.bindInt64(pstmt, i, x.UnixMilli()); err != nil { + return allocs, err + } + case "unix_micro": + if err := c.bindInt64(pstmt, i, x.UnixMicro()); err != nil { + return allocs, err + } + case "unix_nano": + if err := c.bindInt64(pstmt, i, x.UnixNano()); err != nil { + return allocs, err + } + default: + if p, err = c.bindText(pstmt, i, c.formatTime(x)); err != nil { + return allocs, err + } } + case nil: if p, err = c.bindNull(pstmt, i); err != nil { return allocs, err @@ -1913,6 +1978,19 @@ func newDriver() *Driver { return d } // including the timezone specifier. If this parameter is not specified, then // the default String() format will be used. // +// _time_integer_format: The name of a integer format to use when writing time values. +// By default, the time is stored as string and the format can be set with _time_format +// parameter. If _time_integer_format is set, the time will be stored as an integer and +// the integer value will depend on the integer format. +// If you decide to set both _time_format and _time_integer_format, the time will be +// converted as integer and the _time_format value will be ignored. +// Currently the supported value are "unix","unix_milli", "unix_micro" and "unix_nano", +// which corresponds to seconds, milliseconds, microseconds or nanoseconds +// since unixepoch (1 January 1970 00:00:00 UTC). +// +// _inttotime: Enable conversion of time column (DATE, DATETIME,TIMESTAMP) from integer +// to time if the field contain integer (int64). +// // _txlock: The locking behavior to use when beginning a transaction. May be // "deferred" (the default), "immediate", or "exclusive" (case insensitive). See: // https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions diff --git a/vendor/modules.txt b/vendor/modules.txt index e2faccc3c..5defdf9b4 100644 --- a/vendor/modules.txt +++ b/vendor/modules.txt @@ -1197,8 +1197,8 @@ golang.org/x/image/webp golang.org/x/mod/internal/lazyregexp golang.org/x/mod/module golang.org/x/mod/semver -# golang.org/x/net v0.43.0 -## explicit; go 1.23.0 +# golang.org/x/net v0.44.0 +## explicit; go 1.24.0 golang.org/x/net/bpf golang.org/x/net/context golang.org/x/net/html @@ -1433,7 +1433,7 @@ modernc.org/mathutil # modernc.org/memory v1.11.0 ## explicit; go 1.23.0 modernc.org/memory -# modernc.org/sqlite v1.38.2 => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.38.2-concurrency-workaround +# modernc.org/sqlite v1.39.0 => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.39.0-concurrency-workaround ## explicit; go 1.23.0 modernc.org/sqlite modernc.org/sqlite/lib @@ -1441,4 +1441,4 @@ modernc.org/sqlite/lib ## explicit; go 1.22.0 mvdan.cc/xurls/v2 # github.com/go-swagger/go-swagger => codeberg.org/superseriousbusiness/go-swagger v0.32.3-gts-go1.23-fix -# modernc.org/sqlite => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.38.2-concurrency-workaround +# modernc.org/sqlite => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.39.0-concurrency-workaround diff --git a/web/source/.eslintignore b/web/source/.eslintignore index 4ce4ebebe..ae325e9c8 100644 --- a/web/source/.eslintignore +++ b/web/source/.eslintignore @@ -1,4 +1,3 @@ node_modules prism.js prism.css -nollamasworker/sha256.js \ No newline at end of file diff --git a/web/source/css/_media-wrapper.css b/web/source/css/_media-wrapper.css index 856129f17..e0422d03d 100644 --- a/web/source/css/_media-wrapper.css +++ b/web/source/css/_media-wrapper.css @@ -390,7 +390,7 @@ in "normal" width when caption is rendered on the right side. */ - padding: 0; + padding: 0 0 0 1rem; /* Let it have a bit more width if it diff --git a/web/source/css/nollamas.css b/web/source/css/nollamas.css deleted file mode 100644 index 5fe861364..000000000 --- a/web/source/css/nollamas.css +++ /dev/null @@ -1,30 +0,0 @@ -/* - GoToSocial - Copyright (C) GoToSocial Authors admin@gotosocial.org - SPDX-License-Identifier: AGPL-3.0-or-later - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU Affero General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ - -.nollamas { - display: flex; - flex-direction: column; - - .nollamas-status { - display: flex; - flex-direction: column; - align-self: center; - align-items: center; - } -} diff --git a/web/source/index.js b/web/source/index.js index b9d608318..6a218cd08 100644 --- a/web/source/index.js +++ b/web/source/index.js @@ -73,24 +73,6 @@ skulk({ ["babelify", { global: true }] ], }, - nollamas: { - entryFile: "nollamas", - outputFile: "nollamas.js", - preset: ["js"], - prodCfg: prodCfg, - transform: [ - ["babelify", { global: true }] - ], - }, - nollamasworker: { - entryFile: "nollamasworker", - outputFile: "nollamasworker.js", - preset: ["js"], - prodCfg: prodCfg, - transform: [ - ["babelify", { global: true }] - ], - }, settings: { entryFile: "settings", outputFile: "settings.js", diff --git a/web/source/nollamas/index.js b/web/source/nollamas/index.js deleted file mode 100644 index b04a97c23..000000000 --- a/web/source/nollamas/index.js +++ /dev/null @@ -1,129 +0,0 @@ -/* - GoToSocial - Copyright (C) GoToSocial Authors admin@gotosocial.org - SPDX-License-Identifier: AGPL-3.0-or-later - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU Affero General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ - -const explanation = "Your browser is currently solving a proof-of-work challenge designed to deter \"ai\" scrapers. This should take no more than a few seconds..."; - -document.addEventListener("DOMContentLoaded", function() { - // Get the nollamas section container. - const nollamas = document.querySelector(".nollamas"); - - // Add some "loading" text to show that - // a proof-of-work captcha is being done. - const p = this.createElement("p"); - p.className = "nollamas-explanation"; - p.appendChild(document.createTextNode(explanation)); - nollamas.appendChild(p); - - // Add a loading spinner, but only - // animate it if motion is allowed. - const spinner = document.createElement("i"); - spinner.className = "fa fa-spinner fa-2x fa-fw nollamas-status"; - if (!window.matchMedia('(prefers-reduced-motion: reduce)').matches) { - spinner.className += " fa-pulse"; - } - spinner.setAttribute("title","Solving..."); - spinner.setAttribute("aria-label", "Solving"); - nollamas.appendChild(spinner); - - // Read the challenge and difficulty from - // data attributes on the nollamas section. - const seed = nollamas.dataset.nollamasSeed; - const challenge = nollamas.dataset.nollamasChallenge; - const threads = navigator.hardwareConcurrency; - if (typeof(threads) != "number" || threads < 1) { threads = 1; } - - console.log("seed:", seed); // eslint-disable-line no-console - console.log("challenge:", challenge); // eslint-disable-line no-console - console.log("threads:", threads); // eslint-disable-line no-console - - // Create an array to track - // all workers such that we - // can terminate them all. - const workers = []; - const terminateAll = () => { workers.forEach((worker) => worker.terminate() ); }; - - // Get time before task completion. - const startTime = performance.now(); - - // Prepare and start each worker, - // adding them to the workers array. - for (let i = 0; i < threads; i++) { - const worker = new Worker("/assets/dist/nollamasworker.js"); - workers.push(worker); - - // On any error terminate. - worker.onerror = (ev) => { - console.error("worker error:", ev); // eslint-disable-line no-console - terminateAll(); - }; - - // Post worker message, where each - // worker will compute nonces in range: - // $threadNumber + $totalThreads * n - worker.postMessage({ - challenge: challenge, - threads: threads, - thread: i, - seed: seed, - }); - - // Set main on-success function. - worker.onmessage = function (e) { - if (e.data.done) { - // Stop workers. - terminateAll(); - - // Log which thread found the solution. - console.log("solution from:", e.data.thread); // eslint-disable-line no-console - - // Get total computation duration. - const endTime = performance.now(); - const duration = endTime - startTime; - - // Remove spinner and replace it with a tick - // and info about how long it took to solve. - nollamas.removeChild(spinner); - const solutionWrapper = document.createElement("div"); - solutionWrapper.className = "nollamas-status"; - - const tick = document.createElement("i"); - tick.className = "fa fa-check fa-2x fa-fw"; - tick.setAttribute("title","Solved!"); - tick.setAttribute("aria-label", "Solved!"); - solutionWrapper.appendChild(tick); - - const took = document.createElement("span"); - const solvedText = `Solved after ${e.data.nonce} iterations by worker ${e.data.thread} of ${threads}, in ${duration.toString()}ms!`; - took.appendChild(document.createTextNode(solvedText)); - solutionWrapper.appendChild(took); - - nollamas.appendChild(solutionWrapper); - - // Wait for 500ms before redirecting, to give - // visitors a shot at reading the message, but - // not so long that they have to wait unduly. - setTimeout(() => { - let url = new URL(window.location.href); - url.searchParams.set("nollamas_solution", e.data.nonce); - window.location.replace(url.toString()); - }, 500); - } - }; - } -}); diff --git a/web/source/nollamasworker/index.js b/web/source/nollamasworker/index.js deleted file mode 100644 index 0a7711ce1..000000000 --- a/web/source/nollamasworker/index.js +++ /dev/null @@ -1,56 +0,0 @@ -/* - GoToSocial - Copyright (C) GoToSocial Authors admin@gotosocial.org - SPDX-License-Identifier: AGPL-3.0-or-later - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU Affero General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License - along with this program. If not, see . -*/ - -import sha256 from "./sha256"; - -let compute = async function(seedStr, challengeStr, start, iter) { - const textEncoder = new TextEncoder(); - - let nonce = start; - while (true) { // eslint-disable-line no-constant-condition - - // Create possible solution string from challenge string + nonce. - const solution = textEncoder.encode(seedStr + nonce.toString()); - - // Generate hex encoded SHA256 hashsum of solution. - const hashArray = Array.from(sha256(solution)); - const hashAsHex = hashArray.map(b => b.toString(16).padStart(2, "0")).join(""); - - // Check whether hex encoded - // solution matches challenge. - if (hashAsHex == challengeStr) { - return nonce; - } - - // Iter nonce. - nonce += iter; - } -}; - -onmessage = async function(e) { - const thread = e.data.thread; - const threads = e.data.threads; - console.log("worker started:", thread); // eslint-disable-line no-console - - // Compute nonce value that produces 'challenge', for our allotted thread. - let nonce = await compute(e.data.seed, e.data.challenge, thread, threads); - - // Post the solution nonce back to caller with thread no. - postMessage({ nonce: nonce, done: true, thread: thread }); -}; diff --git a/web/source/nollamasworker/sha256.js b/web/source/nollamasworker/sha256.js deleted file mode 100644 index 553442d7c..000000000 --- a/web/source/nollamasworker/sha256.js +++ /dev/null @@ -1,113 +0,0 @@ -/* - Copyright 2022 Andrea Griffini - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - "Software"), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be - included in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE - LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION - OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - -*/ - -// sha256(data) returns the digest of an input piece of data. -// sha256(none) returns an object you can call .add(data), and .digest() at the end. -// the returned digest is a 32-byte Uint8Array instance with an added .hex() function. -// input should be string (that will be encoded as UTF-8) or an array-like with values 0..255. -// source: https://github.com/6502/sha256 -export default function sha256(data) { - let h0 = 0x6a09e667, h1 = 0xbb67ae85, h2 = 0x3c6ef372, h3 = 0xa54ff53a, - h4 = 0x510e527f, h5 = 0x9b05688c, h6 = 0x1f83d9ab, h7 = 0x5be0cd19, - tsz = 0, bp = 0; - const k = [0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, - 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, - 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, - 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, - 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, - 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, - 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, - 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2], - rrot = (x, n) => (x >>> n) | (x << (32-n)), - w = new Uint32Array(64), - buf = new Uint8Array(64), - process = () => { - for (let j=0,r=0; j<16; j++,r+=4) { - w[j] = (buf[r]<<24) | (buf[r+1]<<16) | (buf[r+2]<<8) | buf[r+3]; - } - for (let j=16; j<64; j++) { - let s0 = rrot(w[j-15], 7) ^ rrot(w[j-15], 18) ^ (w[j-15] >>> 3); - let s1 = rrot(w[j-2], 17) ^ rrot(w[j-2], 19) ^ (w[j-2] >>> 10); - w[j] = (w[j-16] + s0 + w[j-7] + s1) | 0; - } - let a = h0, b = h1, c = h2, d = h3, e = h4, f = h5, g = h6, h = h7; - for (let j=0; j<64; j++) { - let S1 = rrot(e, 6) ^ rrot(e, 11) ^ rrot(e, 25), - ch = (e & f) ^ ((~e) & g), - t1 = (h + S1 + ch + k[j] + w[j]) | 0, - S0 = rrot(a, 2) ^ rrot(a, 13) ^ rrot(a, 22), - maj = (a & b) ^ (a & c) ^ (b & c), - t2 = (S0 + maj) | 0; - h = g; g = f; f = e; e = (d + t1)|0; d = c; c = b; b = a; a = (t1 + t2)|0; - } - h0 = (h0 + a)|0; h1 = (h1 + b)|0; h2 = (h2 + c)|0; h3 = (h3 + d)|0; - h4 = (h4 + e)|0; h5 = (h5 + f)|0; h6 = (h6 + g)|0; h7 = (h7 + h)|0; - bp = 0; - }, - add = data => { - if (typeof data === "string") { - data = typeof TextEncoder === "undefined" ? Buffer.from(data) : (new TextEncoder).encode(data); - } - for (let i=0; i { - buf[bp++] = 0x80; if (bp == 64) {process();} - if (bp + 8 > 64) { - while (bp < 64) {buf[bp++] = 0x00;} - process(); - } - while (bp < 58) {buf[bp++] = 0x00;} - // Max number of bytes is 35,184,372,088,831 - let L = tsz * 8; - buf[bp++] = (L / 1099511627776.) & 255; - buf[bp++] = (L / 4294967296.) & 255; - buf[bp++] = L >>> 24; - buf[bp++] = (L >>> 16) & 255; - buf[bp++] = (L >>> 8) & 255; - buf[bp++] = L & 255; - process(); - let reply = new Uint8Array(32); - reply[ 0] = h0 >>> 24; reply[ 1] = (h0 >>> 16) & 255; reply[ 2] = (h0 >>> 8) & 255; reply[ 3] = h0 & 255; - reply[ 4] = h1 >>> 24; reply[ 5] = (h1 >>> 16) & 255; reply[ 6] = (h1 >>> 8) & 255; reply[ 7] = h1 & 255; - reply[ 8] = h2 >>> 24; reply[ 9] = (h2 >>> 16) & 255; reply[10] = (h2 >>> 8) & 255; reply[11] = h2 & 255; - reply[12] = h3 >>> 24; reply[13] = (h3 >>> 16) & 255; reply[14] = (h3 >>> 8) & 255; reply[15] = h3 & 255; - reply[16] = h4 >>> 24; reply[17] = (h4 >>> 16) & 255; reply[18] = (h4 >>> 8) & 255; reply[19] = h4 & 255; - reply[20] = h5 >>> 24; reply[21] = (h5 >>> 16) & 255; reply[22] = (h5 >>> 8) & 255; reply[23] = h5 & 255; - reply[24] = h6 >>> 24; reply[25] = (h6 >>> 16) & 255; reply[26] = (h6 >>> 8) & 255; reply[27] = h6 & 255; - reply[28] = h7 >>> 24; reply[29] = (h7 >>> 16) & 255; reply[30] = (h7 >>> 8) & 255; reply[31] = h7 & 255; - reply.hex = () => { - let res = ""; - reply.forEach(x => res += ("0" + x.toString(16)).slice(-2)); // eslint-disable-line no-return-assign - return res; - }; - return reply; - }; - if (data === undefined) {return {add, digest};} - add(data); - return digest(); -} diff --git a/web/source/settings/views/user/migration/index.tsx b/web/source/settings/views/user/migration/index.tsx index d2bbbdf12..90abd31b7 100644 --- a/web/source/settings/views/user/migration/index.tsx +++ b/web/source/settings/views/user/migration/index.tsx @@ -29,6 +29,8 @@ import MutationButton from "../../../components/form/mutation-button"; import { useAliasAccountMutation, useMoveAccountMutation } from "../../../lib/query/user"; import { FormContext, useWithFormContext } from "../../../lib/form/context"; import { store } from "../../../redux/store"; +import { useInstanceV1Query } from "../../../lib/query/gts-api"; +import Loading from "../../../components/loading"; export default function Migration() { return ( @@ -142,9 +144,7 @@ function AlsoKnownAsURI({ index, data }) { } function MoveForm({ data: profile }) { - let urlStr = store.getState().login.instanceUrl ?? ""; - let url = new URL(urlStr); - + const instanceURL = store.getState().login.instanceUrl ?? ""; const form = { movedToURI: useTextInput("moved_to_uri", { source: profile, @@ -153,9 +153,22 @@ function MoveForm({ data: profile }) { password: useTextInput("password"), }; - const [submitForm, result] = useFormSubmit(form, useMoveAccountMutation(), { - changedOnly: false, - }); + const [submitForm, result] = useFormSubmit( + form, + useMoveAccountMutation(), + { changedOnly: false }, + ); + + // Load instance data to know the correct + // account domain to provide in form below. + const { + data: instance, + isFetching: isFetchingInstance, + isLoading: isLoadingInstance + } = useInstanceV1Query(); + if (isFetchingInstance || isLoadingInstance) { + return ; + } return (
@@ -170,11 +183,11 @@ function MoveForm({ data: profile }) {
Account handle/username:
-
@{profile.acct}@{url.host}
+
@{profile.acct}@{instance?.account_domain}
Account URI:
-
{urlStr}/users/{profile.username}
+
{instanceURL}/users/{profile.username}

diff --git a/web/template/nollamas.tmpl b/web/template/nollamas.tmpl deleted file mode 100644 index a7511e7ae..000000000 --- a/web/template/nollamas.tmpl +++ /dev/null @@ -1,43 +0,0 @@ -{{- /* -// GoToSocial -// Copyright (C) GoToSocial Authors admin@gotosocial.org -// SPDX-License-Identifier: AGPL-3.0-or-later -// -// This program is free software: you can redistribute it and/or modify -// it under the terms of the GNU Affero General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. -// -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU Affero General Public License for more details. -// -// You should have received a copy of the GNU Affero General Public License -// along with this program. If not, see . -*/ -}} - -{{- with . }} -
-
-

Checking you're not a creepy crawler...

- -
-
-{{- end }} \ No newline at end of file