mirrors/gotosocial
1
0
Fork 0
mirror of https://github.com/superseriousbusiness/gotosocial.git synced 2025-11-02 04:32:24 -06:00
Code Issues Projects Releases Packages Wiki Activity Actions

[chore] remove nollamas middleware for now (after discussions with a security advisor) (#4433)

Browse source 
i'll keep this on a separate branch for now while i experiment with other possible alternatives, but for now both our hacky implementation especially, and more popular ones (like anubis) aren't looking too great on the deterrent front: https://github.com/eternal-flame-AD/pow-buster

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4433
Co-authored-by: kim <grufwub@gmail.com>
Co-committed-by: kim <grufwub@gmail.com>
This commit is contained in:
kim 2025-09-17 14:16:53 +02:00 • committed by kim
commit 6801ce299a
28 changed files with 207 additions and 1395 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/locales/zh/admin/robots.md
View file

@ -10,6 +10,6 @@ GoToSocial 在主域名上提供一个 `robots.txt` 文件。该文件包含试
AI 爬虫来自一个[社区维护的仓库][airobots]。目前是手动保持同步的。如果你知道有任何遗漏的爬虫，请给他们提交一个 PR！
众所周知，很多 AI 爬虫在 `robots.txt` 不允许其 User-Agent 的情况下，仍然会忽略对应规则并继续抓去内容。这意味着 `robots.txt` 文件并不是确保 AI 爬虫不抓取你的内容的万无一失的方法。除此以外，你可能还需要考虑通过[请求标头过滤](request_filtering_modes.md)来阻止对应 User-Agent，以及启用基于工作证明的[爬虫防护](../advanced/scraper_deterrence.md)。
众所周知，很多 AI 爬虫在 `robots.txt` 不允许其 User-Agent 的情况下，仍然会忽略对应规则并继续抓去内容。这意味着 `robots.txt` 文件并不是确保 AI 爬虫不抓取你的内容的万无一失的方法。除此以外，你可能还需要考虑通过[请求标头过滤](request_filtering_modes.md)来阻止对应 User-Agent。
[airobots]: https://github.com/ai-robots-txt/ai.robots.txt/