mirror of
https://github.com/superseriousbusiness/gotosocial.git
synced 2025-10-29 20:02:26 -05:00
# Description
This PR updates the Chinese documentation to 6c879186 (the latest commit at present).
It also fixed a small typo in the original docs. Since the change is so minor, I didn't make a separate PR.
Last docs/zh update PR: #3884
## Checklist
Please put an x inside each checkbox to indicate that you've read and followed it: `[ ]` -> `[x]`
If this is a documentation change, only the first checkbox must be filled (you can delete the others if you want).
- [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md).
Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4117
Co-authored-by: cdn0x12 <git@cdn0x12.dev>
Co-committed-by: cdn0x12 <git@cdn0x12.dev>
This commit is contained in:
parent
6c87918635
commit
4d6408015b
18 changed files with 817 additions and 115 deletions
|
|
@ -10,8 +10,6 @@ GoToSocial 在主域名上提供一个 `robots.txt` 文件。该文件包含试
|
|||
|
||||
AI 爬虫来自一个[社区维护的仓库][airobots]。目前是手动保持同步的。如果你知道有任何遗漏的爬虫,请给他们提交一个 PR!
|
||||
|
||||
已知有许多 AI 爬虫即便明确匹配其 User-Agent,也会忽略 `robots.txt` 中的条目。这意味着 `robots.txt` 文件并不是确保 AI 爬虫不抓取你的内容的万无一失的方法。
|
||||
|
||||
如果你想完全屏蔽这些爬虫,需要在反向代理中根据 User-Agent 头进行屏蔽,直到 GoToSocial 能够根据 User-Agent 头过滤请求。
|
||||
众所周知,很多 AI 爬虫在 `robots.txt` 不允许其 User-Agent 的情况下,仍然会忽略对应规则并继续抓去内容。这意味着 `robots.txt` 文件并不是确保 AI 爬虫不抓取你的内容的万无一失的方法。除此以外,你可能还需要考虑通过[请求标头过滤](request_filtering_modes.md)来阻止对应 User-Agent,以及启用基于工作证明的[爬虫防护](../advanced/scraper_deterrence.md)。
|
||||
|
||||
[airobots]: https://github.com/ai-robots-txt/ai.robots.txt/
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue