mirrors/gotosocial
1
0
Fork 0
mirror of https://github.com/superseriousbusiness/gotosocial.git synced 2025-10-29 12:22:25 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[feature] Change instance-stats-randomize to instance-stats-mode with multiple options; implement nodeinfo 2.1 (#3734)

Browse source 
* [feature] Change `instance-stats-randomize` to `instance-stats-mode` with multiple options; implement nodeinfo 2.1

* swaggalaggadingdong
This commit is contained in:
tobi 2025-02-04 16:52:42 +01:00 • committed by GitHub
commit 07d2770995
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 283 additions and 77 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/admin/robots.md
View file

@ -2,6 +2,10 @@
GoToSocial serves a `robots.txt` file on the host domain. This file contains rules that attempt to block known AI scrapers, as well as some other indexers. It also includes some rules to ensure things like API endpoints aren't indexed by search engines since there really isn't any point to them.
## Allow/disallow stats collection
You can allow or disallow crawlers from collecting stats about your instance from the `/nodeinfo/2.0` and `/nodeinfo/2.1` endpoints by changing the setting `instance-stats-mode`, which modifies the `robots.txt` file. See [instance configuration](../configuration/instance.md) for more details.
## AI scrapers
The AI scrapers come from a [community maintained repository][airobots]. It's manually kept in sync for the time being. If you know of any missing robots, please send them a PR!