mirrors/gotosocial
1
0
Fork 0
mirror of https://github.com/superseriousbusiness/gotosocial.git synced 2025-10-28 22:42:26 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[chore] Update robots.txt (#2856)

Browse source 
This updates the robots.txt based on the list of the ai.robots.txt
repository. We can look at automating that at some point.

It's worth pointing out that some robots, namely the ones by Bytedance,
are known to ignore robots.txt entirely.
This commit is contained in:
Daenney 2024-04-22 11:01:37 +02:00 • committed by GitHub
commit dcab555a6b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 45 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

1
mkdocs.yml
View file

@ -118,6 +118,7 @@ nav:
- "admin/signups.md"
- "admin/federation_modes.md"
- "admin/domain_blocks.md"
- "admin/robots.md"
- "admin/cli.md"
- "admin/backup_and_restore.md"
- "admin/media_caching.md"