mirrors/gotosocial
1
0
Fork 0
mirror of https://github.com/superseriousbusiness/gotosocial.git synced 2025-10-29 19:12:25 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[feature] Block Amazonbot (#2692)

Browse source 
Blocks the Amazon crawler bot.

Closes: #2686
This commit is contained in:
Daenney 2024-02-27 14:25:08 +01:00 • committed by GitHub
commit 6528592dd2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
internal/web/robots.go
View file

@ -64,6 +64,11 @@ Disallow: /
User-agent: WellKnownBot User-agent: WellKnownBot
Disallow: / Disallow: /
# Block Amazonbot, because Amazon.
# https://developer.amazon.com/amazonbot
User-agent: Amazonbot
Disallow: /
# Rules for everything else. # Rules for everything else.
User-agent: * User-agent: *
Crawl-delay: 500 Crawl-delay: 500