feat: add robots.txt allowing all crawlers including social bots
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
14
ui/public/robots.txt
Normal file
14
ui/public/robots.txt
Normal file
@@ -0,0 +1,14 @@
|
||||
User-agent: *
|
||||
Allow: /
|
||||
|
||||
User-agent: facebookexternalhit
|
||||
Allow: /
|
||||
|
||||
User-agent: Twitterbot
|
||||
Allow: /
|
||||
|
||||
User-agent: LinkedInBot
|
||||
Allow: /
|
||||
|
||||
User-agent: WhatsApp
|
||||
Allow: /
|
||||
Reference in New Issue
Block a user