In SquareSpace, you can opt to block AI crawlers in Settings. However it doesn't work since ChatGPT appears in my Analytics. Does anyone know if I could add in Website > Utilities > Website Tools > Code Injection this rule without creating any issues:
User-agent : ChatGPT-User
Disallow: /
Recent searches
Search options
#aicrawlers
0 posts0 participants0 posts today
AI Crawlers stealing your content? Time to fight back! 
LLMs and AI bots are scraping the web, stealing up your data, hogging bandwidth, and even crashing servers under aggressive loads.
Don’t let them freeload! The CrowdSec AI Crawlers Blocklist stops unwanted harvesting before it hurts your site’s performance or privacy.
Regain control over your digital assets: https://crowdsec.net/blog/protect-against-ai-crawlers
So according to the request statistics, since the last rotation of the access log file for the #MacPorts trac this morning, there were:
20.8k requests from IE 3
20.9k requests from IE 4
21.3k requests from IE 5
43 requests from IE 6 and
23 requests from IE 7
These requests came from these Windows versions (roughly 4k per version): CE, 95, 98 (9.5k), NT 4, 2000, XP, NT 5.01(?!), Server 2003, Vista, 7, and 8.0.
I'm sure none of those are AI crawler bots.
Calling all FOSS communities!
Worried about AI crawlers scraping your content or overwhelming your servers? We’ve got your back.
To support open source communities, we’re offering free access to our Platinum AI Crawlers Blocklist. 
Learn how to get started: https://www.crowdsec.net/blog/protecting-foss-with-free-ai-crawlers-blocklist
www.crowdsec.netProtecting FOSS Communities from AI Crawlers with CrowdSecAnnouncing free access to the CrowdSec AI Crawlers Blocklist for all open source projects, to help FOSS communities reduce unwanted traffic from AI bots.
Wikipedia is struggling with voracious AI bot crawlers
https://www.engadget.com/ai/wikipedia-is-struggling-with-voracious-ai-bot-crawlers-121546854.html
Engadget · Wikipedia is struggling with voracious AI bot crawlers
After struggling all night with LLM crawlers, here's a little something I wrote:
A #Fail2Ban filter to block #LLMCrawlers before they start damaging infrastructure. It works by matching HTTP user-agents.
Hopefully this can be of use for other people as well.
Codeberg.orgllm-crawlers-fail2ban-filterThis repository contains a Fail2Ban filter for use with nginx. Its purpose is to block LLM crawlers before they start damaging infrastructure.
Getting #DDOS-ed by #AIcrawlers right now, even though I have a strict robots.txt.
Outgoing traffic suddenly increased (more or less 10 times regular traffic).
I have to block their crap using my firewall. That's NOT OK.
Developers report aggressive AI crawlers overwhelming open-source infrastructure, with LibreNews citing up to 97% of traffic from AI bots on some projects. #AI #OpenSource #TechNews #AIcrawlers #Bots #LibreNews #DeveloperCommunity #Infrastructure #TechIndustry
I wonder if you could devise some sort of reverse LOIC for the damn AI bots crawling the webpages.
95% visitors are bots and crawlers.
