mastodon.gamedev.place is one of the many independent Mastodon servers you can use to participate in the fediverse.
Mastodon server focused on game development and related topics.

Server stats:

5.2K
active users

#anubis

1 post1 participant0 posts today

Woke up this morning to yet more Linode alerts and another failed server as a result of AI bots relentlessly scraping my #Gitea instance.

I heard about #Anubis (anubis.techaro.lol) when Xe Iaso (xeiaso.net) was on a recent episode of the #SelfHostedShow podcast and so it seemed like a great opportunity to give it a try. I don't really need "SEO" or any discoverability on Gitea, so hopefully the only downside is that new visitors need to wait a few secs before things load

anubis.techaro.lolMaking sure you're not a bot!

Looking at most recent logs, there are still some scrapers that fall through the cracks, and reach my backend. They use residential IPs, and user agents I can't filter. Luckily, they don't put much pressure on my infra, but... I don't like them there.

The vast majority of them are interested in my forge, so I'm thinking about deploying #Anubis there, on some URLs that humans don't usually visit.

I don't like proof-of-work things, because they penalize the legit visitor too. But there's a point where the passive defenses do not scale anymore. I can still limit the damage, though. But I've got to do some serious log digging to figure out the patterns I can shove behind Anubis.

I will also have to figure out how often the same IP address is used. As in... can I set things up in a way that if the Anubis check fails, I temporarily route that IP into a maze?

Continued thread

As my initial use case for #swad was to stop #AI #bots clogging my DSL upstream, and the #FreeBSD / #poudriere build logs they were downloading in gigabytes aren't secret at all (to the contrary, it can sometimes be helpful to share them when hunting down build issues in the community) I wonder whether I should add a module somewhat similar to #anubis[1] for "guest logins"? 🤔 Might be a lot of work though...

[1] github.com/TecharoHQ/anubis

Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers - TecharoHQ/anubis
GitHubGitHub - TecharoHQ/anubis: Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlersWeighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers - TecharoHQ/anubis

Is everyone who’s using #Anubis SURE that it’s fine? It was already suspicious when it was half open source and used AI art, but now the repo got copied to a GH org “anti-AI AI company” with a domain that doesn’t resolve and the blog was silently updated (but still mentions commercial licensing)? Have you double-checked the install commands don’t do anything nasty and the service is isolated?

👀 Esta mañana al comentar los problemas de Wikimedia con el scrapping, un amigo programador me han hablado del proyecto Anubis github.com/TecharoHQ/anubis/
"Es bastante sencillo y fácil de implementar en cualquier web medio seria, te cargas automáticamente cualquier scrapper (sea de IA sea de lo que sea). Además, no pueden inventar nada que haga que sea rentable el scrapping con eso puesto." #aiscraping #aiscrapers #wikimedia #anubis #iahastaenlaputasopa