The biggest offender is, surprisingly, cloudflare. They will straight up refuse to serve you any site if your user agent is not one of the mainstream ones. It’s not even “find the traffic light to prove you’re human”, but a page basically saying “fuck you, go away”.
what is more likely to be a bot? a unique and trackable useragent for a semi-niche browser engine, or a vanilla Chromium+Windows which half of everyone uses ?
The biggest offender is, surprisingly, cloudflare. They will straight up refuse to serve you any site if your user agent is not one of the mainstream ones. It’s not even “find the traffic light to prove you’re human”, but a page basically saying “fuck you, go away”.
Well their job is to block weird bot-looking traffic…
what is more likely to be a bot? a unique and trackable useragent for a semi-niche browser engine, or a vanilla Chromium+Windows which half of everyone uses ?