Block AI Crawlers on Your Own Server
Measure how much of your traffic is AI crawlers from your own access log, then block or rate limit them in nginx, Apache or Caddy without losing real visitors.
Filtering by topic #web-server · clear
Measure how much of your traffic is AI crawlers from your own access log, then block or rate limit them in nginx, Apache or Caddy without losing real visitors.
HTTP explained for people who run servers: methods, status codes, the headers that matter, what nginx logs, and where HTTP/3 and TLS fit in.