cm0002@lemmy.world to Technology@lemmy.worldEnglish · 8 days agoOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comexternal-linkmessage-square25fedilinkarrow-up1390arrow-down14cross-posted to: technology@lemmy.ml
arrow-up1386arrow-down1external-linkOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comcm0002@lemmy.world to Technology@lemmy.worldEnglish · 8 days agomessage-square25fedilinkcross-posted to: technology@lemmy.ml
minus-squaretaladar@sh.itjust.workslinkfedilinkEnglisharrow-up9·8 days agoThe tricky bit is recognizing that the requests are all from the same source. Often they use different IP addresses and to even classify requests at all you have to keep extra state around that you wouldn’t need without this anti-social behavior.
The tricky bit is recognizing that the requests are all from the same source. Often they use different IP addresses and to even classify requests at all you have to keep extra state around that you wouldn’t need without this anti-social behavior.