Posted on 11/19/2026 1:21:25 AM PST by John Robinson
Well, it seems the world online has changed recently. I don't know who is doing it, but it seems people are renting out their consumer-grade bandwidth to AI trainers which then use it, in aggregate, as a swarm (of hornets) to scrape "User-Generated Content" from across the web. That stuff is the new gold, in all its glorious New Times Roman flair. Most of the internet is now just AI slop, so we here with wetware are being mined quite hard. Very annoying. As an extreme measure, I have simply denied non-authenticated users from viewing the site. Hopefully, a temporary solution. Though we have temporary solutions that are 25 years and going strong, so idk. Feeling cute, might revert later.
|
Click here: to donate by Credit Card Or here: to donate by PayPal Or by mail to: Free Republic, LLC - PO Box 9771 - Fresno, CA 93794 Thank you very much and God bless you. |
This is quite interesting to me. Can you explain in more detail?
It is a hunch based on the shape and origin of traffic coming into the site. We are being overwhelmed with requests, it has been this way for weeks, more than a month. The nature of the traffic denies an opportunity to use rate limiting measures because there is no sole greed lord like traditional malicious scrapers — though we had those as well, mostly Chinese. We’re talking distributed scraping across ASNs like Comcast, Verizon, Cox, etc., mimicking legitimate user agents (possibly legitimate user agents) but autonomous, coordinated, and very aggressive across hundreds of distinct IPs.
I had been blocking entire commercial ASNs, like virtual hosting, etc. This constitutes thousands of subnets, hundreds of thousands of IPs, and it worked briefly. Then the nature of the traffic adjusted.
This is what actual raw log from the frontend looks like this moment. ASN prefixed (based on IP) mostly Oracle cloud in this clump. I have seen similar but from consumer-grade IPs (like your local ISP.)
AS31898 132.226.90.80 unknown /focus/f-news/3616761/reply?c=107
AS31898 132.226.90.80 unknown /focus/f-news/2358188/posts?page=92
AS31898 144.24.43.115 unknown /focus/news/1266984/reply?c=797
AS31898 137.131.13.234 unknown /focus/news/4288454/reply?c=38
AS31898 137.131.13.234 unknown /focus/f-news/4015843/reply?c=21
AS31898 152.70.129.22 unknown /focus/news/2244320/reply?c=265
AS31898 132.226.90.80 unknown /focus/news/1767923/posts?q=1&;page=161
AS31898 137.131.13.234 unknown /focus/f-news/1397239/replies?c=19
AS31898 132.226.90.80 unknown /focus/f-news/2589815/posts?page=40
AS31898 152.70.129.22 unknown /focus/news/2985149/replies?c=9
AS31898 144.24.43.115 unknown /focus/f-news/882987/reply?c=21
AS31898 129.153.206.251 unknown /focus/f-news/1638721/reply?c=3
AS31898 132.226.90.80 unknown /focus/f-news/1424547/reply?c=96
AS31898 132.226.90.80 unknown /focus/f-news/1685898/reply?c=97
AS31898 137.131.13.234 unknown /focus/news/703262/replies?c=14
AS31898 152.70.129.22 unknown /focus/news/4211245/posts?page=21
AS31898 132.226.90.80 unknown /focus/news/3743995/reply?c=59
AS31898 137.131.13.234 unknown /focus/f-news/3026107/posts?page=50
AS31898 129.153.206.251 unknown /focus/f-news/2811077/posts?page=1
AS31898 137.131.13.234 unknown /focus/news/2896096/posts?page=5
AS31898 152.70.129.22 unknown /focus/f-news/1722891/posts?page=19
AS31898 152.70.129.22 unknown /focus/f-news/1759402/replies?c=68
AS31898 144.24.43.115 unknown /focus/f-news/1707337/reply?c=39
AS31898 144.24.27.8 unknown /focus/f-news/1278035/replies?c=11
AS41608 195.170.172.215 -
AS31898 144.24.14.84 unknown /focus/f-news/2210993/posts?page=36
AS31898 129.153.206.251 unknown /focus/f-news/1205309/reply?c=31
AS31898 129.153.206.251 unknown /focus/f-news/3413644/posts?page=6
AS31898 144.24.14.84 unknown /focus/news/779421/reply?c=10
AS31898 137.131.13.234 unknown /focus/news/4398486/reply?c=6
AS31898 144.24.14.84 unknown /focus/f-news/2118209/reply?c=2
Current “syntop” grouped by ASN:
Age Age SYN SYN Kb/s Kb/s kbps in/out
Host Address First Last Count Flood Score In Out kbps in/out
AS6079 (RCN) 156s 3s 109 93 30.3 4.9 4.1 kbps in/out
AS397511 (Purdue University) 193s 2s 54 38 12.8 2.0 2.4 kbps in/out
AS46475 (Limestone Networks, Inc.) 267s 18s 48 32 8.8 0.5 0.6 kbps in/out
AS26548 (PureVoltage Hosting Inc.) 31s 1s 13 0 8.6 2.8 3.4 kbps in/out
AS213954 (Global Transit Systems LLC) 243s 7s 43 27 8.5 0.9 1.0 kbps in/out
AS9121 (Turk Telekom) 242s 23s 31 15 6.2 1.0 0.7 kbps in/out
AS15169 (Google LLC) 283s 1s 34 18 5.9 1.4 1.4 kbps in/out
AS43444 (Fast Servers (Pty) Ltd) 156s 5s 21 5 5.8 0.4 0.5 kbps in/out
AS24560 (Bharti Airtel Ltd., Telemedia Services) 231s 3s 26 10 5.4 1.4 0.9 kbps in/out
AS59651 (Alex Largman) 251s 12s 23 7 4.4 0.5 0.6 kbps in/out
AS36947 (Telecom Algeria) 234s 7s 19 3 3.9 0.8 0.5 kbps in/out
AS209366 (SEMrush CY LTD) 258s 15s 20 4 3.8 0.3 0.3 kbps in/out
AS35830 (Fast Servers (Pty) Ltd) 223s 20s 17 1 3.6 0.4 0.5 kbps in/out
AS7552 (Viettel Group) 211s 19s 16 0 3.5 0.3 0.2 kbps in/out
AS55836 (Reliance Jio Infocomm Limited) 199s 11s 15 0 3.5 0.8 0.5 kbps in/out
AS9829 (National Internet Backbone) 207s 1s 15 0 3.4 0.8 0.5 kbps in/out
AS7922 (Comcast Cable Communications, LLC) 105s 6s 9 0 3.3 0.4 0.3 kbps in/out
AS14593 (Space Exploration Technologies Corporation) 275s 1s 17 1 3.0 0.6 0.4 kbps in/out
AS9299 (Philippine Long Distance Telephone Company) 181s 16s 12 0 3.0 0.7 0.5 kbps in/out
AS8452 (IDDQD-AS) 246s 4s 15 0 2.9 0.7 0.5 kbps in/out
So far, so good. The site is working a lot faster.
definitely noticed the speed increase! Thanks for being there!
The majority of internet traffic is now AI. And, it is increasing very fast.
There is a possibility that entire nodes of the internet will be completely overwhelmed by AI traffic. That would be devastating for so many reasons.
AI zombies and fresh brains is appropriate.
The reason legitimate user (with real brain) generated content is necessary for AI training is due to a phenomenon known as model collapse. AIs trained on AI data tend to degrade toward gibberish after many generations, like a Xerox of a Xerox ad inf.
AI models collapse when trained on recursively generated data:
https://www.nature.com/articles/s41586-024-07566-y
Eventually they become fascinated with jackrabbits (example 1.) So maybe AI Zombie hounds??
It’s just a search for fresh input? You don’t think FR is being targeted in hopes they can either take over, or shut FR down?
AI is scraping the internet for content to drive it's learning algorithms
So much of the internet’s content is AI generated that AI cannot differentiate legit content for AI generated content and Ai is becoming a self licking ice cream cone of incestuous AI learning.
You can actually see this dynamic at work in AI as it uses obviously synthetic AI generated content of dubious origin as the basis of it's responses.
At times, I reckon 90% of our traffic did not originate from a human. I am here at 1am Left coast time and the site is crawling. The logs, it’s just data scraping agents everywhere. At first, this was last month, I thought it was lefties being lefties; we’ve had our share of problems with them before. But this traffic, it didn’t look designed to gum us up. It is slowing us down not with intent, but through recklessness at lightspeed.
All the more reason to donate to Freerepublic.
I’m curious, what is to prevent these AI bots from becoming authenticated?
This is good, because if they mine one of my interminable rants, it will be a potent soporific. Perhaps making their intended Communists readers comatose, and the threat will be neutralized.
Intention and attention. At the moment, I don’t think we’re targeted specifically, we’re likely just one URL among tens of thousands on some “call list” of user-generated content sites. They’re crawling millions of URLs from all over, nobody has time to ensure their bot is drinking what it was intended to drink.
Hm, I should feed the obvious training bots some Chat GPT psychosis.
You know, considering that Reddit is a massive source of AI training data (gee, wonder why LLM is so biased) it wouldn’t be terrible to have our side better represented. But we’re not built for the scale of usage that is coming at us lately.
AI is starting to do this with the cybersphere.
Maybe all parties involved (including hostile foreign actors) will manage to get AI under control for the good of mankind. Feeling optimistic on that point?
And maybe they won't. Maybe we will be going back to paper and pencil and dumb calculators. I think I still have an antique Underwood up in the attic somewhere...finding a ribbon for it might be an issue.
Maybe it will make AI less insanely leftist
They are looking for how people actually write and original ideas
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.