Skip to comments.
AI’s Bandwidth Boom: How Artificial Intelligence Is Reshaping Internet Usage
GCG ^
| Aug 6, 2025
| Lara Mayes
Posted on 10/12/2025 12:33:37 PM PDT by Openurmind
Artificial Intelligence is no longer a futuristic buzzword — it’s an everyday business tool. From generative AI platforms like ChatGPT and Microsoft Copilot to machine learning models powering real-time analytics, customer service, and cybersecurity, AI has gone mainstream.
But while most of the conversation has focused on data privacy, automation, and ethics, there’s another massive shift underway: AI is transforming the way we use the internet.
As organizations across every industry embrace AI on a scale, they’re also encountering a new challenge: exponential growth in data transfer, processing requirements, and bandwidth demand. This is not a theoretical problem — it’s already happening. And IT leaders must start preparing now.
AI doesn’t just require computing power — it demands consistent, high-volume connectivity. Whether you’re using a generative model for content creation or deploying real-time fraud detection tools, AI workloads are data-intensive by nature.
TOPICS: Business/Economy; Computers/Internet
KEYWORDS: killitb4itstoolate
Navigation: use the links below to view more comments.
first previous 1-20, 21-40, 41-58 next last
To: Angelino97
>>Don’t most AIs spider only higher ranked sites, like Wikipedia?
I’ve had AI return information from my personal websites many times.
I have a log that shows who is touching which pages of my two sites (about 1600 pages in each). I watch spiders constantly walking my sites, though I’ve never back searched in the last few years when AI was more prevalent.
21
posted on
10/12/2025 2:21:27 PM PDT
by
mairdie
To: ProtectOurFreedom
Interesting. I have had Verizon since they took over LA Cellular, 20+ years ago. Definitely expensive, but never a complaint that they could not resolve. Usually blow right past the AI. If you call, and hit 0 (zero) enough times, you get a person. Only call at the beginning of the day, like 7am on the west Coast. Thanks !
To: Openurmind
There is dark irony in that communist bug eaters like Bill Gates are all in on profligate AI use of electricity. It's driving up rates, and polluting the air. I believe that in the State of Maryland, the NSA is the number one consumer of electricity.
23
posted on
10/12/2025 2:43:45 PM PDT
by
Governor Dinwiddie
( O give thanks unto the Lord, for He is gracious, and his mercy endures forever. — Psalm 106)
To: ProtectOurFreedom
We ordered from Amazon recently but only received half the order.
The AI customer service refused to let us cancel the half of the order which never arrived; it only canceled the entire order.
I wonder how much that will continue before either the vendors or Amazon itself notices and does something to fix it.
24
posted on
10/12/2025 3:19:32 PM PDT
by
grey_whiskers
(The opinions are solely those of the author and are subject to change without notice.)
To: Worldtraveler once upon a time
Will your server allow you to review your stats the last several months? Go check, I’ll bet your hit traffic has doubled or tripled in the last six months... It has happened to every domain owner I have discussed this issue with. But member traffic and participation has not increased.
25
posted on
10/12/2025 3:23:56 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Darth Gill
“I use cloudflare to protect my site from bots.”
Just so you are aware, CloudFlare is a censorship information control tool. Tested and proven...
26
posted on
10/12/2025 3:28:05 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Chickensoup
“Except for going to a few sites I have found that the internet is becoming more useless than not.
Searches are terrible.
The Internet as we know it is going away.”
Yep, me too. In fact the other day I was searching for some Conservative related information I KNEW was out there because I had been there and used it before. duck Duck Go actually gave me a “No Results” page. No Results? I have never seen a No Results page since Google was A start up and the internet was still pretty much empty! This is ABSOLUTELY IMPOSSIBLE without it being censored. And AI is giving them this censorship power.
Are we having fun yet?
27
posted on
10/12/2025 3:34:30 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Chickensoup
Except for going to a few sites I have found that the internet is becoming more useless than not.
yep, it is like asking the kid at the hardware store a question. You always get an answer but not one that is useful.
28
posted on
10/12/2025 3:37:29 PM PDT
by
PeterPrinciple
(Thinking Caps are no longer being issued, but there must be a warehouse full of them somewhere)
To: Openurmind
I use Grok a lot and am ramping up ChatGPT queries. What I always struggled to find in Google those systems find in seconds. They find the right sites to query and bring back the info in an easy to use format.
But, I see them regularly scraping info from 5-10 sites and maybe taking a look at another 5 to 10. It supports what you are saying that sites are getting hammered with AI requests.
The nice thing is I don’t need ad blockers, “Reader” pages, or “Hide distracting items” myself to cut out the BS crap.
To: fireman15
Oh I know, the hits to my site have increase twenty fold in the last two months. It is like getting hit with a DDoS attack constantly. So here is the situation we are in now. It doesn't look like hosts are going to help us filter it, so we are on our own to set up a defense. The problem is setting it up to allow SEO crawling while still blocking the AI agents. Or ditch the want for SEO altogether and block ALL Bots altogether. I have played with re-configuring robot.txt files. But it is only a "Request" not to crawl your domain. So the only way I found that can indeed block them is to specifically list the known agents in your .htaccess file. There are master lists that are being updated so you can update your own as new ones come out. Here is an example of a current master list: RewriteEngine On RewriteCond %{HTTP_USER_AGENT} (\.ai|-ai|_ai|ai\.|ai-|ai_|ai=|AddSearchBot|Agentic|AgentQL|Agent\ 3|Agent\ API|AI\ Agent|AI\ Article\ Writer|AI\ Chat|AI\ Content\ Detector|AI\ Detection|AI\ Dungeon|AI\ Journalist|AI\ Legion) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (AI\ RAG|AI\ Search|AI\ SEO\ Crawler|AI\ Training|AI\ Web|AI\ Writer|AI2|AIBot|aiHitBot|AIMatrix|AISearch|AITraining|Alexa|Alice\ Yandex|AliGenie|AliyunSec|Alpha\ AI|AlphaAI|Amazon|Amelia) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (AndersPinkBot|AndiBot|Anonymous\ AI|Anthropic|AnyPicker|Anyword|Applebot|Aria\ AI|Aria\ Browse|Articoolo|Ask\ AI|AutoGen|AutoGLM|Automated\ Writer|AutoML|Autonomous\ RAG|AwarioRssBot|AwarioSmartBot|AWS\ Trainium|Azure) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (BabyAGI|BabyCatAGI|BardBot|Basic\ RAG|Bedrock|Big\ Sur|Bigsur|Botsonic|Brightbot|Browser\ MCP\ Agent|Browser\ Use|Bytebot|ByteDance|Bytespider|CarynAI|CatBoost|CC-Crawler|CCBot|Chai|Character) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Charstar\ AI|Chatbot|ChatGLM|Chatsonic|ChatUser|Chinchilla|Claude|ClearScope|Clearview|Cognitive\ AI|Cohere|Common\ Crawl|CommonCrawl|Content\ Harmony|Content\ King|Content\ Optimizer|Content\ Samurai|ContentAtScale|ContentBot|Contentedge) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (ContentShake|Conversion\ AI|Copilot|CopyAI|Copymatic|Copyscape|CoreWeave|Corrective\ RAG|Cotoyogi|CRAB|Crawl4AI|CrawlQ\ AI|Crawlspace|Crew\ AI|CrewAI|Crushon\ AI|DALL-E|DarkBard|DataFor|DataProvider) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Datenbank\ Crawler|DeepAI|Deep\ AI|DeepL|DeepMind|Deep\ Research|DeepResearch|DeepSeek|Devin|Diffbot|Doubao\ AI|DuckAssistBot|DuckDuckGo\ Chat|DuckDuckGo-Enhanced|Echobot|Echobox|Elixir|FacebookBot|FacebookExternalHit|Factset) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Falcon|FIRE-1|Firebase|Firecrawl|Flux|Flyriver|Frase\ AI|FriendlyCrawler|Gato|Gemini|Gemma|Gen\ AI|GenAI|Generative|Genspark|Gentoo-chat|Ghostwriter|GigaChat|GLM|GodMode) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Goose|GPT|Grammarly|Grendizer|Grok|GT\ Bot|GTBot|GTP|Hemingway\ Editor|Hetzner|Hugging|Hunyuan|Hybrid\ Search\ RAG|Hypotenuse\ AI|iAsk|ICC-Crawler|ImageGen|ImagesiftBot|img2dataset|imgproxy) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (INK\ Editor|INKforall|Instructor|IntelliSeek|Inferkit|ISSCyberRiskCrawler|Janitor\ AI|Jasper|Jenni\ AI|Julius\ AI|Kafkai|Kaggle|Kangaroo|Keyword\ Density\ AI|Kimi|Knowledge|KomoBot|Kruti|LangChain|Le\ Chat) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Lensa|Lightpanda|LinerBot|LLaMA|LLM|Local\ RAG\ Agent|Lovable|Magistral|magpie-crawler|Manus|MarketMuse|Meltwater|Meta-AI|Meta-External|Meta-Webindexer|Meta\ AI|MetaAI|MetaTagBot|Middleware|Midjourney) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Mini\ AGI|MiniMax|Mintlify|Mistral|Mixtral|model-training|Monica|Narrative|NeevaBot|netEstate|Neural\ Text|NeuralSEO|NinjaAI|NodeZero|Nova\ Act|NovaAct|OAI-SearchBot|OAI\ SearchBot|OASIS|Olivia) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Omgili|Open\ AI|Open\ Interpreter|OpenAGI|OpenAI|OpenBot|OpenPi|OpenRouter|OpenText\ AI|Operator|Outwrite|Page\ Analyzer\ AI|PanguBot|Panscient|Paperlibot|Paraphraser\.io|peer39_crawler|Perflexity|Perplexity|Petal) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Phind|PiplBot|PoeBot|PoeSearchBot|ProWritingAid|Proximic|Puppeteer|Python\ AI|Qualified|Quark|QuillBot|Qopywriter|Qwen|RAG\ Agent|RAG\ Azure\ AI|RAG\ Chatbot|RAG\ Database|RAG\ IS|RAG\ Pipeline|RAG\ Search) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (RAG\ with|RAG-|RAG_|Raptor|React\ Agent|Redis\ AI\ RAG|RobotSpider|Rytr|SaplingAI|SBIntuitionsBot|Scala|Scalenut|Scrap|ScriptBook|Seekr|SEObot|SEO\ Content\ Machine|SEO\ Robot|SemrushBot|Sentibot) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Serper|ShapBot|Sidetrade|Simplified\ AI|Sitefinity|Skydancer|SlickWrite|SmartBot|Sonic|Sora|Spider/2|SpiderCreator|Spin\ Rewrite|Spinbot|Stability|StableDiffusionBot|Sudowrite|SummalyBot|Super\ Agent|Superagent) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (SuperAGI|Surfer\ AI|TerraCotta|Text\ Blaze|TextCortex|Thinkbot|Thordata|TikTokSpider|Timpibot|Tinybird|Together\ AI|Traefik|TurnitinBot|uAgents|VelenPublicWebCrawler|Venus\ Chub\ AI|Vidnami\ AI|Vision\ RAG|WebSurfer|WebText) [NC,OR] RewriteCond %{HTTP_USER_AGENT} (Webzio|WeChat|Whisper|WordAI|Wordtune|WPBot|Writecream|WriterZen|Writescope|Writesonic|xAI|xBot|YaML|YandexAdditional|YouBot|Zendesk|Zero|Zhipu|Zhuque\ AI|Zimm) [NC] RewriteRule (.*) - [F,L] This site is one of many who provide these master lists. https://perishablepress.com/ultimate-ai-block-list/
30
posted on
10/12/2025 3:46:39 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Openurmind
Oops, should have paragraphed that...
31
posted on
10/12/2025 3:47:53 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Openurmind
32
posted on
10/12/2025 3:53:41 PM PDT
by
Lazamataz
(I figure if Charlie Kirk can die for free speech, I can be mildly inconvenienced.)
To: mairdie
“I’ve never back searched in the last few years when AI was more prevalent.”
I have been keeping track of mine for the last six months and bots have increased twenty fold... It is becoming a huge problem.
33
posted on
10/12/2025 3:54:07 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Openurmind
Etaoin Shridlu opened the book to where his thumbnail had randomly parted the pages and began reading from the center of the page but noticed something strange about the pattern of the rivulets of space around the words and soon had lost all comprehension of the literal meaning of the passage when a picture clearly asserted itself in the apparently meticulously intricate typing which was simultaneously a long unpunctuated nonsensical run on sentence and yet a photograph perfectly halftoned into dots shaped as letters of the alphabet no doubt generated by a cybernetic system that creates fractal branched sentences which never end but instead sort of parenthesize and go for a ways and then change course usually by going into detail in a tangential sort of stackpush that never pops but instead keeps zooming in yet another level each line or so like an infinite inwards motion that doesn’t even need punctuation and takes the reader on a one-way trip forwards into progressively refined images much like the incredible shrinking man or Fantastic Voyage except that it just keeps on going so actually the effect is more like the colored oilslick breakdowns in 2001 and if you could get the wave action of the text into just the right peristaltic contractions in sync with the eye and mind of the audience it could get pretty awesome like nanotech cities living on the bark of redwood trees in undulating valleys of forests on nonspherical fiberspace membranes extruded by femtopicotek nested-space 3.2-dimensional manifolds “bigger at the small end of the scale” provided you have senses which don’t peter out when things get subtle thus allowing you to encode information in the very borderlines of its very existence thus relying on the viewer’s idiosyncrasies more than on his common-ness because things which are on the edge of not being there at all are going to probably excite a set of simultaneous responses which don’t triangulate or intersect or agree or synergize in an unambiguated crisp singleness and are also somewhat partial and even flickering or dynamic or changing rapidly with time which requires their mind(s) to choose from a
bifurcating multiplexing divergence of (mildly) resonant rapidly-evolving attractors forming a ghostly community of evaporatory cooperative “temporary friends” sharing fleeting memories because a localized continuity of thought is maintained on more than one level (continued on next page)
34
posted on
10/12/2025 3:57:04 PM PDT
by
Lazamataz
(I figure if Charlie Kirk can die for free speech, I can be mildly inconvenienced.)
To: Lazamataz
Yes, I know... My screw up I should have reformatted that code before posting it. I should have at least used the paragraph separations.
Anyone want that again formatted better?
35
posted on
10/12/2025 3:57:06 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Lazamataz
Will this let me wrap code?
36
posted on
10/12/2025 4:25:11 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Openurmind
Thanks for the hint. As bad as this seems right now, it reminds me of a song.

https://youtu.be/4cia_v4vxfE
I have been noticing a bunch of weird messages from Hostinger lately. I didn't bother investigating because all of my sites are strictly for learning and my own entertainment. I did not realize what was causing this, but I think you have found the answer!
To: fireman15; Lazamataz
38
posted on
10/12/2025 4:31:27 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: fireman15
I was actually just listening to BTO yesterday, and Chicago. Remembering back when the world was still sane.
I agree, We haven’t seen nothing yet. I’m not sure most folks truly understand where this is going to end up. It is fun for now, but we are going to absolutely pay dearly for it later. If the FR has the normal rates and plan their host provides they are already paying dearly for the extra bandwidth AI bots are eating up. It just isn’t right at all...
39
posted on
10/12/2025 4:38:18 PM PDT
by
Openurmind
(AI - An Illusion for Aptitude Intrusion to Alter Intellect. )
To: Openurmind
My guess is that it won’t be long before more sophisticate tools are available to the administrators of sites that do not want to be used by AI models to help prevent excessive bandwidth use. I have been using hosting service for many years and am happy that the latest hosting plan that I purchased has unlimited bandwidth. Since I purchased 4 years in advance several months ago all of their plans have been revised.
With this type of situation, it is likely in the hosting service’s best interest to come up with a filtering solution.
Navigation: use the links below to view more comments.
first previous 1-20, 21-40, 41-58 next last
Disclaimer:
Opinions posted on Free Republic are those of the individual
posters and do not necessarily represent the opinion of Free Republic or its
management. All materials posted herein are protected by copyright law and the
exemption for fair use of copyrighted works.
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson