New Numbers for the Curious (or FR Traffic Stats)

Free Republic Website Usage ^

[John Robinson]

For those that love numbers, I have three links to Free Republic usage statistics:

• Free Republic Website Usage NEW!
  compiled from server logs; ready by 2am PT daily

• Free Republic Daily Stats
  compiled from posting logs; ready by 2am PT daily

• Traffic Rank [Alexa]
  see their website

Caveat:

1. I recently upgraded the program (AWStats) used to generate the Website Usage pages. The old pages, though built with the new program, utilized the old format database which did not provide some fields. Those fields will be 0 or not present on the older reports.

2. For some reason the reports switched to GMT some months ago, I switched back to PT this month. Each switch fouled the "Hours" graph for the month of the switch.

3. We lost our senses Mar 12-14, or rather, the stat program forgot how to tell the difference between a Hit (any request to the web server) and a Page (a request for a HTML page, or similar page of content); all hits were erroneously counted as pages, the actual number of pages for that time period is actually quite a bit less.

4. And, on Wednesday Mar 12th, I enabled HTTP compression, practically halving our bandwidth usage! Just in time for the overwhelming load that day!

5. The Daily Stats page is clearly showing signs of age. It no longer collects the "Front-Side" and "Back-Side" statistics. However, you will find "Front-Side" had become a duplicate effort, see the Website Usage pages to continue on.

6. Numbers on the Daily Stats page and the new Website Usage pages are generally an even comparison. However, consider the Daily Stats pages are generated on daily GMT time boundaries, and the Website Usage pages are generated at PT (or GMT--oops) boundaries.

And now the ramble.

One thing I don't yet have for readers is a report of the actual Mbps that flows through our Internet connection. Leading up to the war we've routinely saturated our link, that's 10 Mbps, or 10 million 1's and 0's per second. Those numbers are very interesting as well!

As we reach our bandwidth limit, the network link becomes saturated and we begin to slow down. Last Wed (Mar 12) was the breaking point, the server was falling over as network congestion backed up through the kernel and into my application-- what a mess that will make!

Foreseeing impending trouble days before, I had previously installed a little gadget called mod_gzip. The mod_gzip is an Apache webserver extension that compresses HTML pages on the fly. After an abbreviated test run of just a few days I enabled mod_gzip to bail me out of this crisis.

My savior! The compression is highly effective, reducing pages by an average of 79% the original page size. However, the true savings are closer to 50%. Whether the module is used at all is dependent upon the browser used to view the page. Due to brain damage, some browsers do not work properly or at all when viewing a compressed page, and therefore must not be sent compressed pages. And images are not compressed at all-- mostly because images are already highly compressed, but partly because of even more brain damage in even more browsers.

50% isn't bad, not bad at all. We've now effectively doubled our bandwidth capacity. Have we tamed the traffic? Mostly. Bandwidth usage is now routinely in the 5-6 Mbps range, which is effectively 10-12 Mbps. Thursday morning from 7am-1pm PT we were in the 7-8 Mbps range (14-16 Mbps!) and often times pushing the ceiling, pushing our full link capacity of 10 Mbits hard for extended periods of time. That's 20 Mbits of bandwidth, 12 T-1 lines, or nearly one half T-3. Simply amazing!

[Bahbah]

I don't understand any of that, but I know I am glad you do.

[John Robinson]

I'll proofread that later. I'm going to bed.

[LowOiL]

on Wednesday Mar 12th, I enabled HTTP compression, practically halving our bandwidth usage! Just in time for the overwhelming load that day!

Wow, and we did not have to download a WinZip file to do this?

[grania]

There used to be information about how many people were logged in to FR at any given time. It was kind of a neat thing to know.

[FreedomPoster]

Thanks for all you do. Love seeing the techie minutia every now and then.

Doesn't a T-3 get cheaper than all the separate T-1's at some point, less than a T-3's worth? I know it does for voice, but I guess it might not make as big of a difference for an Internet connection.

[Admin Moderator]

From the posting log stats:

Distinct Posters on Thursday: 4,358...a new record topping the previous high of 4,313 from the day before.

Total articles + replies on Thursday: 35,055...a new record topping the previous high of 31,153 set on Tuesday.

Articles posted on Thursday: 1,222...not a new record, second most ever to the 1,301 posted the day after the 2000 elections.

Replies posted on Thursday: 33,833...a new record, topping the previous high of 30,102 on Tuesday.

New signups on Thursday: 350...the fourth highest total behind 3 days in the timeframe just after the 2000 elections.

Busy busy busy

[Free the USA]

fyi

[tictoc]

On-the-fly compression works only if it also is enabled in the "client browser" (the browser software that people use to log on to FR).

I believe that in Internet Explorer 6, this feature is already turned on by default.

Here is how to check:

Go to Start+Settings+Control Panel, then double click the Icon that says "Internet Options." Now click the tab that says "Advanced" scroll down until you see "Use HTTP 1.1"; make sure there is a check in the box and press ok. Press ok again to close Internet Options, and close any open IE windows. When you launch IE again you should see that surfing is much faster than it was before.

In addition, if you are using a proxy like Proxomitron (Proxo), WebWasher, etc., you should also turn on "Use HTTP 1.1 through proxy connections" by putting a checkmark into the box next to it.

[Quix]

FANTASMAGORICALLY WONDERFUL!

GREAT WORK JOHN. THANKS TONS!

GOD'S BEST TO THE PRES AND THE TROOPS AND YOU, THE FAMILY, AND OF COURSE, FR!

[Peach]

A fantastic job! What a resource. Hard to believe there will ever be a Democrat elected to any high office with the info Freepers are able to gather - heh heh.

[nicmarlo]

Free Republic Website Usage NEW!

You are not authorized to view this page

So, what did I do wrong JR?? : )

[Alas Babylon!]

How's your cpu load? Usually enabling heavy compression will further tax a cpu. Anything over a consistant 75% usage indicates need for a faster cpu or additional cpu's. Then again, for all I know, you're already using SMPs.

[brityank]

You are not authorized to view this page

Ditto -- with Netscape 4.77 and IE 5.5.

Help !!

[backhoe]

"You are not authorized to view this page"

Yep, I have the same problem, too.

[Xthe17th]

Our REACH is now up to 640 (per million internet users). WOOHOO and BUMP!!! Reach is the key stat, IMHO. Yahoo's Reach is something like 300,000 - so there's our goal. Kudos for the great work Robinsons.

[Admin Moderator]

You are not authorized to view this page

Fnord.

[error99]

Thanks for everything, John !
Hope you are getting enough sleep during this !

[Congressman Billybob]

Thank you very much for translating just a few of the high points in JimRob's analysis, into actual English that us tech-dumb Freepers can understand. I do understand the benefits to FR of the compression, to stay within the bandwidth cap.

Especially good is the number of new sign-ups. FR has been mentioned in a number of lamestream media articles of late. Though most of those have been slams, not praises, both generate traffic that takes a look-see at FR. And some of those stop and stay. One more example of Mayor Curley's comment, "Say what you want about me, just make sure you spell my name correctly."

Congratulations, JimRob, and all.

Congressman Billybob

Latest column, not yet up on UPI, and FR, "Don't Trust My Words"

Latest book(let), "to Restore Trust in America."

[eureka!]

John, kudos and thanks for the job you do here. I have the stats link in my profile and check it out every so often. I noticed that signups have tripled recently. Hopefully, all the new Freepers will open their wallets come the next FReepathon fundraiser. *ducking for saying the dreaded words*