New Numbers for the Curious (or FR Traffic Stats)

Free Republic Website Usage ^

[John Robinson]

For those that love numbers, I have three links to Free Republic usage statistics:

• Free Republic Website Usage NEW!
  compiled from server logs; ready by 2am PT daily

• Free Republic Daily Stats
  compiled from posting logs; ready by 2am PT daily

• Traffic Rank [Alexa]
  see their website

Caveat:

1. I recently upgraded the program (AWStats) used to generate the Website Usage pages. The old pages, though built with the new program, utilized the old format database which did not provide some fields. Those fields will be 0 or not present on the older reports.

2. For some reason the reports switched to GMT some months ago, I switched back to PT this month. Each switch fouled the "Hours" graph for the month of the switch.

3. We lost our senses Mar 12-14, or rather, the stat program forgot how to tell the difference between a Hit (any request to the web server) and a Page (a request for a HTML page, or similar page of content); all hits were erroneously counted as pages, the actual number of pages for that time period is actually quite a bit less.

4. And, on Wednesday Mar 12th, I enabled HTTP compression, practically halving our bandwidth usage! Just in time for the overwhelming load that day!

5. The Daily Stats page is clearly showing signs of age. It no longer collects the "Front-Side" and "Back-Side" statistics. However, you will find "Front-Side" had become a duplicate effort, see the Website Usage pages to continue on.

6. Numbers on the Daily Stats page and the new Website Usage pages are generally an even comparison. However, consider the Daily Stats pages are generated on daily GMT time boundaries, and the Website Usage pages are generated at PT (or GMT--oops) boundaries.

And now the ramble.

One thing I don't yet have for readers is a report of the actual Mbps that flows through our Internet connection. Leading up to the war we've routinely saturated our link, that's 10 Mbps, or 10 million 1's and 0's per second. Those numbers are very interesting as well!

As we reach our bandwidth limit, the network link becomes saturated and we begin to slow down. Last Wed (Mar 12) was the breaking point, the server was falling over as network congestion backed up through the kernel and into my application-- what a mess that will make!

Foreseeing impending trouble days before, I had previously installed a little gadget called mod_gzip. The mod_gzip is an Apache webserver extension that compresses HTML pages on the fly. After an abbreviated test run of just a few days I enabled mod_gzip to bail me out of this crisis.

My savior! The compression is highly effective, reducing pages by an average of 79% the original page size. However, the true savings are closer to 50%. Whether the module is used at all is dependent upon the browser used to view the page. Due to brain damage, some browsers do not work properly or at all when viewing a compressed page, and therefore must not be sent compressed pages. And images are not compressed at all-- mostly because images are already highly compressed, but partly because of even more brain damage in even more browsers.

50% isn't bad, not bad at all. We've now effectively doubled our bandwidth capacity. Have we tamed the traffic? Mostly. Bandwidth usage is now routinely in the 5-6 Mbps range, which is effectively 10-12 Mbps. Thursday morning from 7am-1pm PT we were in the 7-8 Mbps range (14-16 Mbps!) and often times pushing the ceiling, pushing our full link capacity of 10 Mbits hard for extended periods of time. That's 20 Mbits of bandwidth, 12 T-1 lines, or nearly one half T-3. Simply amazing!

[backhoe]

Say, that's gorgeous! Thanks!

[Billthedrill]

John, congratulations on running the state-of-the-art world news distribution process, period. This is history in the making. Hang in there!

[Bob J]

John, is that 1 million uniques for Feb for the whole month or unique visitors counted each day?

[John Robinson]

It is the unique visitors for the entire month, February had more than one million.

I should warn you, "visitors" by definition of web statistics programs is the number of distinct Internet addresses hitting the website. There are a large number of reasons why such numbers do not directly translate to actual people, it is possible for one person to use many Internet addresses, and it is possible for one Internet address to service many people. See: Lies, Damned Lies, and Unique Visitors. It is a useful number when compared against other websites, though.

[gcruse]

Got it.
In the case of Netscape 7, go to Edit Preferences, Advanced, HTTP networking. My HTTP 1.1 was selected by default.

[gitmo]

the server was falling over as network congestion backed up through the kernel and into my application-- what a mess that will make!

That's why you should always keep the bit bucket handy.

[Ernest_at_the_Beach]

We need more traffic! More tuning! MORE TUNING!

It's coming!

Great job John!

OFFICIAL BUMP(TOPIC)LIST

[CheneyChick]

Cool nerd talk!

Cheers, CC :)

[ChefKeith]

You forgot the "secret handshake"

[my_pointy_head_is_sharp]

bump

[untenured]

	Operating Systems	Hits	Percent
	Atari	1	0 %

Somebody wanna fess up?

[RightWhale]

information about how many people were logged in to FR

It was useful. If it got down to 100 or so, it was a signal to go do some of the stuff you should have been doing instead of staying logged in past your bedtime.

[Sir Gawain]

Broke? :-(