What is wrong with Free Republic?

Suggestions to make your FR pages load faster ^ | 10/29/12 | self

[boop]

Why is Free Republic so slow to load, and when it does, it throws me off and I get the "Cannot connect" message from Windows. And it's NOT just during debates. It's happening all the time now. Is it just MY computer? I know HuffPo is down because of the storm, but FR is out of California. In the 12 years I've been here I've never had so much trouble with this site. Until about a month ago.

[Mama_Bear]

Thanks for pinging me to this discussion, Jim. Good luck with that new prototype program. I think it looks great, and promising! I know you have been working hard on it and you are anxious to get it up and running. We are pulling for you. As far as the next couple weeks go, hey, it will be what it will be. This too shall pass. :-)

[newheart]

See tomkat’s post #93 for an additional clue.

[Lazamataz]

I think that’s the same conclusion we’re coming to. Very likely a database/software/hardware design vs actual usage issue. And there is no quick cure. A complete redesign and rewrite using all newer technologies in hardware and software ought to do the trick. Thankfully, I’ve been working on a new system using AJAX based techniques for a couple years (in my spare time) and even have a bare bones prototype working, but it’s months away from even being ready for beta testing. And John is beginning to think Cloud. Who knows? We may be rolling out something brand spanking new in the months ahead. But it’s going to be hell around here for the next week or so. Praying we make it.

Hey Jim! A few thoughts:

• As you already well know, AJAX is a technique and the language you choose implements that. I might be wrong but I think you mentioned Javascript at the Shoot. Pros and cons to Javascript, if that's the chosen language. I've seen some very highend systems written entirely in javascript but obfuscation is an issue. You have to issue the source code to the browser (though you can obfuscate it somewhat by stripping meaningful variable names and method names) but still, you have to issue the javascript for the browser to be able to work with it. Javascript also lends to being unstructured, it isn't a Strongly-Typed language (you can change an int variable to a string variable on the fly) and does not lend itself well to Object Oriented Programming.
• I've looked a lot at the various ways to host what I have in mind for the system... Linux running Mono (which is a Common Language Runtime interpreter of the Bytecodes that .NET emits) is a very good solution. But I spoke last night to a fellow named David C., he was my boss at a former company, who then went on to found a very successful Enterprise Solution company. He actually convinced me that Cloud was the way to ensure -- and remove from the variable list -- hardware scalability issues.
• I'm a big proponent of dot Net. It's a VERY easy to maintain and VERY easy to debug language, when you write OOP code. However, I just might open my mind to other languages.
• I've collected a series of desired features from a variety of users. I can share those with you.
• Should my efforts be fruitful, I will gladly release source code to you guys, to modify and use as you will. Consider it an in-kind donation. I would like to be thought of as one of the maintenance crew, in that case. Of course, all standards of professionalism would apply.

[Lazamataz]

However, a NoSQL database

Yes, my former boss David C. mentioned this as well. There are pros and cons to this approach as well. One major one, to my mind, is there is no ACID for NoSQL (atomicity, consistency, isolation, durability). There's also little standardization in API's or Query Languages for NoSQL. Therefore, I conclude that it is an immature -- albiet promising -- technology.

SQL doesn't necessarily scale, but there are methods of writing code so that it scales a lot better:

1. Connecting when request required, dropping connection immediately. Seems counterintuitive, but it works.
2. No Dynamic SQL *EVER*!!!!

   On the SQL side itself:

3. Watching carefully the number of indexes. Inserts slow down a lot for indexes.
4. Watching cartesians!!!
5. Good PK selection, low use of FK.
6. Use of Profiler to see where most of your time is spent.

[Lazamataz]

the IT staff hollers “Yeehaw!”

I like to sneak up quietly behind coworkers, and yell that, at the top of my lungs.

Keeps people on their toes, and I've only occasionally been shot.

[onyx]

Hi Mama_Bear! Jim's new prototype program is amazing and he's done it in his spare time !

FR worked super last night and is working superbly right now. I'm no techie, but these swings come and go. We'll keep smiling, praying and rolling with the punches.

[RitaOK]

I will try that again. Thx. I have done that and found it did not post lost my post entirely. But, I am on my cell phone working in miniature and not all that proficient.

[Mama_Bear]

Hi, onyx. I made a valiant attempt of acting like I knew what he was talking about when he explained it to me, but I am pretty sure he knew I was clueless. Was surprised to learn there were several more months of work before beta testing could even begin. It looked ready to go live to me, but, what do I know? LOL.

I echo your sentiments re: smiling, praying and rolling with the punches.

[houeto]

But, I am on my cell phone working in miniature and not all that proficient.

Cell phone? (sigh) I feel for you. I wouldn't be able to do diddley squat from a cell phone.

[RitaOK]

LOL! Exactly! You can see why my nerves would be in a state of “prickly” in the best of circumstances, and with FR tech problems becoming so severe I can get plenty cranky. :D).

Everyone seems to take me in stride or run for their lives when they hear me coming. LOL!

You made me laugh. Thank you!

[onyx]

LOL. Well, you’re not alone in being clueless, but I’m sure you’re a lot more savvy than me! It looks perfect and beautiful to me, too! I got a chance in Texas to talk to him about it and of course, being a techie dunce, I didn’t understand much more, but it’s very easy to navigate.

FR is still moving smooth and it’s almost Noon here! I’m happy, happy.
Prayers, love and hugs, (((Mama-Bear)))

[Mama_Bear]

...but it’s very easy to navigate.

From what little I saw, it looks pretty simple to me. :-)

Sending you a PM in a few.

Love and hugs backatcha!

[HamiltonJay]

As a professional in the web arena since 1994 I am not someone who likes to jump into these types of discussions because as someone outside looking in I do not have enough information available to offer true insight.

HOWEVER, blaiming user hardware is not correct, the sites performace has degraded noticeably, and the fact you do not have a few people complaining lets you know its not just one persons hardware.

I personally can attest that the performace of FR has been slow from at leas 3 different networks that I am part of and on 3 different pieces of physical hardware.. whatever is going on its not something caused by users hardware.. it is either performance at the site itself or something in the network.

If it is network it certainly seenms to be something network related to FR or something very close to FR as I have not experienced any latency on any other locations around the globe I deal with.

I have confidence whatever the issue is, folks are working on it. I won’t try to second guess the folks who are far more intimately familiar with their configurations than I am.

If there are others like yourself using the site fine without noticing issues, it could be a networking/routing issue outside of FR itself that could be affecting some but not others... However this isn’t something where its user hardware caused for sure.

[Henchster]

Thanks Jim. We definitely appreciate all the work you and John have done over the years, you guys are Patriots in the truest sense of the word.

Just for info, from the time I saw “New Posts to You” - to click on your reply, to get to the post page, 3 minutes - we’ll see how long it takes to post - current time 10:53pst.

[tomkat]

In the last 20 minutes FR has turned to total molasses on it's way to central PA   :^(

[tomkat]

             

[justlurking]

One major one, to my mind, is there is no ACID for NoSQL (atomicity, consistency, isolation, durability).

FR doesn't need ACID. It's a news aggregation site, with user comments. If it loses a comment, or posts a couple of simultaneously posted ones out of order, it's no big deal.

There's also little standardization in API's or Query Languages for NoSQL.

Once you settle on a database and API, you stick with it. There's no need to make it portable to other databases.

[rabidralph]

TPNN is stealing our bandwidth!

[justlurking]

I don’t know where you got your information, justlurking, but all offers of professional help have not been spurned, publicly or otherwise.

I got my information from the last thread about this issue. Several people, including you, offered assistance. Jim replied: "Thanks, but no thanks." However, I hope they privately accepted assistance from someone who knows what they are doing.

Jim & John aren’t being difficult, prideful, or stubborn. They’re doing what they can with the resources they have in a tough situation.

And that's why no progress is being made.

This is what I do for a living. I collect information from a client's systems, analyze it, and show them where it is failing or will fail as load increases. No changes are required: almost all of the data is already available, and you just need someone that knows how to make sense of it. On the rare occasions that I need more data, the customer uses tools they already have and I help them to configure them to collect the additional data.

I've done this for long enough and at enough different clients that I can build a working hypothesis from limited information. My next step would be to construct a data collection and analysis strategy to either prove or disprove the hypothesis, and then iterate/adjust until the customer gets the answer they asked for.

I don't make changes to the customer's system(s). They do it, with my direction/assistance if they need it. But, any wholesale implementation change is beyond the scope of what I do. I only identify the problem they need to address, and it is typically a software design issue, not a hardware capacity problem. Sometimes we get lucky and it's just a small software configuration change, but it's rare.

I've been fingerprinted, background-checked, and drug-tested so many times that I've lost count. I just finished analysis of a system that half of the people reading this message probably used in the past year, albeit unknowingly. If my clients can trust an outsider to help them, any "security issues" at FR are easily addressed by checking references.

But, I'm not volunteering. I can't. Even if I did it for free, my employer would consider it a conflict of interest. And my typical hourly rate would consume FR's entire hardware upgrade budget in a few days.

[justlurking]

I'm not saying hackers but there are several of us that can tell you for a fact, the instant ConEd blew up and all of Lower Manhatten went down last night, every function on FR started working perfectly. That lasted the rest of the night.

This is typical behavior for a load-related problem. Everything works fine until you hit the tipping point, and then the system starts thrashing so badly that everyone gets poor service. And as soon as load drops below the threshold, everything goes back to normal.

The problem is most likely being aggravated by a DDoS (distributed denial of service) attack -- by FReepers. No, I don't mean we are doing it intentionally: when an error occurs, FReepers have learned to just hit reload again, or repost again -- because that "usually" works. If every person does that 2 or 3 times, it doubles or triples the load on the servers. Until enough people get frustrated and go elsewhere for news, go watch TV, or lose power and can't use their computer at all, the problem persists. Then, everything goes back to normal.