[Jim Robinson]

No excuses, no denials. The system is failing under peak loads. Don’t know why yet. I thought we might be able to solve it by beefing up the servers and the RAM, and raised $6,000 to do so, but guess John does not agree. He’s wrestling with some systems configuration problems at the moment and has not yet found the solution. Unfortunately, I cannot help him with these issues as they’re way over my abilities. I’ve asked him if bringing in a consultant will help, but he appears reluctant to do so. I’ve also asked him to come on to a thread to post the status and answer questions, but so far he hasn’t wanted to do that either. Hopefully he will eventually find the problem. Sorry about the slowdowns during the debates and praying we get it fixed soon. If we end up not needing the $6,000, I will offer to return it to the donors. But I’m still thinking we need to beef up the hardware.

[Randy Larsen]

Jim, That $6000 will be needed eventually no matter what. It’s hard enough to collect it when planning for the future, imagine trying to collect it in an emergency.

My vote would be to save it for later if not needed right away.

[tomkat]

Jim,

I have nothing but respect for you & yours and the wonderful work of love and dedication and patriotism that is FreeRepublic.

Please do not misconstrue any of my occasional gripes re system sluggishness as commentary other than technical.

The integrity and honesty of you and all those behind the scenes of this place have never been in doubt in what little is left of my mind.

(well, there may have been a couple mods i wanted to slap over the years but . .  ;-)

Seriesly tho, my very best hopes to you and John and whoever else is/may be working on getting us back up to speed, literally.

FRegards,

[Chickensoup]

Jim use the money. Get doo dads for the board.

Why was my post deleted tonight? Was I bad?

Soup

[Yardstick]

Roger that, Jim. We know you guys are trying to get this worked out. John’s forum software is the best bar none. Given what he’s accomplished there’s no loss of face in letting a consultant come in to do some housekeeping if that’s what’s needed. I have no idea what the right solution is. Something definitely needs to be done though. We’re all pulling for you.

[Col Freeper]

No excuses, no denials.

Jim, in all the years I've been at FR, I've never known you to give excuses, nor denials. I'm not expecting to hear them from you now.

I suspect that some FReepers don't fully understand that this is your baby (and now John's as well), and just how frustrating it is to not see them quickly resolved.

I spent many years at remote locations, maintaining national defense radars, and I am well aware of the frustration of having an intermittent and/or non-repeatable failure drive one to the point of hair pulling.

On one of those systems, I arrived at day shift to find the maintenance crew had removed and replaced every Line Replaceable Unit (LRU) with one or more of the spares.

They had depleted all of the spares stock based on the Main Computer's software "instructing" them what LRU to replace.

They had followed the automated troubleshooting software's instructions throughout the night, and just before I arrived, it had gone full circle and was now instructing them to replace the first LRU it had called out at the beginning of their shift. Spares that had been called out, and subsequently identified by the computer as failed after retest, were to be found all over the maintenance area benches and shelves.

It was a nightmare, Headquarters was readying an emergency response team and transport to fly them in to "assist" us. The Site Commander was in a bad situation because, if they did have to be sent in, he was probably going to lose his Command position, and his upward mobility in rank would be forever lost.

As the Technical Services "expert", I was on the hot seat as well to find and fix the problem asap. This National Missile Early Warning System had been down for over 8 hours, and it's expected outage time was expected to be not more than 15 minutes (hey, it was an automated, multiple-stage computer-controlled, self-diagnosing and failure-unit-identifying system after all).

It took a couple of hours more to find the problem, and at first the maintenance crew I was technically guiding didn't believe the failure I led them to. After fixing what I showed them, the system was restored and operational within a few minutes. The Site Commander also wanted to be personally shown what the problem was, and some help in verbalizing the failure and fix to the General and his staff.

We just managed to restore the system in time to cancel the visit from the assistance crew (bet their families liked that because they didn't have to fly across country to help us).

As a Technical guy, it was a high stress, frustrating, and yet very satisfying experience.

So while some here are convinced that I am just a "cheerleader for FR", based on my experiences in finding and fixing hardware and/or software problems, you have, and continue to have my support.

FR is unique. For those who don't believe it, probably any other site/forum will do.

Oh yea, what was the problem I found? Two wire-wrapped power distribution standoff posts in the Radar Receiver Cabinets had gotten bent so that they were just close enough together to intermittently "arc-over", spiking and/or temporarily dropping the voltage to the LRUs below acceptable levels. This confused the computer software's automatic single-point failure diagnostics into reporting all of those (actually functional) LRUs as failures.

Keep on trucking Jim, and tell John to do the same. Hopefully you will both know if/when outside experts are required, and they will be summoned (and be of use).

[paul544]

That $6k can buy a lot of on-demand cloud computing. Any number of services will allow you to spin up extra capacity for peak times then shut it down.

[Revel]

It sounds like you are in between a rock and a hard place. As the old saying goes. Not wanting to insult John, who has over time, proven that he is a smart fellow. But as to John not wanting help at this point. Maybe Remind John that Pride goeth before the fall. Sometimes no matter how smart we are then we can’t see something that is right in front of our face. A Fresh look from someone else might discover something that was missed.

[katnip]

Jim, I come from old mainframe COBOL DMSII programming environment.

When we knew our hardware, disk space and database settings were adequate and still had a problem we looked to the software.

Even on good days here on FR the response time is slower than it used to be.

I am wondering if there is a programming glitch that causes each transaction or maybe just one to somehow go into an unnecessary loop (or whatever they call it nowadays) before returning the response. This would eat up your memory and cause all other transactions to wait for this one to end.

This may not be noticeable on a slow day but of course be exacerbated with heavy volume.

Just a thought from a mainframe dinosaur.

[katnip]

One other thought. While I know John takes great pride in his work, most times a pair of “fresh eyes” looking at your code can spot a bug that you’ve overlooked time after time.