Posted on 01/29/2008 6:59:35 AM PST by N3WBI3
According to a press release issued earlier this month by Finjan, a security research firm, compromised Web servers are infecting thousands of visitors daily with malware that turns their Windows machines into unwitting bots to do the bidding of an as yet unidentified criminal organization. Security firms ScanSafe and SecureWorks have since added their own takes on the situation, though with varying estimates on the number of sites affected. All reports thus far say the compromised servers are running Linux and Apache.
According to an article on ServerTune.com, the exploit involves a rootkit installed on the compromised server that replaces several system binaries with infected versions. When the system is booted, the infected binaries are executed, and as a result, dynamically created JavaScript payloads are randomly and intermittently served to site visitors. The malware JavaScript attempts to exploit vulnerabilites in Windows, QuickTime, and Yahoo! Messenger on the visitor's machine in order to infect them.
We asked the Apache Software Foundation if it had any advice on how to detect the rootkit or cleanse a server when it's found. According to Mark Cox of the Apache security team, "Whilst details are thin as to how the attackers gained root access to the compromised servers, we currently have no evidence that this is due to an unfixed vulnerability in the Apache HTTP Server."
We sent a similar query to Red Hat, the largest vendor of Linux, but all its security team could tell us was that "At this point in time we have not had access to any affected machines and therefore cannot give guidance on which tools would reliably detect the rootkit."
cPanel, a popular administration tool used by hosting companies that allows clients to manage their hosted sites, has posted a security note describing what the rootkit does after it's installed, and suggests two ways to check a server for the rootkit.
According to cPanel, if you are unable to create a directory name beginning with a numeral -- as in mkdir 1 -- you're infected. Another test is to monitor the packets from the server with the following tcpdump command:
tcpdump -nAs 2048 src port 80 | grep "[a-zA-Z]\{5\}\.js'"
One great unknown thus far is how the servers come to be infected. Absent any forensic evidence of break-ins, the current thinking is that the malware authors gained access to the servers using stolen root passwords. The earliest known victims, according to quotes by researchers in this ComputerWorld story, were sites run by large hosting companies, which could give attackers root access to hundreds or even thousands of Web sites when compromised.
Other than using and safeguarding secure root passwords, not much can be done at this time to be proactive in preventing servers from being compromised, so searching techniques similar to the tcpdump command above, which check to see if a server has already been compromised, is probably the best course of action available to administrators. We haven't found a good answer yet for disinfecting compromised servers, but a complete reinstall of Linux, Apache, and a new root password would certainly do the trick.
1) If you are running any Linux server you have little to no excuse for not arming it with tripwire the minute it is set up. Tripwire is free and *very* effective at detecting root kits
2) If you are setting up any linux server you have little to no excuse for not enabling selinux
3) If you're system has been the root kit (any os type) you need to wipe it down to and including the mbr and rebuild to the last time you know that the system was not compromised.
--
As to this specific problem it would be nice if someone with an infected machine would just do an md5deep on '/' and send the results the the distro provider a comparison to a similar md5deep from a control box should light up what the root kit infects..
OSS Head up all you web admins... The details are far from solid enough to lose sleep over but some tlc on your servers might be nice..
I wonder who is behind this and what, exactly, is the malware bidding the infected host to do?
Also, this section appears to be written from a windows-centric point of view:
According to an article on ServerTune.com, the exploit involves a rootkit installed on the compromised server that replaces several system binaries with infected versions. When the system is booted, the infected binaries are executed, and as a result, dynamically created JavaScript payloads are randomly and intermittently served to site visitors
On most unix systems, systems are rebooted very infrequently, and don't need to be rebooted in order for the vast majority of binaries (sans kernel) from being updated and executed. Makes me doubt the veracity of those making these claims. The binaries that would most likely need to be replaced to have the http process be doing wanky things are not among those that can't be just replaced and restarted.
Heck, on most current versions of unix you can rename a file while it's being written to and the writing process will be notified of the change on the fly.
Why would you doubt this? Apparently it has taken place. Now, whether the problem is being accurately reported is another question ... but the point is that something has apparently been written to exploit a Linux system. It should be a major wake-up call to those who tout its supposedly un-hackable properties.
The binaries that would most likely need to be replaced to have the http process be doing wanky things are not among those that can't be just replaced and restarted.
Weeellllll, maybe. Then again, a sufficiently adept and dedicated programmer with access to the open source code could probably figure out a clever way to approach the problem, precisely to avoid or disguise the things you've pointed out.
Still needs more details. Taking the usual precautions are always warranted, but panic without information is not.
Could be a system binary so a service restart after a patch would do the trick. but the article says its suspected the systems were rooted by someone with root console access, nothing you can do if someone has the root password (unless, perhaps you have a really nice syslog setup)
“but the point is that something has apparently been written to exploit a Linux system.”
No, something has exploited a web hosting server. The article says the means by which the systems were rooted are not known. If you hand me the root password to a system I dont need to write code to ‘exploit it’
“It should be a major wake-up call to those who tout its supposedly un-hackable properties.”
Read the article, its not knows if the linux system was ‘hacked’ because of technology or social engineering (nothing you can do but education about the later). Or if this system was very poorly configured.
“Then again, a sufficiently adept and dedicated programmer with access to the open source code could probably figure out a clever way to approach the problem, precisely to avoid or disguise the things you’ve pointed out.”
99.999% of systems compromises are inside jobs by people with root..
OK, I went to both us-cert.org and isc.sans.org, and can’t find anything about this ‘infestation’. I would suspect that if it were - or had been - ‘in the wild’ for any length of time that certainly one or both of those would have some prominent warning on their websites.
I’ve got a dedicated Linux box in a major NOC that’s being hammered every second by ChiCom hackers, spammers and general script kiddies/Windoze zombies. Just downloaded the latest chkrootkit and ran it - came up clean. I’m online via SSH for most of every day and while I’ve had some interesting compromises and attacks- generally the script kiddies - there really hasn’t been much to speak of. Other than the standard spammers, and I’ve locked out most non-USA countries from port 25, so I’m not getting a LOT of spam that I would if they weren’t blocked in IPTABLES.
I am gonna have to dl and setup tripwire, but I just haven’t gotten around to it yet.
YMMV
But this article sounds a bit like a ‘security software’ company’s FUD about suddenly discovering a ‘vulnerability’. I’ll wait until some really legitimate security organizations start posting the warning...
#8^D
Yea without tripwire Im not huge on trusting systems... BTW its best to use trip wire on a clean system (fresh install)
"compromised Web servers are infecting thousands of visitors daily with malware that turns their Windows machines into unwitting bots"
This is absolutely nothing new. It happens daily and some eeeeeevil "Linux rootkit" isn't really required. The only thing necessary is to have the client machines running Windoze with any version of IE and/or LookOut.
If this "exploit" is only infecting thousands of Windoze machines, it's pretty lame. One spammer sending out a million or so emails with an infected attachment or a phishing URL will infect more than thousands of Windoze machines. The compromise is because of Windoze total lack of security and complete unpatchability, not any "linux exploit".
Jeez, who wrote this piece of trash? Rob Enderly? Ken Brown of the Alexis de Tocqueville Institution? Some other M$ bought-and-paid-for "independent research company"?
Again, I think I'll wait and see what really legitimate security organizations have to say about this.
Then you had best not go to probably 99.9% of the web sites out there, because probably very few are actually running tripwire. Yeah, the real gearheads will be, but most virtual or dedicated server boxen probably aren't.
As far as a fresh install, my box went online six years ago, and the only major thing that has been done to it was replacing the HD and doing a restore to it. The NOC guys did that, as it's actually their hardware in the server farm and they have to maintain it. About the only times it is rebooted is when there's some power glitch or other major outage. It's been up 181 days since I had them last reboot. If it was a Windoze server, they'd be rebooting it daily or weekly, just to free the memory of all the garbage collection it does.
So doing fresh install of the system just to install tripwire wouldn't be something I worry about. Plus, if the binaries or system files had been corrupted or hacked, I'd probably have found out about it by now, after 5 or 6 years. But I certainly understand -and share- your point of view.
I meant before I would deploy such a system
It's been up 181 days since I had them last reboot. If it was a Windoze server, they'd be rebooting it daily or weekly, just to free the memory of all the garbage collection it does.
The mail server I set up at my previous employer (with tw of course) was up 1018 days before it was decommissioned and replaced with *gag* an exchange environment they made me set up.. Of course all their apps which did mail handling started to blow up and I had to fix that too..
So doing fresh install of the system just to install tripwire wouldn't be something I worry about.
The only point being that if you're rooted already when you set up TW will do *nothing* for you unless you set up a clean control box and use the TWDB from that box.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.