Posted on 07/21/2021 9:16:16 AM PDT by ShadowAce
There is a lot of attention being paid to continuously updating servers to patch security vulnerabilities on Linux servers running in data centers – a basic step underpinning technology infrastructure in every industry. Yet, staff resources to deal with maintaining servers are not sufficient to meet the workload, said 55% of respondents in a worldwide survey by CloudLinux.
The survey finds 76% are deploying automated patching procedures and that live patching to fix vulnerabilities is commonly used (47%) to avoid downtime that is normally associated with patching. This is not surprising given the volume of vulnerabilities that are discovered and patched every week. There are simply too many patches to apply to do so manually and IT professionals are using automated tools to help keep up with the volume.
Yet, the survey found that manually researching vulnerabilities online is the most commonly used method (75%) in vulnerability management. It suggests that while automation has a place, some organizations have not fully embraced automation – and that automation may not cover all aspects of vulnerability management.
“There is no doubt that organizations of every size are struggling to keep their server fleets up to date in their efforts to patch security vulnerabilities,” said Jim Jackson, president and chief revenue officer, CloudLinux.
It was learned that 45% said they cope with vulnerabilities simply by waiting for the next periodic maintenance window before applying patches. This means that during that period of time their servers remain vulnerable — a less than optimal situation.
A notable finding is that 73% of respondents rely on a single operating system in their server fleets suggesting that organizations value the ease of maintenance of using a single Linux distribution rather than utilizing specialized Linux distributions for different roles. Most commonly used were either CentOS or another CentOS fork.
Respondents were asked what features they would like to see in a patch management tool with the three most desired cited as: fast responses to new critical vulnerabilities and exposures (CVEs) (88%); live patching (75%); and automated comprehensive reporting (70%).
Can’t they just hire some nerd from India to do this remotely? It would be cheaper and those old, “experienced” IT people ask for like “decent” salaries. Too expensive.
I update/patch 550-650 servers every month. It takes me about 30-45 minutes of total time every month to perform it. It's not difficult.
I've written a script that takes in a list of servers, along with other information, and then writes out 2 or 3 other scripts onto the target server, and schedules those scripts with an "at" job.
We use at because of our internal policies on updating do not correspond with static days of the month. Also, other co-workers have tried doing the same thing with Ansible, but it lacked the consistency of my scripts, and they were not performing the job correctly.
Scripting makes my job so mu easier.
The fundamental problem with MS security is MS Exchange.
It is broke and the way it interacts with Outlook is flawed.
Linix is pretty solid as long as you use a good firewall.
OSX is limited by what firewall its connected to.
Apple doesn’t build servers but at least their OS is based on a secure platform.
In every case so far, the backdoor for randsomware has been MS Exchange.
Higher end cyber security professionals, the ones with degrees, certifications and years of experience, easily command over $150,000 per year. However, many executives feel that is too much to pay and that the skills can just be hired off the street or overseas. That is simply not the case.
Hiring someone’s nephew who cant pass a drug or background test and has no credentials to speak of is a sure way to mess up your IT environment, sometimes that mess up can be either fatal or far more expensive that the cost of a good IT professional.
I often ask this question. If you business is your “baby” do you want a medial professional trying to heal your baby, or do you want someone who has a criminal record, no experience, no training, not bonded or insured and has no customers that they can reference?
Higher end cyber security professionals, the ones with degrees, certifications and years of experience, easily command over $150,000 per year. However, many executives feel that is too much to pay and that the skills can just be hired off the street or overseas. That is simply not the case.
Hiring someone’s nephew who cant pass a drug or background test and has no credentials to speak of is a sure way to mess up your IT environment, sometimes that mess up can be either fatal or far more expensive that the cost of a good IT professional.
I often ask this question. If you business is your “baby” do you want a medial professional trying to heal your baby, or do you want someone who has a criminal record, no experience, no training, not bonded or insured and has no customers that they can reference?
Not sure how that happened. It posted my last instead of what I was writing. Anyway, trying again.
What scripting language do you use?
Sounds like we need more Diversity, Inclusion, Equity (formely known as Affirmative Action) techie hires. /sarc
I script in Expect and bash.
I also use BASH but have been using Python. Thanks for the pointer to Expect. My quick read of Wiki tells me that it is something useful that I should look into.
Well it’s boring. The IT jobs that young people want are UI, UX and WebDev because you can see your results in a finished product.
I got a chance to support Enterprise patching in a NERC environment. Not fun. Yes, the actual patching took minutes (unless they were dot-net patches which took forever!) but the documentation associated with SOX (Sarbanes–Oxley) and NERC was a nightmare.
Basically, you had to take a “pre” snapshot of every device to be amended - times 1-2K devices. Then you had to analyze the patch library and cull the non-applicable ones (NOTHING goes on a NERC-governed computer that hasn’t been fully vetted as necessary!). This list of recommended patches largely came from Microsoft, but also included firmware updates and 3rd party updates. Then comes patch testing - every month every patch has to be tested for compatibility. Another snapshot has to be done of the test-patched machines so that an analysis of potential compromises in security have not been introduced.
Mitigation takes place on any non-compliant patches. Results of these analyses goes to a board that makes a determination of need (yea or nay to go ahead and include a particular patch or patches).
Once completed, Change Control requests are prepared and submitted. The CC board studies the requests and approves the schedules for implementation. Conflicts in scheduling are resolved (for instance the owner of a particular server has scheduled the same time slot for upgrade of an application or hardware modification/repair). Alternate dates are hammered out and posted.
The patch team then implements the various patches. Following that another snapshot is taken of every patched machine and analyzed for compliance. Requests are forwarded to the server team for machines that reported patching issues.
Keep in mind that some of these steps pertained only to NERC machines. but also keep in mind that there were separate support teams for standard production servers, test servers, and NERC servers.
AFAIK, there is only one book in existence for Expect (O'Reilly), so when you find it, that's the one to buy. :)
I spent an incredibly short time (a 1-week project) in a NERC environment. I do not envy the admins there.
I’m sure congress is ready to authorize another 50K+ H1-B visas to get the foreign help companies need to protect themselves.../s
Thanks! I just grabbed the Kindle for the book.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.