Fun fact: The outdated software runs on outdated hardware, too.
In January 2021, Microsoft pushed a kb that would make your server reboot constantly if it was running server 2012 and was either a domain controller or a hyperV host.
Guess how many domain controllers went down that day.
Yeah but domain controller so hard to migrate and so sensitive! Better let them rot on old unsupported software versions!
i mean yeah but bean counters up top want me solving the current emergency (caused by similar forms of neglect from years ago)
ill get to it when it breaks i guess
So fight bean counting with bean counting
Best formula is average employee pay × number of employees × time to fix = money lost.
The trick is to find a solution that is lower than money lost.
Say 20 employees at €35/h for 4 hours. €2800 is how much the company lost in wages.
You will find bean counters are more acceptable to a €5000 server over 5 years if it prevents €2800 of lost wages per patch Wednesday.
i wish i was in a good position to argue them out of their dumb greed.
Please tell me that was a bad patch and not on purpose
Who knows? :D
Trigger me timbers
What has two thumbs and just spent all week hectoring the boss to upgrade from Server 2008 to 2022 so docker and ssh would finally work?
👍🏻👍🏻
Well girls, we're living in the future now! Five new 2022 servers, all turned into dumb ssh+docker nodes in my job cluster!
Wipes brow with a trembling hand
Grumble grumble... they wouldn't let me upgrade to Linux just yet though... But the plan is coming together... evil laugh
Do they hate money? Paying for Windows server just to run docker is an expensive option.
There's two ways to perform every task. There's the way we say and maintain the illusion of doing. And, there's the practical way we actually get the work done. If we don't maintain the illusion then they'll cut budget. If they cut our budget we can't even afford the practical way, let alone what they think we're doing.
Your success in this position will be determined by how quickly you learn both processes and how well you choose which is appropriate for the situation.
TBF all the jobs are a decade old and written by our researchers in dotnet framework as Winforms apps I hacked up to be console apps so it's gotta be windows. I'm converting them one by one to dotnet core and moving them to my Linux containers but it's a slow process and I've got a v1 release to prepare for next month.
Everyone is just stoked that no longer do a half dozen researchers have to twice a day log in to their pet server, open their Winforms app, run it, and copy paste the results to a shared drive. Now my docker harness does it all on a scheduled task triggered automatically from rundeck server I manage. WE'RE LIVING IN THE FUTURE BABY
I'm sure it's not that simple but .Net is and has been on Linux https://learn.microsoft.com/en-us/dotnet/core/install/linux
Docker images I have run dotnet in a container but the docker server host is Ubuntu. Though I really should flatten it and run it on proxmox.
However, it's not like that would save real dollars on licensing we have Windows servers still for AD et. al. and therefore have to license all CPU cores in a hypervisor cluster so having fewer windows servers is irrelevant in our environment with regards to license costs.
Oh yeah, all my code is dotnet core running on Ubuntu servers in docker.
Just all this legacy code is written in dotnet framework which doesn't run on Linux, and requires some moderate effort to switch (relies on libraries that are framework, and those also rely on framework libraries, etc)
It's completely possible, but for now, I've got these 2022 servers running "good enough" to go to production, and I'll convert them as soon as the first issue arises.
I feel ya man. I spent a year arguing for the existence of a pilot environment.
Because when you test in production, it's bad, mmmkay.
Oof that's a rough one indeed!
Me: "Hey whats that feature we need to implement into our software?" Boss: "Ntlm passthrough" Me: ".... Hey boss about 90% of the stuff i find online is about how ntlm is insecure and should be shut off wherever you see it?" Boss: "Yeah... But everyone still uses it everywhere. Just implement it and dont think about it."
I'm an IT sub roundabout working for the US government. We've a multi-site contract and arrive at the one we'd been vaguely warned about: Some contractors got fired mid-job in the 90s and left some trash.
The hallway we needed to go down was filled with all sorts of shit, waist deep, for about twenty feet. My co-worker and I put on some gloves and started making a path. We found just a little had fallen on a path made by those that came before us.
About halfway through the hallway trash I see a small, solid green light reflecting off the floor. After a little digging we find a beige metal half tower complete with Pentium and Win 3.1 stickers, laying on it's side but upside down, power and network ran into what looked like a hole in the wall made with multiple blows from a hammer. It wasn't in the documentation that we could see.
In the confusion of a vendor fuckup someone decided taking a undocumented hammer to the rules best served society. Everyone who saw it afterwards decided to keep their mouth shut. We favored past wisdom and present uptime. We buried the twenty five year old rig again, hiding it from view while ensuring good air flow.
(Running machines with windows 2000.) You guys are running windows server 2008?!
Real story: our primary and backup DCs are still on server 2003. 🫠
Cries in Windows Server 2003.
Can confirm.
We have several Server 2008 still running.
On the bright side not my problem to fix.
Just don't as me about that switch that is 2 generations old or the AP that has +1000 days uptime.
I've said it before and I'll say it again - Cybersecurity as an enormous global industry owes it's existence to Microsoft. Period.
If Microsoft suddenly disappeared, cybersecurity would be more like Accounting - basic systems, managed in-house.
Horse manure! It owes it to managers that want to invest in new toys and stuff and don't want to hear/invest/spend on keeping stuff operational.
This is why a lot of companies end up leasing notebooks and stuff, cause then IT does not have to explain why it is time to replace hardware.. lease is up is something they understand. If you buy (which is cheaper) you end up fucking yourself cause by the time it needs to be replaced some penny pincher higher up will say.. nah this is still good for a few more years. And before you know it you are stuck with outdated crap that costs more and more time and effort to keep operational.
Same with infra.. and why IT pushes for cloud first. It's working so it's fine. No matter the switches are EOL and the server hardware is EOL and so is the OS without ESU.. we need to invest in this new piece of stuff.. no money for the rest.. just keep it running.
But to that point - they inevitably spend millions on Microsoft either through windows laptops or office bundling because they buy the spew that “Microsoft will support it” and “If we get breached because of a problem with Microsoft they’ll cover us” or some similar crap.
No, and no. By the way, IT managers.
Building it is not always the right answer, and yes a Linux workstation for sales is gonna get people upset still, but. This moron treadmill of chasing Microsoft through whatever their latest absurdities are is heinously expensive and pathetic. Are you an IT company or not? Well?
I was going to make a Linux joke being why my company's security has been stable... Until the XZ Utils exploit.
Yes, cybersecurity wouldn’t disappear, it just wouldn’t be the humongous, roiling, clusterf**k it usually is.
Name a high-profile breach on a *nix system not due to configuration (user) error. I’d add “or a hardware/firmware hack” but you get the idea.
Just this months patch tuesday notes:
Today is Microsoft's August 2024 Patch Tuesday, which includes security updates for 89 flaws, including six actively exploited and three publicly disclosed zero-days. Microsoft is still working on an update for a tenth publicly disclosed zero-day.
Sure, security vulnerabilities exist in the linux world, but luckily not that many.
yes and no. if you look at the number of reported CVEs, debian takes the crown of all operating systems. still feeling more secure on linux than any closed source system
Yeah, that's because there's an entire cottage industry of people scraping old bug reports, and linter errors to create CVEs they can sell to customers worrying about security. It creates a huge number of false positives. E.g. see https://daniel.haxx.se/blog/2023/08/26/cve-2020-19909-is-everything-that-is-wrong-with-cves/
I think any measure that is looking at a raw count is going to be meaningless. Particularly, comparing raw counts between open and closed software.
For context, Vista was 2007.
Vista and Windows Server 2008 are very closely related. In fact, they even use the same installer for Service Packs.
And server 2012 has the windows 8 Fullscreen start menu (and classic shell is compatible, if you can't install SP2). In recent years I hear they're separating from year/Pc version parity? Thankfully I now work in Linux admin, so that's not longer my concern.
GCC 4.1.2 on linux 32-bits system. No worries it's only for hospitals...
Guys who just got put in charge of managing our end of life process
<---- this guy
Servers are about to die
My system at work was built in the late 1960s, but it's not Microsoft software, so it's all good
Holy shit I'm involved in a similar situation - except we really, REALLY want it gone - it's just a game of "if you give a mouse a cookie" trying to upgrade all the similarly fragile downstream stuff so we don't blow up the entire operation when we switch.
One of my friends recently commented on how its funny that they have Server 2008 experience. They're 21 years old and just graduated college last fall, meaning that was a server they encountered in the Year of our Lord, 2024
As annoying as our CISO is, that’s one thing he’s been effective with. We’ve gotten rid of all EOL Windows versions. We’re starting on 2016 right now.
Shoutout to the accountants still running RTS Advantage in 2024 via vDosPro
Accountants have it hard even if they do keep up-to-date. We migrated one to a new server, but some of their clients still use archaic versions of accounting software. Unfortunately beyond a particular age it won't activate the new install, so they are stuck telling the client to upgrade or continue to run the old server as well.
The British government still uses Windows 98 in some capacity, I believe.
Most bank databases run on legacy software from the 80s. AS/400 is alive and kicking.
Windows XP "server" along with 2003 here. We've been trying to get rid of them for years.
as long as updates keep coming, don't fix what isn't broken
Only the hottest memes in Cybersecurity