Amusing virtualization host performance comparison

[OVERKILL]

I have an old ThinkServer RD430 still in a rack, was using it for basically a "scratch" environment, had nothing important on it. OS was Windows Server 2019, had a few VM's spun-up on it, both running OpenBSD.

Had an employee reach out to me as he was wanting to do some dev work and needed a Linux host, so I spun-up two VM's for him, since the Linux host was isolated, one running Rocky 9, the other Server 2019 with TeamViewer so he could get a desktop to access the Linux host while being isolated.

The performance was BRUTAL. Windows took hours to do updates, the thing was choking on any task you gave it. Rocky was also quite slow, despite just being CLI, it struggled just doing updates. I gave him access but told him it was bad. Not surprising given the age of the hardware right?

Got thinking about it further and was like "why the hell am I running this on Windows?" And couldn't come up with a reasonable answer. When the box was commissioned, like >10 years ago, in a bare-metal role for a task it's no longer doing, Windows was the necessary OS, and it was eventually upgraded to Server 2019. When it was removed from that role, why I didn't wipe it, before using it for a VM host, I don't really know, laziness? Probably. But Hyper-V is familiar and easy, and works well enough most of the time and was OK for the nothing I was asking it to do

Anyway, I told the guy to hold off on using it, I was going to redo the host.

Wiped it, installed Rocky 9 (it won't run 10, CPU isn't supported), created his dev environment and then created his access environment (server 2019) and the performance was night and day. This old pile was totally usable! Updates took about 30 minutes, which, I was pleasantly surprised with. Handed access off to him today and he commented on the massive difference in performance.

This was the first time I'd ever back-to-back'd KVM with Hyper-V, and this was NOT a good showing for Microsoft.

 

[MVAR]

Soooooooooooo..................... I'm confident that I can't speak IT.

 

[Rand]

Soooooooooooo..................... I'm confident that I can't speak IT.

Cliff notes:
he switched the VM host from server 2019 to rocky 9 (linux)
performance massively better.

 

[wwillson]

Windows does a lot of writes, many more than Linux.

Does the old server have a hardware RAID controller? If so, make sure the BBU is charged and turn on write-back so the RAID controller can coalesce block writes. You can't believe how much faster the whole system will be compared to writing one block at a time.

 

[JeffKeryk]

Nice work @OVERKILL
I hate computers.

 

[OVERKILL]

Windows does a lot of writes, many more than Linux.

Does the old server has a hardware RAID controller? If so, make sure the BBU is charged and turn on write-back so the RAID controller can coalesce block writes. You can't believe how much faster the whole system will be compared to writing one block at a time.

It does, but it's a steaming pile, it's a MegaRAID SAS controller, a 9240-8i.

 

[wwillson]

The 9240 has no onboard DRAM cache, so heavy writes are glacial. Sorry

 

[OVERKILL]

The 9240 has no onboard DRAM cache, so heavy writes are glacial. Sorry

Yeah, as I said, steaming pile

 

[wwillson]

I have two LSI 9260 RAID controllers that have run almost continuously for 15 years. They have 512MB DRAM for cache and support up to 512GB SSDs for a huge read/write cache. They coalesce block writes to the hard drives and between the 512MB DRAM and the SSD write cache, they absolutely fly, even in RAID 5 mode. You can buy them for $50 now.

 

[OVERKILL]

I have two LSI 9260 RAID controllers that have run almost continuously for 15 years. They have 512MB DRAM for cache and support up to 512GB SSDs for a huge read/write cache. They coalesce block writes to the hard drives and between the 512MB DRAM and the SSD write cache, they absolutely fly, even in RAID 5 mode. You can buy them for $50 now.

I mean, this thing should have been retired a couple of years ago now, I'm not worried enough about it to bother putting anything into it, it's not doing any workloads for me, and as long as it works fine for what the employee is using it for, that's sufficient.

 

[JavierH19]

mhm yes computer beep boop beep (I understood none of this other than linux performance better)

 

[PandaBear]

Windows isn't really a right VM host unless you need it for some odd reason, like being a main machine you just use the VM on demand in light weight load.

I would imagine the right kind of VM host would be as close to a near bare metal RTOS, even Linux would be too heavy. Does FreeRTOS VM host exist? If not maybe Fedora CoreOS or Alpine Linux?

 

[OVERKILL]

Windows isn't really a right VM host unless you need it for some odd reason, like being a main machine you just use the VM on demand in light weight load.

I would imagine the right kind of VM host would be as close to a near bare metal RTOS, even Linux would be too heavy. Does FreeRTOS VM host exist? If not maybe Fedora CoreOS or Alpine Linux?

Fedora is the (free) consumer version of Redhat, Rocky is the free version of RedHat Enterprise, hence it having this flash webUI for managing VM's. KVM is generally regarded as being an extremely efficient virtualization method (kernel-level virtualization).

Hyper-V is easy to use, that's why it's as popular as it is, but it's clearly far less efficient than KVM, as this back-to-back on the same hardware shows.

 

[wwillson]

I would imagine the right kind of VM host would be as close to a near bare metal RTOS, even Linux would be too heavy.

VMWare is the pinnacle of hypervisors and has a dedicated micro kernel that is not Linux. It is very fast and highly capable. However, the cost of the licenses will scare most non-enterprise customers away.

I host lots of VMs on Debian Linux KVM. The difference between running Linux on hardware vs. running it in a VM on Linux KVM is about a 10% penalty in CPU performance. I don't know what the CPU performance penalty is in VMWare VMs is, but I suspect it's similar to KVM. My point is KVM is clean lean and fast. You don't need to go on a wild goose chase to find faster virtualization, KVM is very efficient.

 

[rijndael]

VMWare is the pinnacle of hypervisors and has a dedicated micro kernel that is not Linux. It is very fast and highly capable. However, the cost of the licenses will scare most non-enterprise customers away.

Especially post Broadcom purchase. But at least Workstation is now free.

 

[OVERKILL]

VMWare is the pinnacle of hypervisors and has a dedicated micro kernel that is not Linux. It is very fast and highly capable. However, the cost of the licenses will scare most non-enterprise customers away.

I host lots of VMs on Debian Linux KVM. The difference between running Linux on hardware vs. running it in a VM on Linux KVM is about a 10% penalty in CPU performance. I don't know what the CPU performance penalty is in VMWare VMs is, but I suspect it's similar to KVM. My point is KVM is clean lean and fast. You don't need to go on a wild goose chase to find faster virtualization, KVM is very efficient.

I've read that due to KVM being leaner (less code), that it's actually a bit faster than ESXi, as they are both Type 1 hypervisors.

 

[Vil_Riker]

Especially post Broadcom purchase. But at least Workstation is now free.

Yeah, we are getting pillaged with the new license fees.

 

[TumblrDucks]

I have an old ThinkServer RD430 still in a rack, was using it for basically a "scratch" environment, had nothing important on it. OS was Windows Server 2019, had a few VM's spun-up on it, both running OpenBSD.

Had an employee reach out to me as he was wanting to do some dev work and needed a Linux host, so I spun-up two VM's for him, since the Linux host was isolated, one running Rocky 9, the other Server 2019 with TeamViewer so he could get a desktop to access the Linux host while being isolated.

The performance was BRUTAL. Windows took hours to do updates, the thing was choking on any task you gave it. Rocky was also quite slow, despite just being CLI, it struggled just doing updates. I gave him access but told him it was bad. Not surprising given the age of the hardware right?

Got thinking about it further and was like "why the hell am I running this on Windows?" And couldn't come up with a reasonable answer. When the box was commissioned, like >10 years ago, in a bare-metal role for a task it's no longer doing, Windows was the necessary OS, and it was eventually upgraded to Server 2019. When it was removed from that role, why I didn't wipe it, before using it for a VM host, I don't really know, laziness? Probably. But Hyper-V is familiar and easy, and works well enough most of the time and was OK for the nothing I was asking it to do

Anyway, I told the guy to hold off on using it, I was going to redo the host.

Wiped it, installed Rocky 9 (it won't run 10, CPU isn't supported), created his dev environment and then created his access environment (server 2019) and the performance was night and day. This old pile was totally usable! Updates took about 30 minutes, which, I was pleasantly surprised with. Handed access off to him today and he commented on the massive difference in performance.

View attachment 326825

This was the first time I'd ever back-to-back'd KVM with Hyper-V, and this was NOT a good showing for Microsoft.

Would it perform better if you as server admin disable the update? It's quite annoying that Microsoft force you doing update when you need the computer to work right here, right now lol.

 

[OVERKILL]

Would it perform better if you as server admin disable the update? It's quite annoying that Microsoft force you doing update when you need the computer to work right here, right now lol.

No, I can't see Windows updates having a meaningful impact on the performance, KVM is just a superior (lower resource footprint, more efficient) Hypervisor.

 

[meep]

For reference, I had a dell R740 (2 socket, 2U rack mount) built from refurb parts through an Amazon vendor last month, 2ssd and 2hd with 128G ram, about $2k shipped with sales tax. New stuff is stupid high but a home lab can use these to the nines. I’ve also enjoyed several refurb dell precision T5820s, which are basically single socket xeons with 4 removal front hard drives, under $1k each. My home network uses a 5820 to run blueiris, endian UTM and freenas under the last available free-home use ESX. It’s a nice little beast.