Tip: Adding Support for 802.1x Through the Hyper-V vSwitch in Windows 10 1809 and Windows Server 2019

Starting with Windows 10 1809 and Windows Server 2019, Microsoft added the ability to provide support for 802.1x through the Hyper-V vSwitch.

To enable it, a registry modification is required. Run the following with elevated credentials:

reg add “HKEY_LOCAL_MACHINE\SYSTEM\CURRENTControlSet\vmsmp\parameters” /v 8021xEnabled /t REG_DWORD /d 1 /f
shutdown /r /t 0

This makes the necessary registry modification and then restarts the computer.

AWS Adds Hyper-V Migration Support

Amazon has announced that it has added the ability to mass migrate Hyper-V farms to AWS.

AWS Server Migration Service now supports the ability to migrate Hyper-V VMs to AWS. With this launch, you can now migrate virtual machines running in on-premises virtualization stacks from both Microsoft Hyper-V and VMware ESX/ESXi environments. AWS Server Migration Service is an agentless service that makes it easier to migrate thousands of on-premises workloads to AWS. It allows you to automate, schedule, and track incremental replications of live server volumes, making it easier for you to coordinate large-scale server migrations. Automating incremental replication, Server Migration Service helps you speed up your migration process and reduce the operational cost of migration.

Details: https://aws.amazon.com/server-migration-service/


Looking for an awesome, no-nonsense technical conference for IT Pros, Developers, and DevOps? IT/Dev Connections kicks off in Dallas, Texas in 2018!

IT/Dev Connections

Hyper-V Performance Monitor Tool

Monitor and view Hyper-V VM Guest Performance on your Hyper-V Hosts. This free PerfMon Tool Monitors CPU, Memory, Disk I/O and Network Statistics as seen from inside the Guest. All data is retrieved via WMI only. No modules or software install required. Run from any member server.

Download: Hyper-V Performance Monitor Tool (PowerShell)


Looking for an awesome, no-nonsense technical conference for IT Pros, Developers, and DevOps? IT/Dev Connections kicks off in San Francisco in 2017!

IT/Dev Connections

How to Compact and Shrink the Size of a VHD File

By Garth Jones

In a recent blog post, How to Expand a Disk within a VM That Is Using a VHD File, I showed you how to expand a VHDX file in logical disk size. I did this because I had a problem with the auto-growth of my SQL Server logs. Now that this problem is fixed, I want to return the VHDX file back to its normal logical size of 128GB, and make the VHDX file as small as possible. This blog post will show you how to perform these tasks.

I can hear some of you saying, “Why? You are using a dynamically expanding VHDX file, so why bother compacting and shrinking the file?”

As a refresher, remember that a dynamically expanding VHDX file will start at a few MBs and grow as files are added to the virtual hard drive. If you delete a file within the virtual hard drive, the space is freed up only with the virtual hard drive and not from the physical VHDX files. To put it another way, once the VHDX file is expanded, it will never shrink without you following these steps.

Here are a few reasons why you would want to shrink the VHDX file:

· Shrinking the VHDX file to the smallest possible size will save disk space on the host.

· A small VHDX file leads to faster backups. 3GB is a lot faster to copy than 128GB or 256GB.

· If a problem like the SQL Server log auto-growth happens again, I want to know about it sooner rather than later. By having a small logical VHDX file size, this problem will be seen, hopefully, earlier.

How to Compact and Shrink the Size of a VHD File-New Volume-TB

To begin with, I always recommend that you never install anything on the C:\ drive. It can be a huge pain to logon to a server when the C:\ drive is out of space. This of course will depend on your GPO security settings!

In my blog post, How to Expand a Disk within a VM That Is Using a VHD File, I expanded my VHDX file to 1.2TB. In a later blog post, What Is the Problem with My SSRS Server?, I cleaned up the SSRS log file, thereby returning the server to its normal operating state. The screenshot above shows you that I cleaned up my virtual hard drive and that 1.24TB are free and only 2.2GB of space are used.

How to Compact and Shrink the Size of a VHD File-VHDX-256GB 

Now looking at the physical VHDX file, I can see that it is 256GB in size (of a possible 1.2TB), but in the first screenshot you can see it’s using a little over 2.2GB of space within the VHDX file. Using the processes below, I will first shrink the volume size to 128GB then compact the physical VHDX file to the smallest possible size.

How do you go about this process? In a nutshell, I will perform the same steps I did to expand the VHD, but in reverse. I need to start with the VM itself by cleaning-up, defragging and shrinking the volume size.

VM Clean-up Tasks

There are several things to note before getting started:

· You should make a copy of the VHDX file before starting, or have a good backup of the server.

· Within the VM make sure that the Recycle Bin is emptied.

How to Compact and Shrink the Size of a VHD File-Recycle Bin 

· You need to defrag the drive BEFORE shrinking the volume size and compacting the VHDX file.

· Multiple defrag passes might be needed.

· If the disk has any issues, it will NOT shrink or compact. Make sure that chkdsk is run.

Shrink the Volume Size

How to Compact and Shrink the Size of a VHD File-Shrink Volume

Open Computer Management and select the Disk Management node. Next, select volume (E:), right click on it and select Shrink Volume…

How to Compact and Shrink the Size of a VHD File-Shrink Space

Wait for Computer Management to analyze the volume.

How to Compact and Shrink the Size of a VHD File-Shrink E 

I will select the suggested new size. Click Shrink to continue.

How to Compact and Shrink the Size of a VHD File-Shrink 128GB 

Now my disk is back to 128GB in size. Time to compact the VHDX file. I’ll start by shutting down the VM.

Compact the VHDX File

These next set of steps will reduce/compact the physical size of the VHDX file.

How to Compact and Shrink the Size of a VHD File-Compact Settings

In Hyper-V Manager, bring up the Settings page for the VM. Click on the Edit button.

How to Compact and Shrink the Size of a VHD File-Compact Locate Disk 

Click Next.

How to Compact and Shrink the Size of a VHD File-Compact Choose Action

Select Compact and then click Next. This will reduce the phyiscal size, but not the logical size of the VHDX file.

How to Compact and Shrink the Size of a VHD File-Compact Summary

Click Finish.

How to Compact and Shrink the Size of a VHD File-VHDX-3GB

Now if you look at the VHDX’s file size, it was reduced to the bare minimum size of 3GB. Keep in mind that your minimum size may be different.

Shrink the VHDX File

These next steps will shrink the logical size of the VHDX file.

How to Compact and Shrink the Size of a VHD File-Settings

Again, in Hyper-V Manager, bring up the Settings page for the VM. Click on the Edit button.

How to Compact and Shrink the Size of a VHD File-Locate Disk

Click Next.

How to Compact and Shrink the Size of a VHD File-Choose Action

Select Shrink and then click Next.

How to Compact and Shrink the Size of a VHD File-Configure Disk 

Set the New size to 128GB, and then click on the Finish button.

How to Compact and Shrink the Size of a VHD File-OK 

Click OK to close the VM’s Settings window. Now start the VM.

How to Compact and Shrink the Size of a VHD File-New Volume-GB

After logging onto the VM, review the disk size. Notice that it is back to its original size.

How to Compact and Shrink the Size of a VHD File-VHDX-April 

Now look at the physical VHDX file (see screenshot above). It is now 3.03GB in size. When you compare this to the 256GB it was at the beginning of this blog post, that’s a huge size difference. This will help make my backups faster due to the smaller file size!

If you have any questions, please feel free to contact me @GarthMJ.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance

By Garth Jones

Recently, I noticed that one of my Hyper-V test labs consistently had a Disk Queue Length of 5 or more disk read/writes. A disk queue length of one or more means that there’s a bottleneck every time the server/application tries to access the disks. In most cases, the server/application is doing nothing. This explains why you will see pauses in your server/application when it is stuck waiting. Since disk I/O is the number one killer of VM performance, anything to reduce it is a good thing!

This blog post will focus solely on reducing unnecessary disk I/O caused by Endpoint Protection (when it scans for viruses and malware) in System Center Configuration Manager.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-First Disk Queue Length

Not too long ago, I wrote a blog post about how to exclude VHD* files from Endpoint Protection scanning, but I wanted to do more, so I searched for a guide that listed other exclusions. I came across this helpful article, Hyper-V: Anti-Virus Exclusions for Hyper-V Hosts, on TechNet.

In this blog post I will show you how to implement this article’s suggestions for Endpoint Protection.

Background

In my test lab, similar to any other type of server (physical or virtual), the C:\ drive is used for hosting the operating system (OS) only (in this case RAID 1), and the physical E:\ drive is setup as a RAID 10, which hosts all my VM’s VHD files. My D:\ drive is the DVD drive.

An increase in disk queue length, in my case, meant that the VMs “felt” slow. If I only looked at the VMs I’d never know the real reason behind this problem. Instead, I had to look at the Hyper-V host first in order to understand the true nature of the problem.

Anti-virus (AV) software inspects a file to check for all known viruses and malware. It is this scanning process that slows down an application’s/server’s access to the file. The AV software must finish its scanning before access is granted to the file. It also means that the bigger the file, the longer it takes to scan for viruses and malware.

VM files (VSV, VHD, VHDX, VHDA, & VHDXA) along with ISO are huge, and therefore take a while to scan. This can lead to an increase in disk queue lengths while the AV software is busy scanning these files, as was the case in my test lab. My goal was to reduce the overhead of the file scanning. I did this by telling my AV software (Endpoint Protection) to skip checking for viruses and malware on my VM files and ISO.

As I indicated above, this blog post is not about reducing all disk I/O, but it will focus on reducing unnecessary disk I/O caused by Endpoint Protection (when it scans for viruses and malware) in System Center Configuration Manager.

Recommended Exclusions

Using the AV Hyper-V exclusions listed on the TechNet article’s page, I will implement the File, Directory and Process exclusions that are recommended for Endpoint Protection.

The file types to exclude:

· VHD

· VHDX

· AVHD

· AVHDX

· VSV

· ISO

Process exclusions:

· Vmwp.exe

· Vmms.exe

Directory exclusions:

· %systemdrive%\ProgramData\Microsoft\Windows\Hyper-V\Snapshots

· C:\ProgramData\Microsoft\Windows\Hyper-V

· C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks

In my opinion, the file types and directory exclusions are self-explanatory, but the process exclusions are not, so here is a short note about both processes.

Vmwp.exe also known as Virtual Machine Worker Process is a component of the virtualization stack. There will be one VMWP.exe for each running VM. This exe can be found within the %windir%\system32\ directory.

Vmms.exe is the Virtual Machine Management Service. This is the main service used to manage Hyper-V VMs. This exe can be found within the %windir%\system32\ directory.

We can see below that vmwp.exe is listed several times in Resource Monitor.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Resource Monitor 

Adding the Exclusions

File Exclusions

Starting with the file exclusions, I will ensure that all recommended file exclusions are added to Endpoint Protection. For detailed how-to steps, please refer to my blog post, Configuration Manager, Endpoint Protection and Hyper-V.

Ultimately your file exclusions window will look similar to the screenshot below.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-File Exclusions 

With the file exclusions step completed, it is now time to exclude the two processes.

Process Exclusions

Start by opening the Configuration Manager console and expanding Endpoint Protection | Antimalware Policies. Next, select the Default Client Antimalware Policy and click on the Properties button.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Properties

In the Default Antimalware Policy window, select the Exclusion settings node. Then click on the Set button for Excluded processes. In the Processes text box, enter %windir%\system32\vmwp.exe and then click on the Add button. Repeat the same step for vmms.exe. Once completed click on the OK button to close the open window.

Your process exclusions will look similar to the following screenshot:

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Process Exclusions

With that last step completed, Endpoint Protection will exclude scanning vmwp.exe and vmms.exe.

Directory Exclusions

It is never recommended to install anything on the OS drive. Instead you should always have a separate drive for your applications and files. Hyper-V is no exception to this rule. Therefore you should never host your VMs on your C:\ drive; this also includes VM snapshots.

After reviewing my server, even though I don’t store anything on the C:\ drive, Hyper-V itself still stores some files within the listed directories. As such, I will exclude each of these directories. Additionally, you should add a few others to match-up with your Hyper-V server configuration. In my case, E:\VM\* is where I store all of my VMs, so I will exclude this directory. Once the folders (directories) are added to the exclusions list, Endpoint Protection will no longer scan these folders for viruses.

Back in the Default Antimalware Policy window, select the Exclusion settings node. Then click on the Set button for Excluded files and folders.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Exclusion Settings

The first thing you will notice is that there are already exclusions listed.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Already Listed Exclusions

In the Files and Folders text box, add the directory that you want to exclude and then click on the Add button. In my example, it’s the %systemdrive%\ProgramData\Microsoft\Windows\Hyper-V\Snapshots directory.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Add

Repeat this process for the remaining folders that you want to exclude and finally click on OK to close all of the open windows. Your results will look similar to the one below.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-File and Folder Exclusions

Now that all of the exclusions (AV files, directories and processes) are added to Endpoint Protection, I know that any “slowness” on the part of my Hyper-V servers will not be as a result of Endpoint Protection. If my Hyper-V servers seem slow, I will need to look elsewhere for the cause.

Looking at the scale within the screenshot below you will notice that it is now between 0-1 compared to 0-50 in the first screenshot of this blog post! The disk queue length is now ~0.5 which is much better than before I excluded the files, directories and processes.

I know that there are many factors that affect disk queue length but, for now I am happy with my Hyper-V lab.

Reducing the Effects of Endpoint Protection on Hyper-V Server Performance-Second Disk Queue Length

If you have any questions, please feel free to contact me @GarthMJ.

Veeam Finds that KB4013429 Breaks Hyper-V 2016 Backup

Microsoft’s monthly updates always provides fodder for complaint – and that’s just with Microsoft’s own products. But, the inability to test against 3rd party products is another potential pitfall of Microsoft updates. That’s why its so important to test the updates before deployment.

Case in point: Veeam has found that the recent KB4013429 breaks the company’s Hyper-V 2016 Backup product.

Veeam’s alert here: Microsoft KB4013429 breaks Hyper-V 2016 Backup

The KB alert states that…

…the issue persists if the following conditions are met:

  • Cluster resource owner node is not upgraded
  • VM owner node is upgraded

 


Looking for an awesome, no-nonsense technical conference for IT Pros, Developers, and DevOps? IT/Dev Connections kicks off in San Francisco in 2017!

IT/Dev Connections

Configuration Manager, Endpoint Protection and Hyper-V

By Garth Jones

More and more companies are using Endpoint Protection in System Center Configuration Manager, along with the Hyper-V role, to virtualize computers within their environments.

It should be noted that in order to get the most out of the Hyper-V role you should exclude VHD* files from being scanned by any antivirus (AV) products, including Endpoint Protection. Excluding these files will prevent unnecessary AV scans on these very large files which then helps to increase disk I/O.

This blog post will show you how to exclude VHD* files from Endpoint Protection scanning.

Start by opening the Configuration Manager console and expanding Endpoint Protection | Antimalware Policies. Next, select the Default Client Antimalware Policy and click on the Properties button.

Configuration Manager-Endpoint Protection and Hyper-V-Properties

In the Default Antimalware Policy window, select the Exclusion settings node. Then click on the Set button for Excluded file types.

Configuration Manager-Endpoint Protection and Hyper-V-Exclusion Settings

In the Configure File Type Exclusions window, type .vhd and then click on the Add button.

Configuration Manager-Endpoint Protection and Hyper-V-Configure File Type Exclusions

Repeat this process for .vhdx extensions too. Your window should now look similar to the one below.

Configuration Manager-Endpoint Protection and Hyper-V-File Types 

Once all file type exclusions are added, click on the OK button twice to close the open windows.

Now all VHD* files will be excluded from AV scanning. This will ultimately lead to faster disk access on your Hyper-V role. If you have any questions, please feel free to contact me @GarthMJ.

Notes from the Field: Converting Hyper-V Server to Windows Server

By Garth Jones

In a previous blog post I talked about Broadcom NIC issues that cropped-up when I upgraded a Hyper-V Server to Windows Server 2012 R2 Standard. In this blog post I’ll talk about the lessons I learned during the upgrade.

During the upgrade process I also needed to convert a pass-through disk to a VHDX file, adjust the RAID configuration, and convert the host to Windows Server 2012 R2 Standard. I decided to change the RAID configuration to RAID 1 for the host OS C:\ and RAID 5 to hold all the VHDX files on D:\. This was done to allow the maximum disk space without sacrificing performance on the VMs.

At a high level the following steps were performed.

1. Backed-up both VMs to a Network Attached Storage (NAS) device.

2. Created an iSCSI (Internet Small Computer System Interface) target that was bigger than the pass-through disk.

3. Installed Windows Server 2012 R2 Standard on the C:\ drive.

4. Attached the iSCSI target as T:\.

5. Used Hyper-V Manager to create a VHDX file; saved the file to T:\ using the pass-through disk as the source disk.

6. Validated that the VHDX file was created correctly.

7. Reconfigured 10 of the drives to a RAID 5 configuration.

8. Formatted and assigned the newly created drive as D:\.

9. Copied the original VHDX file to the new drive from the NAS device.

10. Copied the new VHDX file (pass-through disk) from the iSCSI target.

11. Created both VMs within Hyper-V Manager.

12. Removed the iSCSI target from the host server.

13. Tested the VMs to ensure they were working correctly.

14. Created new back-ups of both VMs to the NAS device, including the new VHDX file from the pass-through disk.

15. Kept the migrated VHDX file for approximately 14 days as a backup.

This plan may sound simple, but what did I learn?

· You can’t directly upgrade from Hyper-V 2012 Server to Windows Server 2012 R2 Standard. You must format the OS drive and start fresh.

· Even if you select a dynamically expanding VHDX file of 128 GB (default size) when you copy the pass-through disk to the VHDX file, it will take the original size of the pass-through disk.

· Converting from a pass-through disk to VHDX file is extremely slow! A 6TB drive took ~16 hours to convert.

· If you have a Broadcom NIC, regardless of which manufacturer (Dell, HP, etc.), you have to check your settings on the host server.

· Set aside at least a few days to complete this task!

If you have any questions, please feel free to contact me @GarthMJ.

Notes from the Field-Converting Hyper-V Server to Windows Server

Hyper-V Server Slow? Check Broadcom NIC KB

By Garth Jones

The first day after upgrading from Hyper-V 2012 Server to Windows Server 2012 R2 Standard with the Hyper-V role installed, I was in the office early, like any good administrator, making sure that there weren’t any issues.

Unfortunately, before I had a chance to look at my emails, a colleague told me that the server was slow. Specifically, file transfers were very slow.

What happened?

When the request to upgrade the server originally crossed my desk, I wanted to upgrade Hyper-V 2012 Server to Windows Server 2012 R2 Standard, but my research told me that you couldn’t do that.

I had to change my plan to format the hard drives and install Windows Server 2012 R2 Standard. The update was also complicated by the fact that the Hyper-V host server had a pass-through disk for one of the Virtual Machines (VMs). Since I needed to start over, I also decided to maximize the hard drive (HD) space at the same time because there was a weird configuration for 10 of the 12 HDs.

This blog post isn’t about how to upgrade from Hyper-V Server 2012 to Windows Server 2012 R2 Standard. Instead, it explains how I fixed the slow file transfers to the Virtual Machine.

By way of background, this server has two VMs; one Domain Controller (DC) and one SharePoint server, plus the host server itself.

I started troubleshooting by copying a 195 MB file from my admin workstation to the server. I was getting 2.5 MB/second. I then copied the same file to the SharePoint server. I was getting 5 MB/second. I checked the setting for both VMs and I noticed that they shared the same physical NIC. Since this server has 4 NICs, I assigned each VM its own dedicated NIC. I then mapped one of the remaining NICs to the host server itself.

I re-did the same test, but the results were more or less the same. Next, I copied the file to the host server and it was so fast that the copy was done before any performance data could be displayed!

The test files were copied to the exact same drive, so disk I/O was clearly not the problem. It also meant that the physical network between my desktop and the server wasn’t the problem.

I did some more research and found that Dell has a KB that lists a known issue with Broadcom NIC. Even though I was working on an HP Enterprise server, also with a Broadcom NIC, I tried the suggestion within the KB. Within seconds my file transfers to the VMs mimicked what I could get to the host server. Problem solved.

If you have any questions, please feel free to contact me @GarthMJ.

Hyper-V Server Slow-Check Broadcom NIC KB

Windows 10 Creators Update’s Virtual Machine Quick Create and Memory Allocation Changes in Hyper-V

One of the new features introduced in Build 15002 of Windows 10 is the ability to quickly create virtual machines using an enhanced Hyper-V Manager feature.

Still a work in progress, Microsoft says…

Hyper-V Manager has a new page that makes it faster and easier to create virtual machines. Just open Hyper-V Manager and click ‘Quick Create’. The simple wizard will help you set up a VM in just a few clicks. After the virtual machine has been created, the confirmation page will allow you to connect to it or enter advanced settings.

Additionally, Build 15002 changes how memory is allocated for virtual machines. With these changes, Hyper-V starts allocating memory in small chunks (to give the operating system a chance to trim memory from other applications) and will use all available memory (no root reserve). 


Looking for an awesome, no-nonsense technical conference for IT Pros, Devs, and DevOps? IT/Dev Connections kicks off in San Francisco in 2017!

IT/Dev Connections

New Feature for Hyper-V

By Garth Jones

This blog post has been a long time coming. It started with an open letter to Ben Armstrong. Ben’s one of the Lead Program Managers on the Microsoft Hyper-V Team. I asked Ben to consider adding a feature that would allow for Virtual Machines (VMs) to be started in a pre-defined order with delays between each VM. I would love to see this in Hyper-V!

I can hear what you’re thinking, “This is a great idea, but doesn’t it already exist in Hyper-V?” No, not what I’m looking for.

You might ask, “What about the delayed start option?”

New Feature for Hyper-V-Delayed Start Option

Yes, that kind of works, but there are several major drawbacks with the delayed start option. Mainly having to:

· Set this option manually for EACH VM (see Row #1 in my example below).

· Track what you have set for each VM.

· Manually create a delay before starting each VM by adjusting each VM’s start-up delay.

· Edit each affected VM to change the start-up order of VMs.

· And, for VMs that were not on when the server shutdown (VM2 in my example below), using the delay start option will result in VM2 not starting at all (see Row #2 below) and therefore leaving a 30 second gap. Ideally, what you see in Row #3 is what should happen.

New Feature for Hyper-V-Example 

I think my suggestion for this new Hyper-V feature would be very helpful because it would help solve the following problems:

1. When a Hyper-V server shuts down, by default all VMs go into saved state simultaneously, so by delaying the shutdown of all VMs, they won’t be competing for resources.

2. When starting up the Hyper-V server, all VMs start at the same time (by default), so by delaying the start-up of all VMs, they won’t be competing for resources.

3. Since disk subsystems are the number one performance issue, any delay of disk IO, will help prevent overloading disk subsystems.

With added benefits of:

1. Allowing the most important VMs to start-up first (if they were on when the server shutdown).

2. Allowing the most important VMs to be shut down last before shutting down the server.

3. Allowing VMs to be shut down manually instead of using the default saved state (this is my preference).

Not only did I write a letter to Ben, but I also wrote a pseudo code for this feature. I spoke to Sean Kearney @energizedtech who is a PowerShell MVP and a long-time friend. During one of our discussions I told him about my idea for this new feature. Sean said that everything could be done with PowerShell, so to help me get started, he sent me several lines of PowerShell code.

Between Sean’s code and my pseudo code, I now had a starting point to create this feature within PowerShell until the Hyper-V team can add it in.

My script is only in its first version and I definitely plan to upgrade it later. What does it do?

Shutdown

· Creates a list comma-separated values (CSV) file of running VMs.

· Shuts down each VM one at a time with a 5-second delay.

· Reboots the server.

Startup

· Reads the CSV file and starts each VM one at a time.

· There’s a 20-second delay between each startup.

· Deletes the CSV file.

For a first version it does the basics, so what’s next? Here’s my list, in no particular order:

· Create a priority list of VMs to shut-down last and to startup first.

· Determine the process to have this script run automatically at server boot-up.

· Determine the process to have this script run automatically on server shutdown.

· Create a better log.

I’m sure there are more features to add, but I haven’t thought of them yet. If you have any ideas to help upgrade my code, let me know by tweeting me @GarthMJ. You can download my PowerShell script via the Enhansoft website.

Why Are My Virtual Machines Slow?

By Garth Jones

The other day, I was working on my test SQL 2016 server and everything was “slow.” Since this was a virtual machine (VM), and we all know that disk I/O (input/output) is the number one killer of VM performance, I quickly looked at Resource Monitor. I saw that the host’s Disk Queue Length was over 50! Normally, it should be a max of 5 (red arrow) when I have all of my test labs up and running at the same time.

Why Are My Virtual Machines Slow-Resource Monitor

Having a disk queue length of 50 told me right away that one or more of my VMs was writing a lot of data to the disk!

Unfortunately, I’ve seen this problem before, so it didn’t take me that long to track down which VM was the culprit.

All of my VMs are set to use Dynamic Memory. This means that they rarely have a multiple of 512 MB of memory assigned to them. I opened Hyper-V Manager and looked for any VMs that were assigned the default startup RAM size of 1024 MB, as this generally indicates a problem.

Why Are My Virtual Machines Slow-cm-cas-rs1

I found that the highlighted VM (cm-cas-rs1) was at fault.

On a side note, I love how cm-cas-rs1 was only using 1% of the CPU for this server. If you looked at only the CPU usage you would think that everything was great! Heck all of my VMs are only using 2% of the CPU on the host server.

Why Are My Virtual Machines Slow-Memory

Also, if you only looked at the memory for the host server, you would see that it had tons of extra RAM. Clearly this is not the problem!

Why Are My Virtual Machines Slow-Dynamic Memory

Next, I confirmed that the VM was set for Dynamic Memory.

Why Are My Virtual Machines Slow-Performance Monitor

Now that I confirmed that Dynamic Memory was enabled, I logged onto the VM and opened Performance Monitor.

I wanted to take a look at the average pages/second, so I added the Pages/sec performance counter. Notice how the average Pages/sec is over 4000! This is a HUGE number! The average should be less than 50.

By the way, if Operations Manager is running, an alert is triggered when this counter is over 50 pages/second for more than 5 minutes.

How do you fix this problem?

You have two options:

1. The easy fix is to reboot the server. Yes, it is that simple, BUT I also found that it could take two or three reboots before things get back to normal.

2. The harder fix is to re-apply the Integration Services for Hyper-V.

Once I rebooted the server, everything went back to normal and my other VMs were no longer “slow.”

I hope you find these tips useful.