Dario IT Solutions Blog

When trying to Live Migrate a virtual machine, the process might fail and inform you that your virtual machine is not compatible with the target node.

If you try to Move (instead of Live Migrate) the virtual machine to another node, the process will again fail and you the following event will be logged:

Log Name:      System
Source:        Microsoft-Windows-FailoverClustering
Event ID:      1205

The Cluster service failed to bring clustered service or application ‘xxxxx’ completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered service or application.

This situation is indeed caused, as the error indicated, by different processor capabilities. Check the exact processor version of your Nodes. You can use msinfo32 to get a detailed view. Here’s, for example, the types of CPUs I see:

Node1

Node2

What happens is that when the virtual machine starts, the hypervisor exposes certain CPU capabilities to the virtual machine. When Live Migrating or Moving the VM to a different host, the VM isn’t aware of the hardware change, tries to use capabilities that it’s new CPU doesn’t support, and fails to start.

Luckily, the solution is simple. Open the VM’s settings, and under the CPU settings check the option named “Migrate to a physical computer with a different processor version".

Note that the VM has to be off in order for you to check this option.

When checking this option, the hypervisor is only exposing the VM to the features of the processor that are available on all versions of a virtualization-capable processor by the same processor manufacturer. Note that this option doesn’t allow you to migrate between AMD and Intel CPUs. Check this Microsoft Technet article for further details: http://technet.microsoft.com/en-us/library/dd446679(WS.10).aspx

It is important to read between the lines in this case. Although you will be able to migrate without errors, your VM does LOSE PERFORMANCE .

This means that when planning to build a new Hyper-V Cluster, make sure that you get exactly the same CPUs on your physical hosts. Otherwise, you will have to downgrade the CPU capabilities of your VMs.

Hope this is helpful.

Kobi

I know its a bit late, but here is the presentation I gave on November 3rd at the Microsoft convention – Let’s Talk Business-Tech that was held at Tel Aviv.

During the presentation, my co-presenter Ronen Gabay and myself talked about the new features of Failover Clustering in Windows 2008 R2.

We talked about Geo-Clusters, PowerShell 2.0 extensive support, Cluster Shared Volumes and mainly about how simple it is to create and manage complex cluster (everyone can do it! believe me! ).

We finished the presentation with a nice demo of RDS 7.0 media enhancement when we showed a 1080P HD movie (Terminators 2 Trailer) over RDP 7.0 while doing a Live Migration of the RDS Session Host virtual server that we RDPed into.. very nice!

What do you think, will we start seeing more clusters from now on? or something is still missing?

I’ll love to get your input.

You can now download VMM 2008 R2 Beta from https://connect.microsoft.com.

I’ve already tested it with Live Migration and it work great! (we had a demonstration for it at a Municipality Convention in Jerusalem recently) so take the time to play with it a little.

What’s New in VMM 2008 R2 Beta

System Center Virtual Machine Manager 2008 (VMM) is a comprehensive management solution for managing virtualized infrastructure running on Windows Server 2008 with Hyper-V, Virtual Server 2005 R2 and VMware ESX through Virtual Center.  Recently, Windows Server 2008 R2 Beta was released which included significant feature improvements to Hyper-V-the underlying hypervisor platform.  A corresponding beta version of VMM R2 – the next version of VMM – is due for release shortly.  VMM R2 Beta  leverages the new platform enhancements and extends the feature set of VMM 2008. This overview highlights the most important new and significantly enhanced features in the VMM 2008 R2 Beta:

Support for new features of Windows Server 2008 R2 Beta

• Live Migration: – Seen through the VMM console, this enables administrators to move virtual machines from one machine in a virtual host cluster to another with no downtime. This allows administrators greater flexibility in responding to planned or unplanned downtime, provides higher machine availability and more robust fault tolerance within virtualized infrastructure. The basic requirements for Live Migration are that all hosts must be part of a cluster and host processors must be from the same manufacturer. Additionally all hosts in the cluster must have access to shared storage. No changes are required to existing virtual machines, network, or storage devices in moving from Quick Migration to Live Migration other than upgrading to beta versions of Windows Server 2008 R2 and VMM 2008 R2.
• Hot addition/removal of VHDs: Allows the addition and removal of new virtual hard disks (VHDs) on a running virtual machine. This enables storage growth in virtual machines without downtime. Additionally, ‘live" VHD management allows administrators to take advantage of additional backup scenarios and readily use mission critical and storage-intense applications (eg: SQL Server and Exchange).
• New optimized networking technologies: VMM 2008 R2 Beta supports two new networking technologies – Virtual Machine Queue (VMQ) and TCP Chimney – providing increased network performance while demanding less CPU burden. NICS that support VMQ, create a unique virtual network queue for each virtual machine on a host that can pass network packets directly from the hypervisor to virtual machine. This speeds throughput as it bypasses much of the processing normally required by the virtualization stack. With TCP Chimney, TCP/IP traffic can be offloaded to a physical NIC on the host computer reducing CPU load and improving network performance.

Enhanced storage and cluster support

• Clustered Shared Volumes (CSV): Provides a single, consistent storage space that allows virtual hosts in a cluster to concurrently access virtual machine files on a single shared logical unit number (LUN). CSV eliminates the previous one LUN per virtual machine restriction and coordinates the use of storage with much greater efficiency and higher performance. CSV enables the Live Migration of virtual machines in and out of the shared LUN without impacting other virtual machines. Enabling CSV on failover clusters is straightforward and easy to monitor through the VMM administrator’s console; many storage configuration complexities prior to CSV have been eliminated.
• SAN migration into and out of clustered hosts: This allows virtual machines to migrate into and out of clustered hosts using a SAN transfer, which automatically configures the cluster nodes to recognize and support the new workload.
• Expanded Support for iSCSI SANs: Previously, only one LUN could be bound to a single iSCSI target whereas now — with VMM 2008 R2 Beta — multiple LUNS can be mapped to a single iSCSI target. This provides broader industry support for iSCSI SANs allowing customers more flexibility in choosing storage providers and iSCSI SAN options.

Streamlined process for managing host upgrades:

• Maintenance Mode: Allows administrators to apply updates or perform maintenance on a host server by safely evacuating all virtual machines to other hosts on a cluster using Live Migration or putting those workloads into a saved state to be safely reactivated when maintenance or upgrades are complete. Maintenance mode is enabled for all supported hypervisor platforms on Windows Server 2008 R2 Beta.

Other VMM 2008 R2 Beta enhancements

• Support of disjoint domains: Reduces the complexity of reconciling host servers with differing domain names in Active Directory and DNS. In these situations, VMM 2008 R2 Beta automatically creates a custom service principal name (SPN) configured in both AD and DNS allowing for successful authentication.
• Use of defined port groups with VMware Virtual Center: On installation, VMM 2008 R2 Beta will present available port groups for VMM’s use with VMware Virtual Center thus allowing administrators to maintain control over which port groups are used.

Windows 2008 R2 will feature some major DFS improvements, following is an explanation of them:

1. Support for Windows Failover Clusters

In Windows Server 2008 R2, Windows Failover clusters can be configured to be part of a replication group. Windows Failover clustering technology enables administrators to configure services and applications to be highly available.

Busy hub servers located in the datacenter that replicate with many branch office servers are perfect candidates for clustered DFS Replication. These servers are critical to the replication infrastructure and administrators expect high availability from these servers. A failure (hardware/software) on such crucial servers has the potential to bring all replication activity to a standstill.

2. Read-only Replicated Folders

Often, customers use the DFS Replication service to publish data from a central server out to many branch office servers. A typical characteristic of this data is that it is created/modified at one location (typically the hub/datacenter server) and changes aren’t expected to occur on any of the other member servers. Usually, administrators configure strict ACLs for the replicated data to ensure that changes aren’t made by end-users in branch offices.

Configuring and maintaining strict ACLs to block accidental modifications or recovering data that has been accidentally deleted entail high administrative overheads. A new feature in Windows Server 2008 R2 called ‘Read-only replicated folders’, offers an easy to manage solution to this problem.

In essence, on a read-only replica:

• Local modifications are blocked by the DFS Replication service. Changes to files/folders including creation, deletion, modification of attributes/permissions etc. are not possible. Semantically, the read-only replicated folder mimics a read-only share.

• Changes from members hosting read-write copies are replicated in. The DFS Replication service replicates in changes from other replication partners that host read-write enabled copies of the replicated folder. This ensures that the data remains up to date on the read-only replica.

3. SYSVOL on Read-only Domain Controllers

Windows Server 2008 introduced support in the DFS Replication service for Read-only Domain Controllers (RODC).

Building on this implementation, in Windows Server 2008 R2 the concept of read-only replicated folders has been extended to the SYSVOL replicated folder. This means that on read-only domain controllers running Windows Server 2008 R2 and using the DFS Replication service to synchronize the SYSVOL share, the SYSVOL share is configured as a read-only replicated folder. Therefore, there is a change in the end-user experience for the SYSVOL share exposed by a Windows Server 2008 R2 domain controller.

On Windows Server 2008 based RODC: Changes can be made to the contents of the SYSVOL share on RODCs. However, the DFS Replication service monitors these changes and then asynchronously overwrites these changes with updates from a writable domain controller. Therefore, any changes made on a Windows Server 2008 RODC will be visible for a short duration, until they are reverted by the DFS Replication service.

On Windows Server 2008 R2 based RODC: Changes cannot be made to the contents of the SYSVOL share on RODCs. Any attempts to make such modifications will encounter an ACCESS DENIED error. Therefore, the SYSVOL share on such RODCs looks like a read-only share.

4. Diagnostics Improvements

Windows Server 2008 R2 also features a set of powerful enhancements to the diagnostics capabilities of the DFS Replication service. These are in the form of new command line options to the dfsrdiag.exe command line diagnostics tool.

Dfsrdiag.exe ReplState : This command line switch provides an insight into the current working state of the DFS Replication service. This command initiates a snapshot of the internal state of the service and thereby gathers a list of the updates that are currently being processed (downloaded from or served out to replication partners) by the service.

Using this command line switch, an administrator can retrieve a snapshot of the status of replication activity across all connections on a given DFS Replication member server.

Dfsrdiag.exe IdRecord: The DFS Replication service maintains a record for each file and folder in the replicated folder in its database. These are known as ID records. This command line switch can be used to display the ID record information maintained by the DFS Replication service corresponding to a particular file/folder in its database. The ID record also contains information such as version vectors of the file/folder, timestamps etc.

Using this command line switch, an administrator can dump the ID record corresponding to a file/folder on each individual replication member server. In order to check whether a particular update has replicated to the member servers in the replication group, the version information in the ID records on these members can be compared.

Dfsrdiag.exe FileHash: This command line switch computes and displays the file hash generated by the DFS Replication service for a particular file. The file hash can be used to compare two files and determine whether they are similar.

For instance, while pre-seeding the contents of a newly added replicated folder, it is often required to verify whether the file being pre-seeded is identical (attributes, timestamps, ACLs etc.) to that on the authoritative copy being used as the source for pre-seeding. By comparing the file hash generated by this command line option for two files, it is possible to verify if the files are identical. If the data being pre-seeded is identical to that contained on the source server, the DFS Replication service will complete initial sync much faster. This is because it does not need to download the file data and merely downloads metadata.

Microsoft’s Virtualization UA team has put together some great Hyper-V clustering guidance on TechNet. 

Content includes:

1. Design for a Failover Cluster in Which All Nodes Run Hyper-V – http://go.microsoft.com/fwlink/?LinkId=129117
2. Requirements and Recommendations for Failover Clusters in Which All Nodes Run Hyper-V – http://go.microsoft.com/fwlink/?LinkId=129110
3. Checklist: Failover Cluster in Which the Servers Run Hyper-V – http://go.microsoft.com/fwlink/?LinkId=129123

Enjoy!

SCOM MVP Andy Dominey has compiled a very nice document on installing System Center Operations Manager 2007 with high availability.

Get it from here – SCOM in Highly Available and Distributed Enterprise Environments.

As promised, I’ve uploaded the presentations from the sessions I gave on December 18th at Microsoft Israel. Enjoy.

Issue:
Trend Micro OfficeScan Version 8.0 SP1 on a server 2008 Failover Cluster may cause the cluster service to fail while moving a resource group. 

Cause:
Current TDI Driver of the OfficeScan causes a loss of connectivity between the cluster nodes, and may cause the cluster service on one of the nodes to fail. The current version of  office scan doesn’t support Windows Server 2008 Clusters yet. It should be supported in the next version. 

Workaround:
Until the newer version of OfficeScan, a workaround is available for this issue.
Follow these instruction : 

Open Device Manager
Select View > Show Hidden Devices
Expand Non-Plug and play devices
Select Trend Micro TDI Driver  > Properties
Go to Driver tab
Under Current Status press the stop button
Under Startup change the type to Disabled (Uninstalling the driver didn’t work, because it was installed automatically at the next boot.) 

Repeat these steps on both nodes and restart them.

Check out this nice list of new clustering features just around the corner:

New Failover Clustering features in R2:

1. Cluster Shared Volumes (CSV) – A distributed access file system optimized for Hyper-V allowing virtual machines and their disk resources to reside be on any cluster node
2. Live Migration – Move virtual machines between different physical machines while keeping them running and maintaining client connections
3. PowerShell Support – The scripting language of the future which will gradually replace cluster.exe
4. DFS-Replication support – Make the primary member of your replication group highly available so that it always has access to the authoritative copy of the data
5. Remote Desktop / Terminal Services support - Make the Connection Broker highly available ensuring that clients are reconnected to their same session or virtual machine within a server farm
6. Network Prioritization – Plan your internal cluster network for efficiency, giving the fastest network the highest priority for internal traffic (heartbeat, CSV & Live Migration traffic)
7. Read-Only APIs – This security enhancement allows admins to query the state of the cluster, but not perform any actions, making it ideal for first-level triage who “can look, but not touch”
8. ETW logging channels – New Crimson channel makes cluster-wide events easier to manage
9. Performance Monitor – Clustering counters allow the user to monitor the cluster and tweak it for optimal performance
10. Enhanced Validation – Improvements to the functional test tool with additional tests which execute on production cluster nodes to ensure they are running optimally, with best practices deployed
11. Migration – The clustering upgrade path now supports more workloads and configurations and offers detailed information about what will be migrated and any additional steps required by the user

New Network Load Balancing features in R2:

1. Extended Affinity – Ensures clients are reconnected to the same node to retain cached information if they are disconnected
2. PowerShell Support – The new NLB scripting language which will gradually replace nlb.exe and wlbs.exe

I had the opportunity today (16.9.08) to participate in Microsoft’s getVIRTUALnow event, and to present the session “Physical and Virtual Server Management”

I presented my take on data center management, which involves automating management tasks in response to system\application states.

It was impressive to see the number of vendors that were present in the event. Microsoft seem to be quick in building the required ecosystem of partners that will push it’s Virtualization offerings greatly in my opinion.

The main point in my presentation was that when looking at the data center (or IT environment) as a whole, tools like Systems Center Operations Manger and Systems Center Virtual Machine Manager (that has integration built into them from the grounds up) really stand out in the crowd in comparison to tools that are built for a specific environment (like managing only virtual machines).

One of the demonstrations involved Performance Resource Optimization (PRO) which is one of VMM’s top features, as it allows administrators to automate practically any type of response to a given condition, including responding to application, virtualization, system, storage, or network events.

For more demonstrations, take a look at Liran’s post – “Get Virtual Now with Hyper-V Technology“.

Also, You are more then invited to check out a bunch of pictures we took at the event – http://www.facebook.com/album.php?aid=55996&id=15415483905&ref=mf