VI35ICGuideIG

VMware® Education ServicesVMware Infrastructure 3: Install and Configure Instructor Manual ESX Server 3.5 and VirtualCenter 2.5 VMware, Inc. [email protected] ESX Server 3.5 and VirtualCenter 2.5 Part Number EDU-VI3IC-3525-IG-A Instructor Manual All rights reserved. This work and the computer programs to which it relates are the property of, and embody trade secrets and confidential information proprietary to, VMware, Inc., and may not be reproduced, copied, disclosed, transferred, adapted or modified without the express written approval of VMware, Inc. Copyright/Trademark This manual and its accompanying materials copyright © 2008 VMware, Inc. All rights reserved. Printed in U.S.A. This document may not, in whole or in part, be copied, photocopied, reproduced, translated, transmitted, or reduced to any electronic medium or machine-readable form without prior consent, in writing, from VMware, Inc. Copyright © 2008 VMware, Inc. All rights reserved. VMware and the VMware boxes logo are registered trademarks of VMware, Inc. MultipleWorlds, GSX Server, and ESX Server are trademarks of VMware, Inc. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. Linux is a registered trademark of Linus Torvalds. All other marks and names mentioned herein may be trademarks of their respective owners. [email protected] CONTENTS MODULE 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 What is Virtual Infrastructure? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 VMware Infrastructure 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Goals of This Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 VCP on VI3 Certification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Objectives for the Learner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Course Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Course Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 MODULE 2 Virtual Infrastructure Overview . . . . . . . . . . . . . . . . . . . . . . . . . 15 What is Virtualization? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 What is a Virtual Machine (VM)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Benefits of a Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 How Does Virtualization Work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Host Operating System-Based Virtualization . . . . . . . . . . . . . . . . . . . . . . . 21 Virtualization Using a Bare-metal Hypervisor . . . . . . . . . . . . . . . . . . . . . . 22 VMware Infrastructure 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 ESX Server Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 ESX Server 3i: Embedded Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 VMware Infrastructure Components (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . 28 VMware Infrastructure Components (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . 29 Using VMware Infrastructure in a Datacenter . . . . . . . . . . . . . . . . . . . . . . 31 Using VDI with VMware Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Using Lab Manager with VMware Infrastructure . . . . . . . . . . . . . . . . . . . 33 VMware Online Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 MODULE 3 ESX Server Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Lesson 1: ESX Server Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 ESX Server Physical Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Hardware Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Partitioning an x86 Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 VMware Infrastructure 3: Install and Configure i Mount Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Partitions Created During Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Launch ESX Server Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Early Installer Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Caution: Watch for Unpartitioned LUNs. . . . . . . . . . . . . . . . . . . . . . . . . . Build Disk Partitions for the Service Console . . . . . . . . . . . . . . . . . . . . . . Recommended Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Advanced Options: Specify Boot Volume . . . . . . . . . . . . . . . . . . . . . . . . . Configure Service Console Networking . . . . . . . . . . . . . . . . . . . . . . . . . . Perform Remaining Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Confirm and Launch the Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . After Installation is Complete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ESX Server Physical Console After Install . . . . . . . . . . . . . . . . . . . . . . . . Download the VI Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VMware Infrastructure (VI) Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VI Client: Host's Configuration Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . License Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Single-Host Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure ESX Server as NTP Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure an ESX Server User Account . . . . . . . . . . . . . . . . . . . . . . . . . . Install Tip #1: Choose Correct NIC for Service Console . . . . . . . . . . . . . Install Tip #2: Watch Out for Unpartitioned LUNs . . . . . . . . . . . . . . . . . . Lab for Lesson 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson 2: ESX Server Troubleshooting Guidelines . . . . . . . . . . . . . . . . . ESX Server Troubleshooting Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . What Happens If ESX Server Crashes? . . . . . . . . . . . . . . . . . . . . . . . . . . What To Do If the ESX Server Crashes . . . . . . . . . . . . . . . . . . . . . . . . . . Collecting Diagnostics Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MODULE 4 42 44 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 77 Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson 1: Create Virtual Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 A Networking Scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Virtual Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 ii VMware Infrastructure 3: Install and Configure Virtual Switch Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Example: One-Box Firewall Environment . . . . . . . . . . . . . . . . . . . . . . . . . 83 Example: A High Performance Application. . . . . . . . . . . . . . . . . . . . . . . . 84 Network Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Connection Type: Service Console Port . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Connection Type: VMkernel Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Connection Type: Virtual Machine Port Group . . . . . . . . . . . . . . . . . . . . . 88 Defining Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Naming Virtual Switches and Connections . . . . . . . . . . . . . . . . . . . . . . . . 90 Mapping vmnics to Physical NICs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Lab for Lesson 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Lesson 2: Modify Virtual Switch Configurations. . . . . . . . . . . . . . . . . . . . 94 Virtual Switch Properties: Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Virtual Switch Properties: Network Adapters . . . . . . . . . . . . . . . . . . . . . . 96 Virtual Switch and Connection Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Connection Policies: VLANs (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Connection Policies: VLANs (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Network Policy: Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Network Policy: Traffic Shaping (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . 102 Network Policy: Traffic Shaping (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . 103 Network Policy: NIC Teaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Load Balancing Method: vSwitch Port-Based (Default) . . . . . . . . . . . . . 106 Load Balancing Method: Source MAC-Based . . . . . . . . . . . . . . . . . . . . . 108 Load Balancing Method: IP-Based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Detecting and Handling Network Failure. . . . . . . . . . . . . . . . . . . . . . . . . 111 Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Module Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 MODULE 5 Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Lesson 1: Fibre Channel San Storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 What is Fibre Channel (FC)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 How is Fibre Channel Used with ESX Server? . . . . . . . . . . . . . . . . . . . . 120 Fibre Channel SAN Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Addressing and Access Control in a Fibre Channel SAN . . . . . . . . . . . . 123 Contents iii Addressing SAN LUNs in the VMkernel . . . . . . . . . . . . . . . . . . . . . . . . Making SAN Storage Available to ESX Server . . . . . . . . . . . . . . . . . . . Where to Find Information on SAN Troubleshooting . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson 2: iSCSI SAN Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What is iSCSI? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How is iSCSI Used with ESX Server? . . . . . . . . . . . . . . . . . . . . . . . . . . Components of an iSCSI SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Addressing in an iSCSI SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How iSCSI Targets are Discovered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How iSCSI Storage Authenticates the ESX Server . . . . . . . . . . . . . . . . . iSCSI Software and Hardware Initiators . . . . . . . . . . . . . . . . . . . . . . . . . iSCSI Software Initiator Network Configuration . . . . . . . . . . . . . . . . . . Enable iSCSI Traffic Through the Service Console Firewall . . . . . . . . . Configure the iSCSI Software Initiator . . . . . . . . . . . . . . . . . . . . . . . . . . Configure Software Initiator: General Properties (1 of 2) . . . . . . . . . . . . Configure Software Initiator: General Properties (2 of 2) . . . . . . . . . . . . Configure Software Initiator: Dynamic Discovery . . . . . . . . . . . . . . . . . Configure Software Initiator: CHAP Authentication . . . . . . . . . . . . . . . Discover iSCSI LUNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problem: Cannot Access iSCSI Storage . . . . . . . . . . . . . . . . . . . . . . . . . Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson 3: VMFS Datastores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What is a VMFS? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating a VMFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VMFS Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Extend a VMFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Add Extent Candidate to VMFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VMFS Extent List Updated. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multipathing with Fibre Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multipathing with iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manage Multiple Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Labs for Lesson 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 127 128 129 130 131 132 133 134 136 138 139 140 141 142 143 144 145 146 147 148 151 152 153 154 156 157 158 159 161 162 163 164 166 167 168 iv VMware Infrastructure 3: Install and Configure Lesson 4: NAS Storage and NFS Datastores . . . . . . . . . . . . . . . . . . . . . . 169 What is NAS and NFS? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 How is NAS/NFS Used with ESX Server? . . . . . . . . . . . . . . . . . . . . . . . 171 NFS Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Addressing and Access Control with NFS . . . . . . . . . . . . . . . . . . . . . . . . 173 Configure Networking for an NFS Datastore . . . . . . . . . . . . . . . . . . . . . . 174 Configure an NFS Datastore (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Configure an NFS Datastore (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 Problem: Cannot Access NFS Datastore . . . . . . . . . . . . . . . . . . . . . . . . . 177 Lab for Lesson 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 MODULE 6 Virtual Center Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Lesson 1: VirtualCenter Software Installation . . . . . . . . . . . . . . . . . . . . . 184 VirtualCenter Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Optional Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 VirtualCenter Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 VirtualCenter Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Order of Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Hardware and Software Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 VirtualCenter Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Calculating the Database Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Database Access Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 VMware License Server (Centralized Licensing) . . . . . . . . . . . . . . . . . . 195 License Server 14-day Grace Period . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 VirtualCenter Server Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 VMware Infrastructure (VI) Client Overview . . . . . . . . . . . . . . . . . . . . . 200 ESX Server and VirtualCenter Communication . . . . . . . . . . . . . . . . . . . . 201 Managing Across Geographies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Backup Strategy for VirtualCenter Server . . . . . . . . . . . . . . . . . . . . . . . . 204 Problem: Cannot Login Using VI Client (1 of 2) . . . . . . . . . . . . . . . . . . . 205 Problem: Cannot Login Using VI Client (2 of 2) . . . . . . . . . . . . . . . . . . . 206 VirtualCenter Server Service Fails To Start . . . . . . . . . . . . . . . . . . . . . . . 207 License Server Will Not Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Lab for Lesson 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Contents v Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211 Lesson 2: VirtualCenter Inventory Hierarchy . . . . . . . . . . . . . . . . . . . . . 212 VirtualCenter Inventory: Multiple Datacenters . . . . . . . . . . . . . . . . . . . . 213 VirtualCenter Inventory: Folders and Subfolders . . . . . . . . . . . . . . . . . . 215 Organizing Objects in the Inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 VirtualCenter Inventory: Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 View VirtualCenter Inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Problem: Cannot Add Host to Inventory (1 of 2) . . . . . . . . . . . . . . . . . . 219 Problem: Cannot Add Host to Inventory (2 of 2) . . . . . . . . . . . . . . . . . . 220 Problem: ESX Server Not Responding . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Lesson 3: Using VirtualCenter to Manage Hosts and VMs . . . . . . . . . . . 224 Lockdown Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Scheduled Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 System Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 Client Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 MODULE 7 Virtual Machine Creation and Management . . . . . . . . . . Lesson 1: Create a VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What is a Virtual Machine (VM)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Files Make Up a Virtual Machine? . . . . . . . . . . . . . . . . . . . . . . . . Display VM's Files Using the VI Client . . . . . . . . . . . . . . . . . . . . . . . . . VM Virtual Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CPU and Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Create a VM-Other Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Machine (VM) Console. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Install Guest OS into VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 239 240 241 243 244 245 246 247 248 249 vi VMware Infrastructure 3: Install and Configure What are the VMware Tools? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 VMware Tools' Configurable Features. . . . . . . . . . . . . . . . . . . . . . . . . . . 252 Lab for Lesson 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 Lesson 2: Create Multiple VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 What is a Template? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Create a Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 Update a Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 View Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 Deploy VM from Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Clone a VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 Guest OS Customization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 Deploying Across Datacenters (1 of 3) . . . . . . . . . . . . . . . . . . . . . . . . . . 264 Deploying Across Datacenters (2 of 3) . . . . . . . . . . . . . . . . . . . . . . . . . . 265 Deploying Across Datacenters (3 of 3) . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Virtual Appliances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 Export VM with OVF Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Import Virtual Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 Lesson 3: VMware Converter Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . 273 VMware Converter Enterprise Capabilities . . . . . . . . . . . . . . . . . . . . . . . 274 VMware Converter Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Installing VMware Converter Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . 276 VMware Converter Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Hot Cloning - Four Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Cold Cloning - Four Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Importing a Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 Cloning Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Changes to Virtual Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Common Converter Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Lab for Lesson 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Lesson 4: Manage VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Move VM Between ESX servers: Cold Migration . . . . . . . . . . . . . . . . . . 290 Snapshot a VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 Modify Virtual Machine Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Contents vii Example 1: Add Raw LUN Access to VM . . . . . . . . . . . . . . . . . . . . . . . 295 Example 2: Add a Virtual NIC to VM . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Example 3: Resize the Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Virtual Machine Properties Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 Options - General Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Options - VMware Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 Options - Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 Options - Advanced . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Advanced - Boot Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 Swapfile Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Labs for Lesson 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Lesson 5: Guided Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 Guided Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Guided Consolidation Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .311 Physical System Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Add to Analysis (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Add to Analysis (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 Set Authentication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Analyze . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 Plan Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 MODULE 8 Virtual Infrastructure Access Control . . . . . . . . . . . . . . . . . Lesson 1: VMware Infrastructure User Access . . . . . . . . . . . . . . . . . . . . Security Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Users and Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pre-defined and Custom Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Permissions Are Applied: Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . How Permissions Are Applied: Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . How Permissions Are Applied: Scenario 3 . . . . . . . . . . . . . . . . . . . . . . . How Permissions Are Applied: Scenario 4 . . . . . . . . . . . . . . . . . . . . . . . 323 324 325 326 327 328 329 330 331 332 333 334 viii VMware Infrastructure 3: Install and Configure VirtualCenter Security Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Default Permissions for VirtualCenter . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 ESX Server Security Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 Default Permissions for ESX Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 Prevent root Access to VI Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 Labs for Lesson 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 Lesson 2: Accessing VMs Using Web Access . . . . . . . . . . . . . . . . . . . . . 342 What is Web Access? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 Log into Web Access (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 Log into Web Access (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Web Access Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 Generate Remote Console URL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 MODULE 9 Resource Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Lesson 1: Using Resource Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 VMs' CPU Resource Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 VMs' Memory Resource Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 How VMs Compete for Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 What is a Resource Pool? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 Configuring a Pool's Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 Viewing Resource Pool Information (1 of 2) . . . . . . . . . . . . . . . . . . . . . . 360 Viewing Resource Pool Information (2 of 2) . . . . . . . . . . . . . . . . . . . . . . 361 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 Resource Pool Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 Resource Pools Example: CPU Shares . . . . . . . . . . . . . . . . . . . . . . . . . . 364 Resource Pools Example: CPU Contention . . . . . . . . . . . . . . . . . . . . . . . 365 Expandable Reservation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 Example of Expandable Reservation (1 of 2) . . . . . . . . . . . . . . . . . . . . . . 367 Example of Expandable Reservation (2 of 2) . . . . . . . . . . . . . . . . . . . . . . 368 Admission Control for CPU and Memory Reservations . . . . . . . . . . . . . 369 Lab for Lesson 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 Contents ix Lesson 2: Migrate VMs with VMotion . . . . . . . . . . . . . . . . . . . . . . . . . . 372 Move VM Between ESX Servers: VMotion Migration . . . . . . . . . . . . . 373 How VMotion Works (1 of 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 How VMotion Works (2 of 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 How VMotion Works (3 of 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 How VMotion Works (4 of 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 How VMotion Works (5 of 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 How VMotion Works (6 of 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 Virtual Machine Requirements for VMotion. . . . . . . . . . . . . . . . . . . . . . 380 Host Requirements for VMotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 CPU Constraints on VMotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 Enable or Disable Nx/xD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Identifying CPU Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Verify VMotion Layout: Use Maps Panel (1 of 2) . . . . . . . . . . . . . . . . . 385 Verify VMotion Layout: Use Maps Panel (2 of 2) . . . . . . . . . . . . . . . . . 386 Verify VMotion Layout: Use Maps Tab . . . . . . . . . . . . . . . . . . . . . . . . . 387 Checking VMotion Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 Problem: VMotion Fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 Lesson 3: VMware DRS (Distributed Resource Scheduler) . . . . . . . . . . 392 What is a DRS Cluster? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Create a DRS Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 DRS Cluster Settings - Automation Level . . . . . . . . . . . . . . . . . . . . . . . 395 DRS Cluster Settings - Migration Threshold . . . . . . . . . . . . . . . . . . . . . 397 DRS Cluster Settings - Placement Constraints . . . . . . . . . . . . . . . . . . . . 398 DRS Cluster Settings - Automation Level per VM . . . . . . . . . . . . . . . . . 400 DRS Cluster Settings - VM Swapfile Location . . . . . . . . . . . . . . . . . . . . 401 Add Hosts to Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 Best Practices for DRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Lab for Lesson 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Lesson 4: Resource Pools in a VMware DRS Cluster . . . . . . . . . . . . . . . 406 Resource Pools in a DRS Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Delegated Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Monitor Cluster Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 Adding Host to DRS Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .411 Planned Downtime: Maintenance Mode . . . . . . . . . . . . . . . . . . . . . . . . . 412 x VMware Infrastructure 3: Install and Configure Problem: Cannot Power on VM (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . 413 Problem: Cannot Power on VM (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . 414 Lab for Lesson 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 MODULE 10 Resource Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Lesson 1: Tools for Resource Optimization . . . . . . . . . . . . . . . . . . . . . . . 420 Systems for Optimizing VM Resource Use . . . . . . . . . . . . . . . . . . . . . . . 421 Virtual CPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 Hardware Execution Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Hyper-Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 VMkernel CPU Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Transparent Memory Page Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 vmmemctl: The Balloon-Driver Mechanism . . . . . . . . . . . . . . . . . . . . . . 427 VMkernel Swap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 Ballooning vs. VMkernel Swapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 Lesson 2: Monitor VM Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431 Performance Tuning Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Monitoring VM Resource Use with Performance Graphs . . . . . . . . . . . . 433 Tools for Improving VMs' CPU and Memory Performance . . . . . . . . . . . 434 Are VMs Being CPU-Constrained? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 Are VMs Being Memory-Constrained? . . . . . . . . . . . . . . . . . . . . . . . . . . 437 Are VMs Being Disk-Constrained? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438 Are VMs Being Network-Constrained? . . . . . . . . . . . . . . . . . . . . . . . . . . 439 Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441 Lesson 3: Monitoring Using Performance-based Alarms. . . . . . . . . . . . . 442 What is an Alarm? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 Creating a VM-Based Alarm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 Creating a Host-Based Alarm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 Alarm Reporting Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Actions to Take When an Alarm is Triggered . . . . . . . . . . . . . . . . . . . . . 447 Using Alarms to Monitor CPU and Memory Usage . . . . . . . . . . . . . . . . 448 Configure VirtualCenter Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . 449 Contents xi Lab for Lesson 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M O D U L E 11 450 451 452 453 Data and Availability Protection . . . . . . . . . . . . . . . . . . . . . . 455 456 457 459 460 462 463 464 465 466 468 469 470 471 472 473 474 475 476 478 479 480 481 482 483 485 Lesson 1: Backup Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What to Back Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . General Guideline for VM Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strategies for VM Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strategies for Service Console Backups . . . . . . . . . . . . . . . . . . . . . . . . . Lesson Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lesson 2: Virtual Machine High Availability . . . . . . . . . . . . . . . . . . . . . Clustering Inside VMs for High Availability . . . . . . . . . . . . . . . . . . . . . What is VMware HA? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VMware HA in Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VMware HA Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VMware HA Host Network Configuration . . . . . . . . . . . . . . . . . . . . . . . Create Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configure Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Failover Capacity Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Add Host to Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Which VMs Should Be Restarted First? . . . . . . . . . . . . . . . . . . . . . . . . . Architecture of a VMware HA Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . What if a Host is Running but Isolated? . . . . . . . . . . . . . . . . . . . . . . . . . Choose Isolation Response per VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troubleshooting VMware HA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lab for Lesson 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MODULE 12 Planning VI Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ESX Server 3 Hardware Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ESX Server Sizing: Core Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . ESX Server Sizing: VM Load Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . ESX Server Sizing: VMkernel Resources . . . . . . . . . . . . . . . . . . . . . . . . Lesson 1: Plan VMware Infrastructure Deployment . . . . . . . . . . . . . . . . 486 487 488 489 490 xii VMware Infrastructure 3: Install and Configure ESX Server Sizing: Service Console Resources . . . . . . . . . . . . . . . . . . . 492 Booting ESX from a Fibre Channel or iSCSI SAN LUN. . . . . . . . . . . . . 493 Example: Booting ESX Server from a Fibre Channel SAN LUN . . . . . . 495 Your VirtualCenter Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 VirtualCenter Inventory Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 Lesson 2: Storage Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499 Storage Comparison-Fibre Channel, NAS, iSCSI . . . . . . . . . . . . . . . . . . 500 ESX Server Feature Comparison by Storage Type . . . . . . . . . . . . . . . . . . 501 Storage Considerations (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 Storage Considerations (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 General SAN Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504 Two Schemes for Locating Virtual Disks . . . . . . . . . . . . . . . . . . . . . . . . . 505 Lesson Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506 Module Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508 Contents xiii xiv VMware Infrastructure 3: Install and Configure MODULE 1 Introduction 1 1 Introduction Virtual Infrastructure 3: Install and Configure v3.5 Importance • This module is intended to set course expectations as well as provide a general overview for this course. Objectives for the Learner • Define Virtual Infrastructure and VMware Infrastructure • Understand the course goals • Understand the course objectives • Get familiar with the course outline VMware Infrastructure 3: Install and Configure 1 COURSE TIMING: This assumes an 8-hour day, which includes 6.5 hours of instruction and 1.5 hours of breaks/lunch per day: DAY ONE: Module 1, Introduction (included student introductions): 30 minutes Module 2, Virtual Infrastructure Overview: 45 minutes BREAK: 15 minutes Module 3, ESX Server Installation • Lesson 1 - ESX Server Installation: 45 minutes • Lab for Lesson 1 - Install ESX Server: 60 minutes • Lab for Lesson 2- Troubleshooting Guidelines: 60 minutes LUNCH: 60 minutes Module 4, Networking • Lesson 1 - Create Virtual Switches: 30 minutes • Lab for Lesson 1 - Create Virtual Switches and Connections: 15 minutes • BREAK: 15 minutes • Lesson 2 - Modify Virtual Switch Configuration: 60 minutes • Lab for Lesson 2 - Networking Scenario: 30 minutes 2 VMware Infrastructure 3: Install and Configure 1 DAY TWO: Introduction Review of Module 4, Lab for Lesson 2 (Networking Scenario): 20 minutes Module 5, Storage • Lesson 1 - Fibre Channel SAN Storage: 25 minutes • Lesson 2 - iSCSI SAN Storage: 30 minutes • Lab for Lesson 2 - iSCSI SAN Storage: 30 minute • BREAK: 15 minutes • Lesson 3 - VMFS Datastore: 25 minutes • Labs for Lesson 3 - Create VMFS Datastore: 20 minutes • Lesson 4 - NAS Storage and NFS Datastores: 30 minutes • Lab for Lesson 4 - Create NFS Datastore: 20 minutes LUNCH: 60 minutes Module 6, VirtualCenter Installation • Lesson 1 - VirtualCenter Software Installation: 30 minutes • Lab for Lesson 1 - Install VirtualCenter: 60 minutes • BREAK: 15 minutes • Lesson 2 - VirtualCenter Inventory Hierarchy: 20 minutes • Lab for Lesson 2 - Add ESX Server to Inventory: 20 minutes Module 7, VM Creation and Management • Lesson 1 - Create a VM: 25 minutes • Lab for Lesson 1 - Create a VM: 45 minutes DAY THREE: Module 7, VM Creation and Management (Continued) • Lesson 2 - Create Multiple VMs: 25 minutes • Lab for Lesson 2 - Template Provisioning: 35 minutes • Lesson 3 - VMware Converter Enterprise: 30 minutes • Lab for Lesson 3 - Use VMware Converter to Create Virtual Machines • Lesson 4- Manage VMs: 20 minutes • BREAK: 15 minutes • Labs for Lesson 4: 35 minutes total • Lesson 5 - Guided Consolidation: 30 minutes Module 8, Virtual Infrastructure (VI) Access Control • Lesson 1 - VMware Infrastructure - User Access: 25 minutes • LUNCH: 60 minutes • Labs for Lesson 1: 45 minutes total Module 1 Introduction 3 • Lesson 2 - Accessing VMs using Web Access: 15 minutes • Activity for Lesson 2 - Log into Web Access and Perform VM Tasks 10 minutes Module 9, VM Resource Management • Lesson 1 - Using Resource Pools: 30 minutes • BREAK: 15 minutes • Lab for Lesson 1 - Create Resource Pools on a Standalone Host: 20 minutes • Lesson 2 - Migrate VMs using VMotion: 30 minutes • Lab for Lesson 2 - VMotion Migrate VMs: 25 minutes • Lesson 3 - VMware DRS: 30 minutes • Lab for Lesson 3 - Create a DRS Cluster: 15 minutes • Lesson 4 - Resource Pools in a DRS Cluster: 25 minutes • Lab for Lesson 4 - Resource Pools and DRS: 20 minutes DAY FOUR: Module 10, VM Resource Monitoring • Lesson 1 - Tools for Resource Optimization: 25 minutes • Lesson 2 - Monitor VM Performance: 25 minutes • Lab for Lesson 2 - Monitor Your Virtual Machines: 40 • BREAK: 15 minutes • Lesson 3 - Monitoring using Performance-based Alarms: 20 minutes • Lab for Lesson 3 - Use Alarms to Detect VM State Changes: 30 Module 11, Data and Availability Protection • Lesson 1 - Backup Strategies: 20 minutes • LUNCH: 60 minutes • Lesson 2 - High Availability Strategies: 30 minutes • Lab for Lesson 2 - Create a VMware HA Cluster: 30 minutes Module 12, Planning VI Deployment • BREAK: 15 minutes • Lesson 1 - Plan ESX Server and VirtualCenter Deployment: 30 minutes • Lesson 2 - Storage Considerations: 15 minutes 4 VMware Infrastructure 3: Install and Configure What is Virtual Infrastructure? 1 Introduction Virtual Infrastructure allows dynamic mapping of compute, storage, and network resources to business applications In traditional datacenters, there is a tight relationship among particular computers, particular disk drives, particular network ports, and the applications they support. VMware's Virtual Infrastructure allows us to break those bonds. Virtual Infrastructure allows the dynamic mapping of compute, storage and network resources to business applications. A famous aspect of VMware's Virtual Infrastructure is the VMotion feature, which allows a running virtual machine to be moved from one server to another with minimal interruption. This is possible because VMware detaches the operating system and its applications from the hardware they run on. But Virtual Infrastructure enables many other flexible management techniques, which we will learn in this course. Module 1 Introduction 5 VMware Infrastructure 3 • A software suite for optimizing and managing IT environments through virtualization • VMware ESX Server or ESX Server 3i • VMware Virtual SMP • VMware High Availability (HA) • VMware VMotion • VMware Distributed Resource Scheduler (DRS) • VMware VMFS • VMware Consolidated Backup (VCB) • VMware Update Manager • VMware Storage VMotion • VMware VirtualCenter • Provisions, monitors and manages a virtualized IT environment VMware Infrastructure 3 is a suite of software for optimizing and managing IT environments through virtualization. In this course, we will focus on installing and configuring VMware ESX Server and VMware VirtualCenter. We will also learn about the following additional features: VMware Virtual SMP, VMware High Availability (HA), VMware VMotion, VMware Distributed Resource Scheduler (DRS) and VMware VMFS. NOTE VMware Consolidated Backup, VMware Update Manager and VMware Storage VMotion are covered in the course, VI3: Deploy, Secure and Analyze. 6 VMware Infrastructure 3: Install and Configure Goals of This Course 1 Introduction • To prepare you to manage your virtual infrastructure environment, with emphasis on ESX Server and VirtualCenter • To prepare you to achieve the status of VMware Certified Professional (VCP on VI3) We plan to teach you how to administer ESX Server and virtual machines using it. We also plan to teach you how to administer VirtualCenter and take advantage of its capabilities to manage ESX Servers and their virtual machines. This course is also required to achieve the status of VMware Certified Professional on VI3. Module 1 Introduction 7 VCP on VI3 Certification • • The VMware Certified Professional (VCP) Program For technical individuals who want to demonstrate their VI expertise and advance their career • Three steps to become a VCP 1. Participate in a VMware authorized course 2. Gain hands-on experience with VMware 3. Enroll and pass the certification exam The VMware Certified Professional Program is designed for any technical individual -- partners, end-users, resellers, and consultants - who wants to demonstrate their expertise in virtual infrastructure and increase his potential for career advancement. Becoming a VMware Certified Professional is a straightforward, three-step process: 1 Participate in a VMware authorized course that is instructor-led to learn best practices and gain hands-on experience. If you are a current VCP, there are no course prerequisites. Gain hands-on experience with VMware. Individuals who do not have the hands on experience find it very difficult to pass the exam. Enroll and pass the certification exam. To register to take the VMware Certified Professional examination please contact Pearson VUE, a thirdparty testing center at www.pearsonvue.com/vmware. 2 3 To best prepare for this course, we recommend using the VI3 exam blueprint as a study guide. The blueprint includes the list of topics covered in the exam as well as references for these topics, such as the VMware product documentation and the VMware Web site. The blueprint is available on the VMware Certification Web page at http://mylearn1.vmware.com/ portals/certification. NOTE This course will give you most of the information you need for the exam, but not everything. Use the VI3 exam blueprint as a reference tool when 8 VMware Infrastructure 3: Install and Configure preparing for the exam. Hands-on experience is also a key component to passing the exam. 1 Introduction Module 1 Introduction 9 Objectives for the Learner • Install ESX Server • Configure networking and storage for the ESX Server • Create VMFS and NFS datastores for the ESX Server • Install and configure VirtualCenter • Deploy virtual machines using templates and VMware Converter Enterprise • Implement virtual infrastructure access policies • Manage virtual machines’ access to resources using shares, resource pools, VMotion, and VMware DRS • Provide virtual machines with high availability using VMware HA • Plan a basic Virtual Infrastructure deployment • Troubleshoot common Virtual Infrastructure problems This course is specifically named "Install and Configure" because the main focus of this course is the tasks of installation and configuration. It is recommended that you get familiar with the topics covered in the VI3:Deploy, Secure and Analyze course. If a student asks about a topic that is not in this course, you can tell him/her if it is covered in the DSA course. This course discusses and demonstrates the benefits of using virtual infrastructure, guidelines for installing and configuring ESX Server and VirtualCenter, components that make up the products; various tasks that can be performed on virtual machines including deploying and migrating VMs, and ways to monitor virtual machine and ESX Server activity. This course also introduces other components of VMware Infrastructure: VMware DRS, VMware HA and VMware Consolidated Backup. • Installation of ESX Server • Configuration of networking, storage, and datastores for the ESX Server • Installation of VirtualCenter • Configuration of virtual machines and installation of the guest OS • Configuration of user access to virtual machines • Configuration of shares, resource pools, VMotion and VMware DRS • Configuration of VMware HA • Troubleshooting common Virtual Infrastructure problems • Planning-related topics are discussed, such as planning your ESX Server deployment, planning your VirtualCenter deployment, and planning your storage. These topics are presented to help users take an initial pass at planning their Virtual Infrastructure environment. • Backup strategies and high availability strategies are discussed in order to get users to start thinking about these important tasks. 10 VMware Infrastructure 3: Install and Configure Course Outline 1. Introduction 2. Virtual Infrastructure Overview 3. ESX Server Installation 4. Networking 5. Storage 6. VirtualCenter Installation 7. VM Creation and Management 8. Virtual Infrastructure Access Control 9. Resource Management 10.Resource Monitoring 11.Data and Availability Protection 12.Planning Virtual Infrastructure Deployment These are the modules presented in the course and are usually presented in sequence. The daily schedule of topics will be covered by your instructor. For course timing, see the Guidance for Instructor note on slide 1. 1 Introduction Module 1 Introduction 11 Course Map Virtual Infrastructure Virtual Infrastructure Overview ESX Server Installation ESX Server Installation Networking Storage Networking ESX Server Installation Storage VirtualCenter Installation VM Creation and Management Operations VI Access Control ESX Server Installation Networking Resource Management Storage Resource Monitoring ESX Server Installation Data and Availability Protection Planning VI Deployment This course's modules fall into two categories: • Modules in the Virtual Infrastructure category discuss system-wide technologies. • Module in the Operations category concerns features related to ongoing day-to-day management of a virtual infrastructure. Each module contains one or more lessons, most of which consist of a lecture and one or more laboratory exercises. This course map, also referred to as the "You Are Here" diagram, will be used throughout the course to indicate our progress. 12 VMware Infrastructure 3: Install and Configure Questions? 1 Introduction Questions? Module 1 Introduction 13 14 VMware Infrastructure 3: Install and Configure MODULE 2 Virtual Infrastructure Overview 2 2 Virtual Infrastructure Overview Importance • Virtualization is a technology that is revolutionizing the computer industry. It is the foundational technology for VMware Infrastructure. This module introduces the topic to students who are new to virtualization. Objectives for the Learner • Understand the concept of virtualization • Identify the differences between host-based virtualization and baremetal hypervisors • Identify the benefits of using a virtual machine VMware Infrastructure 3: Install and Configure 15 What is Virtualization? • Virtualization is a technology that allows you to transform hardware into software • Virtualization allows you to run multiple operating systems simultaneously on a single computer • Each copy of an operating system is installed into a virtual machine • Virtualization is not: • Simulation • Emulation Instructor, to prepare yourself to discuss virtualization, it is recommended to read the white paper, Understanding Full Virtualization, Paravirtualization, and Hardware Assist, available on the VMware Web site at http://www.vmware.com/ files/pdf/ VMware_paravirtualization .pdf, which provides a very good discussion on x86 virtualization. As desktop and server processing capacity has consistently increased year after year, virtualization has proved to be a powerful technology to simplify software development and testing, to enable server consolidation, and to enhance datacenter agility and business continuity. As it turns out, fully abstracting the operating system and applications from the hardware and encapsulating them into portable virtual machines has enabled virtual infrastructure features simply not possible with hardware alone. For example, servers can now run in extremely fault tolerant configurations on virtual infrastructure 24x7x365 with no downtime needed for backups or hardware maintenance. Virtualization is an architecture that allows you to run multiple operating systems simultaneously on a single computer. Each copy of an operating system is installed into its own virtual machine. Virtualization is often confused with simulation and emulation. It is neither of these things. Simulation is something that looks like something else. A Flight Simulator is a well known example in common usage. This is a machine (or a computer program) that can make it look like you are flying a plane. Virtualization is not simulation. The actual operating system is installed on the virtualized hardware. Emulations requires software to translate commands for the emulated hardware into commands the real hardware can understand. This translation process is slow, and usually causes software packages running inside an emulator to run slowly. Also emulation packages can sometimes fail to 16 VMware Infrastructure 3: Install and Configure translate correctly some of the machine-language commands. Virtualization is not emulation. No command translations are taking place when using VMware virtualization products. Additional information on simulation and emulation: A good simulation example in the computer industry is a Cisco IOS simulator. The Cisco corporation developed a special standardized operating system called IOS to run its network routers, switches, and firewalls. Some software-based training tools for Cisco come with an IOS simulator. The student starts the simulator and is able to enter IOS commands. The simulator looks like a network router or switch running IOS. But the Cisco IOS operating system is not really running on the computer. Emulation is software that acts like something else. Usually emulators are hardwareemulators. This is where a program running on one type of hardware emulates some other hardware in order to enable you to run a specific program that requires that hardware. A well known example is a game console emulator. This could enable you to run game software designed for a specific hardware game system (such as X-Box, Nintendo, Playstation, etc.) on an Intel-PC. Another type of emulator is where computer hardware is emulated. This could allow you to run an operating system that requires different hardware than what is installed in your computer. For example the MAC Operating System normally requires special computer hardware manufactured by the Apple corporation. An emulator could allow you to run the MAC OS on an Intel-based PC. A different kind of emulator could allow you to run a copy of Windows on a MAC. Other emulators exist for SPARC and PowerPC hardware. 2 Virtual Infrastructure Overview Module 2 Virtual Infrastructure Overview 17 What is a Virtual Machine (VM)? • A software platform that, like a physical computer, runs an operating system and applications • A discrete set of files, the main files being • Configuration file • Virtual disk file • File for NVRAM settings • Log file Virtual Machine Cover this slide briefly. Do not get into a discussion of virtual machine’s files at this time. Defer to module 6. A virtual machine is a software platform that, like a physical computer, runs an operating system and applications. An operating system that has been virtualized is called a “guest” operating system. One supported guest operating system runs in each virtual machine that is created. Each virtual machine is completely independent and can have its own applications and its own security. From the perspective of the ESX Server, a virtual machine is a discrete set of files, including a configuration file, virtual disk files, a file for NVRAM settings, and a log file. Virtual machines are portable. They can be easily backed up and easily cloned. They are just an encapsulated set of files. NOTE Virtual machines will be discussed in detail in Module 6, VM Creation and Management. 18 VMware Infrastructure 3: Install and Configure Benefits of a Virtual Machine Physical Machine • Difficult to move or copy • Bound to a specific set of hardware • Often has short life cycle • Requires personal contact to upgrade hardware • Difficult to manage remotely Virtual Machine 2 • Easy to move and copy • Encapsulated into files • Independent of physical hardware Virtual Infrastructure Overview • Easy to manage • Isolated from other virtual machines running on the same physical hardware • Insulated from physical-hardware changes In a physical machine the operating system (Windows, UNIX, Linux, etc.) is installed directly on the hardware. This requires specific device drivers to support specific hardware. If the computer is upgraded with new hardware new device drivers are required. Actual hardware upgrades also require direct hands-on contact by tech support personnel. Also, physical machines are often difficult to manage remotely. Virtual machines though are 100% software. The VM is nothing more that a set of files. This includes files known as virtual disks that replace hard disk storage. All of the files for a single VM are located in one directory. Since it uses standardized virtual device-drivers the hardware can be upgraded without any change to the virtual machine. Multiple VMs are isolated from each other. So now you can have your database server and your email server running on the same physical computer. The isolation between the virtual machines means that software dependency conflicts and performance tuning conflicts are not a problem. Since a virtual machine is just a set of files it is very simple to move the entire VM to a new server in order to perform hardware upgrades. This also makes disaster recovery planning and testing much easier. Module 2 Virtual Infrastructure Overview 19 How Does Virtualization Work? • Allows multiple operating system instances to run concurrently within virtual machines on a single computer • A virtualization layer is installed, which uses either a hosted or hypervisor architecture The term virtualization broadly describes the separation of a service request from the underlying physical delivery of that service. With x86 computer virtualization, a virtualization layer is added between the hardware and operating system. This virtualization layer allows multiple operating system instances to run concurrently within virtual machines on a single computer, dynamically partitioning and sharing the available physical resources such as CPU, storage, memory and I/O devices. A virtualization layer or platform is installed on the physical server. For industry standard x86 systems, virtualization approaches use either a hosted or a hypervisor architecture. 20 VMware Infrastructure 3: Install and Configure Host Operating System-Based Virtualization 2 Virtual Infrastructure Overview A host-based virtualization system requires an operating system (such as Windows or Linux) to be installed on the computer A hosted architecture, or host operating-system (host-based) virtualization, installs and runs the virtualization layer as an application on top of an operating system and supports the broadest range of hardware configurations. For example, VMware Server is a free application that can be installed on a supported Windows or Linux system and provides host-based virtualization. Once VMware Server is installed, virtual machines can be created and employed. Other VMware applications that employ a hosted architecture are VMware Player, ACE, and Workstation. Module 2 Virtual Infrastructure Overview 21 Virtualization Using a Bare-metal Hypervisor A bare-metal hypervisor system does not require an operating system. The hypervisor is the operating system. In contrast, a hypervisor (bare-metal) architecture installs the virtualization layer directly on a clean x86-based system. Since it has direct access to the hardware resources rather than going through an operating system, a hypervisor is more efficient than a hosted architecture and delivers greater scalability, robustness and performance. A hypervisor is the primary component of virtualization that enables basic computer system partitioning (i.e., simple partitioning of CPU, Memory and I/O). ESX Server employs a hypervisor architecture on certified hardware for datacenter class performance. NOTE For a very good discussion on virtualization, consult the white paper, Understanding Full Virtualization, Paravirtualization, and Hardware Assist, available on the VMware Web site at http://www.vmware.com/files/pdf/ VMware_paravirtualization.pdf. 22 VMware Infrastructure 3: Install and Configure VMware Infrastructure 3 • A software suite for optimizing and managing IT environments through virtualization • VMware ESX Server or ESX Server 3i • VMware Virtual SMP • VMware High Availability (HA) • VMware VMotion • VMware Distributed Resource Scheduler (DRS) • VMware VMFS • VMware Consolidated Backup (VCB) • VMware Update Manager • VMware Storage VMotion Virtual Infrastructure Overview 2 • VMware VirtualCenter • Provisions, monitors and manages a virtualized IT environment VMware Infrastructure 3 is VMware’s product family designed for building and managing virtual infrastructures. It is a suite of software that provides virtualization, management, resource optimization, application availability and operational capabilities. VMware Infrastructure 3 consists of the following products: • VMware ESX Server 3 and ESX Server 3i: Bare metal hypervisors that partition physical servers in multiple virtual machines. They form the foundation of the VMware Infrastructure 3 suite • VMware Virtual SMP: Multi-processor support (up to 4) for virtual machines • VMware High Availability (HA): VirtualCenter's high availability feature for virtual machines • VMware Distributed Resource Scheduler (DRS): VirtualCenter's feature for dynamic balancing and allocation of resources for virtual machines • VMware VMotion: Migration of virtual machines while they are powered on • VMware VMFS: Technology unique to VMware, it is the default storage system for virtual machine files on physical SCSI disks and partitions • VMware Consolidated Backup (VCB): Centralized backup software for virtual machines • VMware Update Manager: Manage tracking and patching of ESX ServerESX Servers, as well as select Windows and Linux virtual machines. The VMware Web site lists the VMware Infrastructure 3 products in this way. Even though VirtualCenter is not part of the VMware Infrastructure 3 software suite, it is a key component of the Virtual Infrastructure. Also note what is NOT covered in the course. See the Note below. Module 2 Virtual Infrastructure Overview 23 • VMware Storage VMotion: Migration of virtual machines while they are powered on and relocating virtual machine disk files between and across shared storage locations. VMware VirtualCenter is a centralized management tool for ESX Servers and virtual machines. VirtualCenter lets you provision virtual machines, monitor performance of ESX Servers and virtual machines, optimizes resources, and ensures high availability. NOTE ESX Server 3i is covered in a separate, eLearning course. VMware Consolidated Backup, VMware Update Manager and VMware Storage VMotion are not covered in this course. They are covered in the course, VI3: Deploy, Secure and Analyze. 24 VMware Infrastructure 3: Install and Configure ESX Server Architecture 2 Virtual Infrastructure Overview VMkernel x86 Architecture Under ESX Server, applications running within virtual machines access CPU, memory, disk, and network interfaces without direct access to the underlying hardware. The ESX Server's hypervisor (virtualization layer) is nown as the VMkernel. The VMkernel intercepts these requests and presents them to the physical hardware. The service console supports administrative functions for the ESX Server. The service console is based on a modified version of Red Hat Enterprise Linux 3 (Update 6). Users of ESX Server who use the command line find that Red Hat Linux experience, or experience with other versions of Unixfamily operating systems, can be very helpful to them. The VMkernel always assumes that it is running on top of valid, properly functioning x86 hardware. Hardware failures, such as the failure of any physical CPU, can cause ESX Server to fail. If you are concerned about the reliability of your server hardware, the best approach is to cluster either virtual machines or ESX Servers. High availability strategies are discussed later on in the course. ESX 3 is supported on Intel processors, Xeon and above, or AMD Opteron (32-bit mode) processors. ESX 3 offers support for a number of 64-bit guest operating systems. For the complete list of supported systems for ESX Server, consult the ESX Server 3.x Systems Compatibility Guide, available on the VMware Web site at http://www.vmware.com/support/pubs/vi_pubs.html. Additional Information ==> Does the service console still manage cdrom and floppy for the VM? Management of the cdrom and floppy is done by the VM's user world. In ESX Server 3, the concept of a world is broadened. Now VMware engineers can create general-purpose binaries (that aren't monitors) to run under the VMkernel. This is how mouse, keyboard, CDROM, and floppy are implemented for ESX Server 3. It's a win because there's no longer a bottleneck in the service console. Module 2 Virtual Infrastructure Overview 25 ESX Server 3i: Embedded Hypervisor • Compact, 32MB footprint • Only architecture with no reliance on a general purpose OS • Integration in hardware eliminates installation of ESX • Intuitive wizard driven start up experience dramatically reduces deployment time • Standards-based management of the underlying hardware • Server boot to running virtual machines in minutes • Simplified management • Increased security and reliability Again, if a student asks, this course focuses on installing and configuring ESX Server, not ESX Server 3i. ESX Server 3i is the next generation, thin hypervisor integrated in server hardware: • Hypervisor: on its own, ESX Server 3i offers basic partitioning of server resources. Howerver, it also acts as the foundation for virtual infrastructure software, enabling VMotion, DRS, etc, the keys to the dynamic, automated datacenter • Thin architecture: Small footprint (32 mb) for security, reliability and simplified management • Server integration: Default feature makes deployment easy and fast Additional functionality on top of the hypervisor just requires the right licenses, not any changes to the code itself. No reinstalls and no VMFS changes to go from running a standalone instance of the hypervisor to a full VI3 Enterprise deployment. Management is simplified because no Linux command line skills are required, no user accounts or passwords need to be created and maintained, and no OS security hardening, antivirus, or backup effort is required. Security and reliability is increased because fewer interfaces minimize the attack profile, a locked-down, BIOS-like interface prevents users from 26 VMware Infrastructure 3: Install and Configure running arbitrary code, and there is no dependence on failure-prone harddrives, which enables disk-less servers. Additional information on standalone availability: In addition to OEM-branded/embedded hardware, customers can download a standalone, hard disk-installable version of ESX Server 3i from VMware to install on their own rather than getting it only through the hardware they buy. This standalone version may have a smaller compatibility list as compared to ESX Server 3. ESX Server 3i is an easy way to get started with virtualization for new customers. 2 Virtual Infrastructure Overview Module 2 Virtual Infrastructure Overview 27 VMware Infrastructure Components (1 of 2) This graphic shows the VMware Infrastructure components without VirtualCenter. The next graphic includes VirtualCenter. To run your VMware Infrastructure environment, you need at least the following items: • ESX Server: A virtualization platform used to create the virtual machines as a set of configuration and disk files that together perform all the functions of a physical machine. The server provides bootstrapping, management, and other services that manage your virtual machines. • VI Client: A graphical user interface used to access either an ESX Server or VirtualCenter Server. • Datastore: The storage locations for the virtual machine files specified when creating virtual machines. Datastores hide the idiosyncrasies of various storage options (such as VMFS volumes on local SCSI disks of the server, the Fibre Channel SAN disk arrays, the iSCSI SAN disk arrays, or Network Attached Storage (NAS arrays) and provide a uniform model for various storage products required by virtual machines. • Host Agent: On each managed host, software that collects, communicates, and executes the actions received through the VI Client. It is installed as part of the ESX Server installation. 28 VMware Infrastructure 3: Install and Configure VMware Infrastructure Components (2 of 2) 2 Virtual Infrastructure Overview The purpose of this and the previous graphic is to set the stage of topics to come. This graphic shows the main components, which we discuss in the course. In many environments, an additional component, VirtualCenter, is added to manage the infrastructure: • VirtualCenter Management Server (VirtualCenter Server): The working core of VirtualCenter. VirtualCenter Server is a single Windows Service and is installed to run automatically. As a Windows Service, the VirtualCenter Server runs continuously in the background, performing its monitoring and managing activities even when no VI Clients are connected and even if nobody is logged on to the computer where it resides. It must have network access to all the hosts it manages and be available for network access from any machine where the VI Client is run. • VirtualCenter Database: A persistent storage area for maintaining status of each virtual machine, host and user managed in the VirtualCenter environment. The VirtualCenter database can be remote or local to the VirtualCenter Server machine and is installed and configured during the VirtualCenter installation. If you are accessing your ESX Server directly through a VI Client, and not through a VirtualCenter Server and associated VI Client, you do not use the VirtualCenter database. • VirtualCenter License Server: A server that stores software licenses required for most operations in VirtualCenter and ESX Server, such as powering on a virtual machine. • VirtualCenter Agent: On each managed host, software that collects, communicates and executes the actions received from the VirtualCenter Server. The VirtualCenter Agent is installed the first time any host is added to the VirtualCenter inventory. Module 2 Virtual Infrastructure Overview 29 • Shared Datastores: Datastores can be shared between one or more ESX Servers. This fundamental design allows VirtualCenter features such as VMotion, VMware DRS and VMware HA to function properly. 30 VMware Infrastructure 3: Install and Configure Using VMware Infrastructure in a Datacenter 2 • Create a responsive data center with a virtualized IT infrastructure Virtual Infrastructure Overview VMware Infrastructure is most commonly used in the datacenter. datacenter administrators use VMware Infrastructure for: • Solving the problems of server proliferation (lack of space, power and cooling in server rooms) by replacing single application servers with virtual machines consolidated onto a much smaller number of physical hosts, • Making better use of server hardware by deploying new servers in virtual machines to avoid adding more underutilized servers to the datacenter • Provisioning new servers in virtual machines, which takes minutes compared to days or weeks for provisioning a physical server Now that the students know what the VMware Infrastructure is, let’s give them three examples of how it is used. The first and foremost example is using VMware Infrastructure to virtualize a datacenter. The VMware Infrastructure software components are highlighted with an orange box. Module 2 Virtual Infrastructure Overview 31 Using VDI with VMware Infrastructure The second example is to use VMware Infrastructure to host individual desktops using VDI. VMware Infrastructure is the foundation for the VMware Virtual Desktop infrastructure (VDI). With VDI, companies can host individual desktops inside virtual machines that are running in their datacenter. Users access these desktops remotely from a PC or a thin client using a remote display protocol. Since applications are managed centrally at the corporate datacenter, organizations gain better control over their desktops. Installations, upgrades, patches and backups can be done with more confidence without user intervention. 32 VMware Infrastructure 3: Install and Configure Using Lab Manager with VMware Infrastructure 2 Virtual Infrastructure Overview The third example is to use VMware Infrastructure to support the software lifecycle process using VMware Lab Manager. VMware Infrastructure can be used with VMware Lab Manager to support the software lifecycle process. VMware Lab Manager provides the ability to: • Allocate resources on an as-needed basis instead of maintaining multiple static systems that are only used sporadically. Resources can be pooled and shared between development and test teams for maximum utilization. • Provision new machines nearly instantly. Software developers and QA engineers can fulfill their own provisioning needs, instead of IT doing it for them. • Quickly reproduce software defects and resolve them earlier in the software lifecycle and ensure higher quality software and systems. Module 2 Virtual Infrastructure Overview 33 VMware Online Resources • VMware Technology Network (VMTN) http://www.vmware.com/vmtn • Start a discussion • Access documentation • Access the Knowledge Base • Access technical papers and compatibility guides • Access various communities • VMware Technical Support http://www.vmware.com/support Making full use of VMware technical support resources will save you time and money. The first place to come is VMware's extensive web-based resources. The web site contains troubleshooting tips that are not in the printed manuals; it also contains a constantly updated Knowledge Base. The VMware Technology Network (VMTN) provides tools and knowledge to help VMware users maximize their investment in VMware products and to help them understand what is happening in virtual infrastructure. VMTN provides information about virtualization technology through technical papers, documentation, a knowledge base, discussion forums, user groups and technical newsletters. It also provides virtual appliances, a collection of free, pre-built, pre-configured and ready-to-run software applications, all packaged within virtual machines and available for download to any user. 34 VMware Infrastructure 3: Install and Configure Module Summary • Virtualization is a revolutionary computer technology that allows you to transform hardware into software • Virtual machines are easy to manage, move, and copy because they are encapsulated in a set of files • VMware has both Bare-Metal Hypervisor and HostBased Hypervisor products for virtualization 2 Virtual Infrastructure Overview Module 2 Virtual Infrastructure Overview 35 Questions? Questions? 36 VMware Infrastructure 3: Install and Configure MODULE 3 ESX Server Installation 3 3 ESX Server Installation Importance • ESX Server is the platform on which virtual machines run. It provides the virtual machine with all its CPU, memory, disk and network resources.. Objectives for the Learner • Install ESX Server on a local volume • Use the VMware Infrastructure (VI) Client • Configure single host licensing Module Lessons • ESX Server Installation • ESX Server Troubleshooting Guidelines VMware Infrastructure 3: Install and Configure 37 Lesson 1 ESX Server Installation Lesson Topics • ESX Server disk partitions • ESX Server install procedure • VMware Infrastructure (VI) Client • Single host licensing : 38 VMware Infrastructure 3: Install and Configure ESX Server Physical Setup • Service console and VMkernel components are installed on either local disk or storage network-based disk 3 ESX Server Installation There are some key things to consider when planning an ESX Server installation: • Physical connectivity from the ESX Server to the virtual machine network • Physical connectivity from the ESX Server to the management network • Installing the software components on either local disk or storage network-based disk (such as Fibre Channel or iSCSI) Boot from storage network-based disks are supported. This method can provide a solution in cases where it would provide an easy means of replication for the boot disk using storage network-based utilities and if there is no local disk, such as in a blade server environment. Supported hardware technologies for boot disks include local SCSI, IDE/ATA drive or storage networks. Module 3 ESX Server Installation: ESX Server Installation 39 Hardware Prerequisites • Processor – Two 1500MHz or higher Intel or AMD x86 processors • Memory – 1GB RAM minimum, 256 GB maximum • Networking – 1 or more Ethernet interfaces • 10 Gigabit Ethernet NIC cards are supported • For best performance and security, use separate Ethernet controllers for the service console and the virtual machines • Disk storage • A SCSI adapter, Fibre Channel adapter, iSCSI adapter or internal RAID controller • A SCSI disk, Fibre Channel LUN, iSCSI disk or RAID LUN with unpartitioned space Your server must meet the minimum set of requirements needed to install ESX Server 3: • At least two processors • At least 1 GB of physical memory (RAM) • One or more Ethernet interfaces • A basic SCSI controller, Fibre Channel adapter, iSCSI adapter or internal RAID controller: • A SCSI disk, Fibre Channel LUN, iSCSI LUN or RAID LUN with unpartitioned space: The ESX Server software components take up less than 4 GB of disk space. The remainder of the space can be used to hold virtual machines. Storage of virtual machines is currently not supported on IDE/ATA drives or RAIDs. VMs must be stored on VMFS volumes configured on a SCSI or SATA drive, a SCSI RAID, or a storage network. The vmkcore partition must also be located on a SCSI disk, not an ATA/SATA disk. ESX Server 3 supports installing and booting from: • IDE/ATA disk drive or a serial ATA (SATA) disk drive, provided they are connected through supported controllers. • SCSI disk drives • Storage networks For details on the minimum server hardware requirements for ESX Server, consult the Installation Guide, available on the VMware Web site. For details on supported hardware, consult the Hardware Compatibility Guide, available on the VMware Web site. 40 VMware Infrastructure 3: Install and Configure Partitioning an x86 Disk 3 ESX Server Installation • Wrong • Right With Red Hat Linux, IDE disks can have up to 63 partitions and SCSI disks can have up to 15. x86-based disks can have a maximum of four primary partitions. x86-based operating systems use this partitioning scheme. In order to break the fourpartition limitation, an extended partition can be created. Within the extended partition, logical partitions further subdivide the space. It is recommended that you create an extended partition. Creating the extended partition is done for you by the ESX installer. What if I deliberately make a disk with four partitions that exhaust all the space on the disk surface - is that a wrong thing to do? No, but usually we want the flexibility to make any number of partitions, not just four. Making the fourth partition an extended partition gives us that flexibility with no cost in performance. Module 3 ESX Server Installation: ESX Server Installation 41 Mount Points Physical disk Windows Linux/Unix Disk space consumption in each directory is capped by the size of its partition Unlike the Windows operating system, which uses drive letters to define top-level "root" partitions (such as C:\, D:\ and E:\, all peers of each other), every Linux file system is mounted on a separate directory under root (/). This directory is called a "mount point". Linux file systems are mounted during the boot process to create a single file system hierarchy. 42 VMware Infrastructure 3: Install and Configure The configuration file that manages the mapping of service console file systems to mount points is /etc/fstab. Do not use this slide as the place to discuss the details of the service console file structure. This slide is intended to illustrate the concept of mount points in general, because many students from a Windows background will not be familiar with mount points: • Windows system administrators may not be familiar with the concept of mount points. This term can actually be applied to both Windows and UNIX/Linux systems. • A Windows mount point is used to map a disk or partition to a drive letter. You can map multiple disks to different drive letters. The amount of disk space available in any folder depends on what drive letter you are sitting at. • UNIX and Linux do not have drive letters. So, how do you know what disk you are on? UNIX/Linux uses the concept of mount points. These mount points are used to assemble a tree, basically a unified structure of files and folders that physically reside in different partitions. • In the example in the slide, you have a disk with different partitions, colored orange, green and blue. • On a Windows system, you might map the orange partition to drive letter C, the green partition to drive letter D and the blue partition to drive letter E. • On a UNIX or Linux system, you choose one partition to contain the root file system, in this example, the root file system is in the green partition. Empty folders in the root file system are created to serve as mount points to other partitions. • For example, if you look at the second row of green folders, the 1st folder in that row is a mount point for the orange partition and the 2nd folder in that row is the mount point for the blue partition. • So, if you are sitting in a blue folder, what determines the amount of disk space that that folder will get? Answer: The size of the blue partition determines the maximum amount of disk space. 3 ESX Server Installation Module 3 ESX Server Installation: ESX Server Installation 43 Partitions Created During Installation Mount point /boot / (none) /var/log (available under /vmfs/volumes) (none) Which disk? Main boot Main boot Main boot Main boot Any local or remote Any local or remote Approximate size 100 MB 5 GB 544 MB 2 GB Varies 100 MB Type ext3 ext3 swap ext3 VMFS-3 VMkernel vmkcore Service console Use boot root swap Log files VM’s Files, ISO images core dump Note that VMFS and vmkcore partitions can only live on SCSI disks, not IDE/ATA disks, which service console partitions can reside. The following partitions are required for the installation of an ESX Server: /boot, swap, /, VMFS-3 and vmkcore. The partition, /var/log, is optional. VMware recommends a separate partition for log files to prevent filling up the root (/) file system with large log files. The minimum size is 500 MB, but VMware recommends 2000 MB for the log partition. None of these file systems will be filled completely during installation. We want to each to have free space so that the service console functions properly over its lifespan. The VMFS-3 partition holds a VMware File System (VMFS). A VMFS is a file system that is optimized for storing virtual machines. The VMkernel core dump partition is only used in the event of a serious error inside ESX Server. If ESX Server crashes, it records a post-mortem in this partition so that VMware Support can diagnose the problem. ISO images provide software to virtual machines once they are mapped to the virtual machine’s virtual CD drive. There are three locations for storing ISO Images: VMFS datastore, NFS datastore and the /vmimages directory on the service console. Storing ISO images on a VMFS or NFS datastore allows you to share the ISO images across multiple ESX Servers, as long as the datastore is visible to the ESX Server. Storing ISO images in the service console's /vmimages directory makes images available to that ESX Server only. Furthermore, by default, the /vmimages directory is part of the service console's root file system. If you make /vmimages its own partition, then that is a better alternative. In general, we recommend using a VMFS or NFS datastore to store your ISO images. 44 VMware Infrastructure 3: Install and Configure In addition to /var/log, the /opt directory is also used to hold log files, specifically for the VMware HA product. Therefore, you might consider having a dedicated partition for /opt as well. 3 ESX Server Installation Module 3 ESX Server Installation: ESX Server Installation 45 Launch ESX Server Installation • Launch the installation (default is graphical mode) The ESX installer runs in one of two modes, graphical or text. If no key is pressed within 1 minute, the installation will proceed in graphical mode. Graphical mode is the typical mode to choose, however, installing in text mode can be useful if you are accessing the ESX Server console using a remote management network adapter and the network between the remote console and the ESX Server is slow. 46 VMware Infrastructure 3: Install and Configure Early Installer Steps • CD media test • Choose Install, not Upgrade • Use mouse or TAB to move past welcome screen • Specify your keyboard and mouse type • Accept license agreement The CD Media Test provides an opportunity to validate a downloaded ISO image prior to installing. Upgrade will allow preservation of an existing ESX Server install and maintain all current configuration files and directories. Install will reformat the boot disk and install new software and configuration files. The keyboard and mouse options will permit additional keyboard languages to be identified. Mouse configurations are not a critical setting. After installation, the mouse setting is ignored since X Windows System (Linux' graphical user interface) is not supported from the service console. Acceptance of the license agreement is required to complete installation. 3 ESX Server Installation Module 3 ESX Server Installation: ESX Server Installation 47 Caution: Watch for Unpartitioned LUNs • If any LUNs are unpartitioned (either local or on the SAN), you will receive a pop-up warning for each • Select Yes only to those disks you wish to reinstall The installer will examine all LUNs it can see, not only on the local controller, but out on the storage network as well. If a LUN is not partitioned, a pop-up box for each LUN will prompt you to initialize this LUN (because its partition table was unreadable.) If the warning message appears during the installation, select Yes only to those disks that you wish to reinstall with new file systems and new software. 48 VMware Infrastructure 3: Install and Configure Build Disk Partitions for the Service Console Choose volume 3 ESX Server Installation Resulting pop-up confirmation The Partitioning Options screen presents you with partitioning choices as well as target install locations. The ESX Installer provides a single install routine for both local and storage network-based disks. This makes your choice of the target drive imperative that the appropriate selection is made. The first thing to consider is whether you want the ESX installer to automatically define the partitions and sizes for you or you want to define the partitions yourself. VMware recommends letting the ESX installer automatically define the partitions for you. Therefore, choose "Recommended". All available LUNs, including local and storage network-based, are listed in the drop-down menu. It is important to make sure you correctly identify the target location. If the target location contains existing partitions, then a warning dialog box will prompt you for a confirmation to remove all existing partitions. Each LUN is identified by a device name. In the example above, the device listed as cciss/c0d0 is the local SCSI device. How are local disks and remote, storage network disks identified? • The drop-down list includes a description of the disk, so from that, you can determine if a disk is local or remote. Also, the disks are identified by a Linux-style device name. In the examples presented in these slides, the local disk is an HP Compaq Proliant disk and has the disk name of cciss/c0d0. The remote storage network disks are identified as "sda", "sdb", "sdc", etc. Note, however, that "sda," "sdb", etc., do not always refer to "storage network disks." The "s" does not stand for "storage network." Rather, the naming sequence is something inherited from Linux. The Linux-based install environment enumerates these disks as SCSI disks, so it uses a generic naming convention for them. But other device drivers might enumerate those disks differently. Module 3 ESX Server Installation: ESX Server Installation 49 Recommended Partitions The Partition Disks screen allows you to partition your disk. The slide represents the partition scheme used when choosing the Recommended option. In this example, the partitions will be created on the local hard drive because that is what was chosen in the Partitioning Options screen. Remote disk drives are also shown in the list of hard drives, for example, /dev/sda refers to a storage network-based LUN 50 VMware Infrastructure 3: Install and Configure Advanced Options: Specify Boot Volume • Boot from a local SCSI LUN, Fibre-Channel SAN LUN or iSCSI SAN LUN 3 ESX Server Installation If you must edit the default bootloader configuration, a warning message appears The Advanced Options screen presents choices for specifying the ESX Server bootloader options. Ideally, the bootloader should be placed where the service console partitions reside. It is imperative that this drive match the first boot device as defined in the host machine's BIOS, otherwise the ESX Server will not boot. Additionally, for legacy systems that store the BIOS in the MBR, use the "From a partition" selection. In version 3.0.2, the LBA32 option has been removed from this screen. Module 3 ESX Server Installation: ESX Server Installation 51 Configure Service Console Networking Select the appropriate network interface for management access to the service console. Fill in the necessary TCP/IP parameters for network operations. Although the network interface can be configured to obtain an address from a DHCP server, VMware strongly recommends using a static IP address for access. If the network requires a VLAN ID, enter it in the provided field. If you select "Create a default network for virtual machines", your virtual machines will share a network adapter with the service console, which is not the recommended configuration for optimum security. Since the service console should always be on a separate, private network, this option should never be used except in a test environment. 52 VMware Infrastructure 3: Install and Configure Perform Remaining Steps • Set local time zone 3 ESX Server Installation • Set the root password ESX Server provides three ways to input time zone information based on the selected tab: • Map - Shows a graphical representation of the world in which one can select the most appropriate location for the desired time zone • Location - Displays a text listing of various time zones located throughout the world • UTC Offset - Time values based on the offset hour from Greenwich Mean Time (GMT). There is also the option to automatically compensate for daylight saving time, if appropriate. ESX Server requires a minimum of six characters for the root password. As always, this password should follow your corporate standards' password conventions. It is considered best practice to implemment a password strategy that introduces complexity which might include, mixed case, nonstandard characters, and numeric values. Module 3 ESX Server Installation: ESX Server Installation 53 Confirm and Launch the Installation Emphasize that it is a good idea to make sure that the volume where the Master Boot Record is located matches the volume where the partitions will be created. Before the installer begins the software installation, the installer displays a screen which summarizes all the selections. If changes need to be made, you can always go back. It is always a good idea to scroll through the summarized information and confirm the values before installing. 54 VMware Infrastructure 3: Install and Configure After Installation is Complete 3 Open web browser and point to ESX Server to proceed with configuration ESX Server Installation After the installation is complete, reboot the system by clicking Finish. This screen also states how to connect to the ESX Server once it is installed using any valid browser. You connect to the ESX Server using a URL constructed with either its IP address or host name. Module 3 ESX Server Installation: ESX Server Installation 55 ESX Server Physical Console After Install • The ESX Server is ready for post-installation configuration once this screen appears on the console After rebooting, the status screen appears on the console. To log in, press Alt-F1. To return to the status screen, press Alt-F11. 56 VMware Infrastructure 3: Install and Configure Download the VI Client Point to ESX Server to get to this screen 3 ESX Server Installation Download VI Client to perform configuration Once connected to the ESX Server, this screen provides the ability to download the VMware Infrastructure Client, or the VI Client for short. The VI Client is the primary interface for managing all aspects of the Virtual Infrastructure environment. For example, it allows configuration of the ESX Servers and management of its virtual machines. Also from this screen, it is possible to start Web Access, which can be used to manage the virtual machines created after the ESX Server environment has been established. Module 3 ESX Server Installation: ESX Server Installation 57 VMware Infrastructure (VI) Client • The VI Client is a graphical user interface used to configure the ESX Server and manage its VMs • At the VI Client login screen: • Enter ESX Server fully qualified domain name (FQDN) or IP address • User root • Password for user root The VI Client provides direct access to an ESX Server for configuration and virtual machine management.access. The VI Client is also used to access VirtualCenter to provide management, configuration, and monitoring of all ESX Servers and their virtual machines within the Virtual Infrastructure environment. However, when using the VI Client to connect directly to the ESX Server, no management of VirtualCenter features is possible. For example, you cannot configure and administer VMware DRS or VMware HA. 58 VMware Infrastructure 3: Install and Configure VI Client: Host's Configuration Tab 3 ESX Server Installation The VI Client allows you to configure the ESX Server, such as its hardware and software. In the example above, the VI Client is used to log directly into the ESX Server. The ESX Server is highlighted and its Configuration tab has been selected. Module 3 ESX Server Installation: ESX Server Installation 59 License Sources • License sources: • Evaluation Mode • Serial Number • Centralized License Server • Single host license file There are several ways to license your ESX Server: • Evaluation Mode: This mode is intended for demonstration and evaluation purposes. The software is completely operational immediately after installation, does not require any licensing configuration and provides full functionality for 60 days from the time you install the software. During the 60-day trial, the software notifies you of the time remaining until the evaluation mode expires. After the 60-day trial period expires, unless you obtain licenses for your software, you are no longer able to perform most operations in ESX Server. For example, you cannot power on virtual machines. • Serial Number: The serial number is not used with ESX Server 3; rather, the serial number is used to license the ESX Standalone edition, also known as ESX 3i. • License Server: This is known as centralized license server licensing mode. Licenses are stored on a license server, which makes these licenses available to one or more hosts. • Host License File: This is known as single host licensing mode. License files are stored on individual ESX Servers. These modes also apply to licensing VirtualCenter Server, which will be discussed in a later module. 60 VMware Infrastructure 3: Install and Configure Single-Host Licensing • License files are stored on individual ESX Server hosts • Three license editions: Foundation, Standard and Enterprise 3 ESX Server Installation In general, licensing the ESX Server is required because it enables the ability for virtual machines to be powered on. With single host licensing, a host license file (.lic), a license file must be installed on the ESX Server. This is done from the ESX Server's Configuration tab in the VI Client. There are three ESX Server license editions: • VI Foundation: This edition includes access to the following features: VMFS, Virtual SMP, VirtualCenter Agent, VMware Update Manager and VMware Consolidated Backup. • VI Standard: This edition includes all features in VI Foundation plus the feature, VMware HA. • VI Enterprise: This edition includes all features in VI Standard plus the following features: VMware VMotion, VMware Storage VMotion, and VMware DRS. There is another edition, ESX Server 3i, which includes access to VMFS and Virtual SMP. Single host licenses are installed locally into a file named /etc/vmware/ vmware.lic. VI Foundation, VI Standard and VI Enterprise can be licensed to include either ESX Server or ESX Server 3i. Module 3 ESX Server Installation: ESX Server Installation 61 Configure ESX Server as NTP Client • Synchronize ESX Server time • For accurate performance graphs • For accurate timestamps in log messages • So VMs have a source to synchronize with The Network Time Protocol (NTP) is an Internet-standard protocol used to synchronize computer clock times in a network. It is important that the ESX Server run with the correct time so that performance data can be displayed and interpreted properly, so that accurate timestamps appear on log messages, and so that virtual machines may synchronize their time with the ESX Server. The ESX Server can be set up as an NTP client, which synchronizes its time with specific NTP servers. Select your ESX Server, click its Configuration tab, then select Time Configuration in the Software section. From this screen, you can enable the NTP client software and specify NTP servers to synchronize with. The NTP protocol port is also opened in the service console firewall. You can also specify one or more NTP servers that the ESX Server (in other words, the NTP Client) can synchronize time with. For more information on configuring NTP, refer to the following links: • http://www.ntp.org • http://www.eecis.udel.edu/~mills/ntp/html/accopt.html. 62 VMware Infrastructure 3: Install and Configure Configure an ESX Server User Account • Create an ESX Server user login • For remotely accessing the command line using a secure shell • Use mainly for troubleshooting, if necessary 3 ESX Server Installation Under certain circumstances, it might be necessary to log directly into the ESX Server in order to get to the command line, for example: • To view system information that cannot be viewed by the VI Client • To troubleshoot a problem which cannot be resolved using information in the VI Client One way to log into the ESX Server (service console) is to use the physical machine console. Alternatively, you could log into the ESX Server using a secure shell client, such as PuTTY or SecureCRT. By default, the service console does not allow the root user account to log in using an SSH client. However, it does permit normal user account login access using secure shell. If secure shell is the preferred way of connecting, it is recommended to create a normal user account on the ESX Server to open a secure shell and log in. Therefore, if you need to log into the service console to troubleshoot as user root, you need to: • Log in as a normal user • Use the Linux su - command (switch user command) to change to user root. Accessing the ESX Server from the command line is covered in more detail in the VI3: Deploy, Secure and Analyze course Module 3 ESX Server Installation: ESX Server Installation 63 Install Tip #1: Choose Correct NIC for Service Console • An incorrect NIC selection will prevent remote network management interfaces, such as the VI Client and SSH, from working Physical NIC selected during installation is identified as vmnic0 after installation The service console network connection is used for remote network management of the ESX Server. The VI Client, VirtualCenter Server, and other remote network management connections, such as SSH and the Web Access console, also connect via the service console network connection. By default, the first service console network connection is always named service console. It is always in vSwitch0. This switch always connects to vmnic0. Note that the commands to correct the service console NIC are described in the last part of the Installation Lab. It is not necessary to go through those commands at this time. The vmnic0 label is a logical label that is assigned to whatever NIC you select during ESX Server installation. Just because ESX Server calls a physical NIC vmnic0 does not mean that it is the CORRECT NIC for the service console. If an administrator chooses the wrong physical NIC during ESX Server installation, the virtual switch, vSwith0, will have to be disconnected and reconnected to the correct NIC later. 64 VMware Infrastructure 3: Install and Configure Install Tip #2: Watch Out for Unpartitioned LUNs • The ESX Installer lets you erase SAN LUNs connected to your server! • Zone and mask all SAN LUNs away from this server except those for its use 3 ESX Server Installation Unless you are installing the ESX Server to boot from a Fibre Channel storage network, a best practice is to unplug all Fibre Channel-attached SAN storage from the server. The installer lets you erase any accessible disks, including Fibre Channel SAN LUNs in use by other servers. Be careful not to intialize any LUN that might contain production data. If the ESX Server is connected to the Fibre Channel SAN, make sure that the Fibre Channel SAN is properly zoned and masked. The warning dialog box allows you to make sure that you are initializing the correct drives. Can we always say that any drive named “sd#” (e.g. sda, sdb, sdc, etc...) is always SAN-based? No. If a normal, local SCSI disk (not a RAID controller) were used in the chassis it would also show up as sd#. This is a case where one would simply have to be familiar with their hardware, just like he/she would have to be when selecting a NIC for the service console. Module 3 ESX Server Installation: ESX Server Installation 65 Lab for Lesson 1 • Install ESX Server on Local Volume • In this lab, you will perform the following tasks: •Install the ESX Server software •Access the ESX Server using the VI Client •Display, then modify the ESX Server’s configuration If you are using a VDC Kit, when the students open an iLO session to their ESX Server, it is highly probable that their system will be booted from the ESX 3 install CD and the first screen they will see is the CD Media Test screen. Explain to students that when you boot from the ESX 3 install CD and no key is pressed within one minute at the initial ESX install screen, the ESX installer will automatically enter graphical mode and will end up at the CD Media Test, waiting for you to press a key to continue. During the class preparation of the ESX3 upgrade / ESX3 install, the instructor should warn the students about this issue and stress that each student should review the Installation Configuration Summary page* before proceeding to run the installation. Encourage the students to reset the server using iLO power control features. This way, they can actually pay attention to the RAM/CPU configuration. In addition, the instructor should guide the students to entering the BIOS (even demonstrate on the extra server or a student server) and walking through how to verify/modify the hardware clock is set to UTC for our labs. This will save everyone a LOT of headache in the performance lab. 66 VMware Infrastructure 3: Install and Configure Lesson Summary • The VMkernel allows the virtual machines as well as the service console access to the system’s hardware • The ESX Server can be installed to boot from a local LUN or a remote, iSCSI or Fibre Channel LUN • The VI Client is the graphical user interface used to configure the ESX Server 3 ESX Server Installation Module 3 ESX Server Installation: ESX Server Installation 67 Lesson 2 ESX Server Troubleshooting Guidelines Lesson Topics • ESX Server troubleshooting philosophy • What to do if ESX Server crashes : 68 VMware Infrastructure 3: Install and Configure ESX Server Troubleshooting Philosophy • Most ESX Server problems are caused by • Hardware problems • Misconfigurations 3 • Inadequate planning • Aggressively validate hardware • Test memory for 72 hours before deployment • Install a temporary OS to test hardware • Check installed items against the hardware compatibility guides at http://www.vmware.com/support/pubs/vi_pubs.html Troubleshooting should be a systematic process. If you use logic and your knowledge of what depends on what, you will be able to isolate the problem in a systematic way. Most ESX Server problems are caused by: • Hardware problems - For example, a faulty CPU or a bad memory card • Misconfigurations - For example, the service console’s virtual switch is not mapped to the proper physical NIC; or, storage network LUNs are not visible due to incorrect zoning configuration on the storage network • Inadequate planning- For example, insufficient memory, CPUs, network interfaces, and/or disk space. You can prevent these problems from occurring if you thoroughly validate your hardware, plan for deployment and develop good data-center policies. ESX Server Installation Module 3 ESX Server Installation: ESX Server Troubleshooting Guidelines 69 What Happens If ESX Server Crashes? • If ESX Server cannot continue without risk of data loss, the system halts • Purple Screen Of Death (PSOD) displayed on console • Most common types of PSODs • Machine check exception •A general hardware problem detected by a CPU •VMware Support can help pinpoint the failing subsystem • NMI ECC or Parity Error •Specifically memory failures •VMware Support can help pinpoint the failing bank When you collect diagnostics data using the VI Client or the vm-support program, the VMkernel core dump is collected as well. The information in the PSOD is displayed on the video monitor connected to the ESX Server. This information is also written to the VMkernel core dump partition in binary (non-human-readable) format. When the ESX Server is rebooted, the core dump is written to a core file, which can then be sent to VMware support for further troubleshooting. The most common cause of a VMkernel panic is a hardware problem, whether general or specific. Using unsupported hardware can also cause the VMkernel to panic. The information in the PSOD can help VMware Support pinpoint the cause of the panic. NOTE NMI stands for Non-Maskable Interrupt and ECC stands for Error Correcting Code. 70 VMware Infrastructure 3: Install and Configure What To Do If the ESX Server Crashes • Copy down the screen display, screen-grab it, or take a photo • If the machine had been running in a steady state, with running virtual machines • Check for environmental factors, especially room temperature • Check for detached external devices 3 ESX Server Installation • If the machine had been recently rebooted • Check for hardware configuration changes • Gather information and send to VMware Support • Use the VI Client to export the diagnostics data The VI Client allows you to gather useful system information such as virtual machine configuration files and log files, core files, and the system's configuration files. It also captures the output generated by a number of system commands. These provide information about the system's network configuration, device configuration and file system configuration. Steps on how to collect diagnostics data using the VI Client are found on the next page. Another way to gather diagnostics information is to run a script from the service console command line named vm-support. This script gathers similar diagnostics data as the equivalent function in the VI Client. In either case, it is a good idea to gather diagnostics data very soon after you encounter a problem. Otherwise, entries related to the error may be overwritten or pushed further back into the log. Module 3 ESX Server Installation: ESX Server Troubleshooting Guidelines 71 Collecting Diagnostics Data • VMware technical support might request several files to help resolve your product issues • Use the VI Client to collect diagnostics data The VI Client has an option for exporting all or part of your log file data. To export diagnostic data from the VI Client, select File -> Export -> Export Diagnostic Data. Browse to a folder in which to save the file containing the diagnostic data. The diagnostic data is stored into a folder named VMware-VirtualCentersupport-date@time. The contents of the folder contain: • A folder named viclient-support, which holds all the VI Client’s log files • A file named [email protected], which is a compressed, archive file contain ESX Server diagnostics information. 72 VMware Infrastructure 3: Install and Configure Lesson Summary • Prevent ESX Server problems by using supported hardware and configuring with care • The VI Client provides a way to collect diagnostics data in a form that can be sent to VMware Support for further diagnosis 3 ESX Server Installation Module 3 ESX Server Installation: ESX Server Troubleshooting Guidelines 73 Module Summary • The VMkernel allows the virtual machines as well as the service console access to the system’s hardware • The ESX Server can be installed to boot from a local LUN or a remote, iSCSI or Fibre Channel LUN • The VI Client is the graphical user interface used to configure the ESX Server 74 VMware Infrastructure 3: Install and Configure Questions? 3 Questions? ESX Server Installation Module 3 ESX Server Installation: ESX Server Troubleshooting Guidelines 75 76 VMware Infrastructure 3: Install and Configure MODULE 4 Networking 4 4 Networking Importance • The networking features of ESX Server allow virtual machines to communicate with other virtual machines within the same box and with the outside world, allow the service console to communicate, and allow the VMkernel to take advantage of IP-based storage and VMotion. Objectives for the Learner • Understand the purpose and configuration of virtual switches • Create virtual switches and connections • Understand virtual switch settings and policies Module Lessons • Create Virtual Switches • Modify Virtual Switch Configurations VMware Infrastructure 3: Install and Configure 77 Lesson 1 Create Virtual Switches Lesson Topics • Structure of ESX Server networking • Virtual switches • Virtual switch connection types • Physical connections : 78 VMware Infrastructure 3: Install and Configure A Networking Scenario 4 Networking This is a depiction of part of the networking of an ESX Server system, showing virtual machines and their virtual NICs, the physical NICs of the ESX Server machine, and the external physical network switches and LANs. It is the job of the ESX Server administrator to connect these components together. To do that, we will use a special software construct called virtual switches. Networking topics not currently covered in this course: • Cisco Discovery Protocol (CDP): This release of VMware Infrastructure 3 incorporates support for CDP to help IT administrators better troubleshoot and monitor Cisco-based environments from within VirtualCenter 2.5 and the VI Client. CDP allows VMware Infrastructure administrators to know which Cisco switch port is connected to each virtual switch uplink (that is, each physical NIC). CDP is mentioned in the VI3: Deploy, Secure and Analyze course. • Enhanced VMXNET: Enhanced VMXNET is the next version of VMware's paravirtulized virtual networking device for guest operating systems. Enhanced VMXNET includes several new networking I/O performance improvements including support for TCP Segmentation Offload (TSO) and jumbo frames. • TCP Segmentation Offload (TSO): TCP Segmentation Offload (TSO) improves networking I/O performance by reducing the CPU overhead involved with sending large amounts of TCP traffic. This is mentioned in the VI3: Deploy, Secure and Analyze course. • Jumbo frames: Jumbo frames allow ESX Server 3.5 to send larger frames out onto the physical network. The network must support jumbo frames (end-to-end) for jumbo frames to be effective. This is mentioned in the VI3: Deploy, Secure and Analyze course. NetQueue Support: VMware supports NetQueue, a performance technology that significantly improves performance in 10 Gigabit Ethernet virtual environments. Module 4 Networking: Create Virtual Switches 79 Virtual Switches A virtual switch is a software construct, implemented in the VMkernel, that provides networking connectivity for an ESX Server. Virtual switches allow access to the service console, VM network connectivity and access to IP storage. A virtual switch provides connections for VMs to communicate with each other, whether they are on the same host or different host. The VMkernel connects to a virtual switch in order to access IP Storage. The service console connects to a virtual switch for remote management of the ESX Server. The net effect is that all networking communication, whether it is internal or external to the ESX Server, must be defined through a virtual switch. Virtual switches work at Layer 2 of OSI Model. You cannot have two virtual switches mapped to the same physical NIC, however, you can have two or more physical NICs mapped to the same virtual switch. In the example above, there are five virtual switches, each devoted to a different purpose. From left to right: • A switch with a single outbound adapter. It is being used only by VM1. • An internal-only virtual switch, which allows VMs within a single ESX Server to communicate. VM2 and VM3 can communicate with one another using this switch • A NIC team, which is simply a virtual switch connected to two or more physical NICs. A NIC team provides automatic distribution of packets and failover. 80 VMware Infrastructure 3: Install and Configure • A switch used by the VMkernel for accessing iSCSI or NAS-based storage. • A virtual switch used to give the service console access to a management LAN. Separate IP stacks are configured for the service console and the VMkernel. In other words, each service console port and each VMkernel port must be configured with its own IP address, netmask and gateway. 4 Networking Module 4 Networking: Create Virtual Switches 81 Virtual Switch Characteristics No Adapter 1 Adapter For networking VMs, the service console and the VMkernel to the outside world > 1 Adapter (NIC Teaming) For networking VMs, the service console and the VMkernel to the outside world, with additional load balancing and redundancy For networking between VMs on a single ESX Server Zero collisions Zero collisions on internal traffic Up to 1016 ports per virtual switch Each virtual NIC has its own MAC address In this module, even though we have really good slides, it helps to draw, draw, draw! And encourage your students to do the same. Reinforce the fact that virtual networks are just like physical networks in topology. So the drawing is the same as for physical networks. On the whiteboard build a network for them both inside and outside of the ESX Server. The number of ports associated with any virtual switch is configurable by the administrator. The default number of ports associated with the virtual switch created during the ESX installation is 24. The default number of ports associated with new virtual switches is 56. There is a maximum of 1016 ports per virtual switch. The MAC address of a physical NIC is not used at all. Instead, each VM's virtual NIC has its own MAC address. Why wouldn't you configure your virtual switches to have the maximum number of ports? • Overhead - the more ports you have, the more memory is used. • Application tidiness - this is what we think is good performance for VMs per switch 82 VMware Infrastructure 3: Install and Configure Example: One-Box Firewall Environment • Virtual switch with one outbound adapter acts as a DMZ • Back-end applications are secured behind the firewall using the internalonly switch 4 Networking Here we use a virtual machine with multiple network adapters as a firewall. The protected virtual machine is inaccessible except through the virtual machine firewall. In addition to creating your own firewall, note that there are existing firewall and security VM appliances that are downloadable from VMTN at http://www.vmware.com/vmtn/appliances. Module 4 Networking: Create Virtual Switches 83 Example: A High Performance Application •Automatic, configurable network load distribution network connectivity with automatic failover •Configurable •Redundant active/standby NICs and failover policies This configuration will only give more bandwidth if the out-ip load balancing policy is chosen. A high performance application can benefit from NIC teaming, which provides more bandwidth, automatic network load balancing and network failover. In the default configuration, this virtual machine will have its outbound traffic mapped to only one of the NICs in the team, based on its virtual switch port ID. You can change the configuration so that traffic is spread across all the NICs in the team based on each IP datagram's source and destination IP address. However, your physical switch must be prepared to see traffic from the same MAC address on different physical ports. 84 VMware Infrastructure 3: Install and Configure Network Connections • There are three types of network connections: • Service console port – access to ESX Server management network • VMkernel port – access to VMotion, iSCSI and/or NFS/NAS networks • Virtual machine port group – access to VM networks • More than one connection type can exist on a single virtual switch, or each connection type can exist on its own virtual switch 4 Networking Some students might interpret the slide to mean that you should create just one virtual switch and place the service console, VMkernel ports, and VM port groups on to it. This is entirely valid because you can separate the traffic by creating at least 3 separate VLANs, one for the service console, one or more for the VMkernel ports, and one or more for the VM port groups. However, if you want potentially better performance and better security, place the ports/ port groups onto different virtual switches. This could be useful in certain cases, for example, isolating iSCSI traffic to its own physical network. Before using a virtual switch, one or more connections must be defined. The graphic above shows a single virtual switch with all three connection types defined. When designing your networking environment, you might choose this arrangement, or opt for multiple virtual switches with different combinations of connection type. The choice will depend partially on the layout of your physical networks. A key point to remember is that physical NICs are assigned at the virtual switch level, so all ports and port groups defined for a particular switch will share the same hardware (although which NICs are active can be configured differently for each port group). Module 4 Networking: Create Virtual Switches 85 Connection Type: Service Console Port When creating a service console port, you will define • A network label -- a user-chosen text string identifying the port • An optional VLAN ID • IP settings, either static or dhcp By defining a service console port on a virtual switch with 2 or more outbound adapters, the service console gains the benefits of NIC teaming in the same way that virtual machines do. It is recommended to use a static IP address instead mainly because it prevents the service console from having to rely on an external source for its IP address. If the DHCP server were to go down, then the ESX Server will not be able to connect to the network, which would impact operations. Multiple service console connections can be configured only if they are configured on different networks. In addition, only a single service console gateway IP address can be defined. This slide is a two-step slide build: • 1st screen: Here is a picture of our complete network configuration. • Press Enter. • 2nd screen: Here is the service console port. 86 VMware Infrastructure 3: Install and Configure Connection Type: VMkernel Port 4 Networking Regarding the network labeled "Storage/VMotion LAN": It is technically possible to have both network-based storage traffic and VMotion traffic on the same LAN. However, the best practice is separate the networkbased storage traffic from the VMotion traffic for both security and performance reasons. A VMkernel port allows the use of iSCSI and NAS-based storage by the VMkernel, and is required for VMotion. When creating a VMkernel port, you will define • A network label • An optional VLAN ID • Whether or not to enable the port for VMotion • IP settings Multiple VMkernel connections can be configured only if they are configured on different networks. In addition, only a single VMkernel gateway IP address can be defined. Again, note that separate IP stacks are configured for the service console and the VMkernel. Each needs to be configured with an IP address, netmask and gateway. Module 4 Networking: Create Virtual Switches 87 Connection Type: Virtual Machine Port Group When creating a virtual machine port group, you will define • A network label • An optional VLAN ID IP settings are configured by the guest OS for each virtual NIC configured for a virtual machine. 88 VMware Infrastructure 3: Install and Configure Defining Connections • A connection type is specified when creating a new virtual switch • Parameters for the connection are specified during setup • More connections can be added later • Existing connections can be modified 4 Networking If you have time, demonstrate how to get to the Add Network wizard screen. To create a network connection, use the VI Client. Select your ESX Server in the inventory, then click its Configuration tab. Select the Networking link, then click the Add Networking... link. This displays the Add Networking Wizard, which steps you through adding a network connection. Module 4 Networking: Create Virtual Switches 89 Naming Virtual Switches and Connections • The virtual NIC on the service console is known as vswif0 • All virtual switches are known as vSwitch# • Every port or port group has a network label • Service console ports are labelled Service Console, Service Console 2, etc. Emphasize once more that there are separate IP stacks for the service console and the VMkernel. Every virtual switch is identified by the name vSwitch#, where # is a sequential number, starting with 0. Every port and port group is given a user-defined network label when it is created. If there are multiple service console ports, each service console port is identified by the name vswif#, where # is a sequential number, starting with 0. To get to this screen, use the VI Client. Select your ESX Server from the inventory, then click the Configuration tab. Finally, click the Networking link. 90 VMware Infrastructure 3: Install and Configure Mapping vmnics to Physical NICs • Is service console port connected to the correct network? • To verify mapping between physical NIC and vmnic: View properties of virtual switch, Network Adapters 4 Use the esxcfg-nics command from the service console command line Networking The ESX Server administrator should be familiar with what networks the ESX Server’s physical NICs are connected to, as well as be able to identify them using the correct vmnic#. Associating the correct vmnic with the correct virtual switch will ensure proper network connectivity. One way to display the mapping between a physical NIC and a vmnic# is to use the VI Client. Select your ESX Server, then click its Configuration tab. Click the Networking link in the Hardware section. Next to a virtual switch, click the Properties link. In the Properties dialog box, click the Network Adapters tab. In this display, you will see the vmnic#’s associated with this virtual switch as well as the physical PCI address associated with it. If you have a physical NIC that is not yet assigned to a virtual switch, the VI Client cannot be used to show you the mapping between physical PCI address and vmnic#. To view this information, you must go to service console command line and run the command esxcfg-nics -l. This command provides information about physical NICs recognized by the VMkernel. It lists information such as the vmnic# and its associated PCI address. Module 4 Networking: Create Virtual Switches 91 Lab for Lesson 1 • Create Virtual Switches • In this lab, you will perform the following tasks: •Create an internal-only virtual switch •Create a virtual switch with one physical adapter 92 VMware Infrastructure 3: Install and Configure Lesson Summary • ESX Server uses virtual switches to implement networking • Physical adapters are assigned at the virtual switch level • There are three connection types for virtual switches • service console port • VMkernel port • Virtual machine port group Networking 4 • Multiple connections can be defined on a single virtual switch Module 4 Networking: Create Virtual Switches 93 Lesson 2 Modify Virtual Switch Configurations Lesson Topics • Virtual switch properties • Number of ports • Network adapters including speed and duplex • Security, Traffic Shaping and NIC Teaming Policies • Connection policies • Label and VLAN ID • Security, Traffic Shaping and NIC Teaming Policies : 94 VMware Infrastructure 3: Install and Configure Virtual Switch Properties: Ports • Number of Ports 4 Networking The virtual switch Properties’ General tab allows you to change the number of ports for the entire virtual switch By default, the number of ports for a new virtual switch is 56. There is an exception - the default number of ports for the virtual switch created during the ESX installation process is 24. The maximum number of ports is 1016. The number of ports is configurable. Virtual switch ports are used for virtual machine connections as well as uplinks (physical NICs). Some ports are also used for internal purposes by the VMkernel. To get to this display, use the VI Client. Select your ESX Server in the inventory, then click its Configuration tab. Click the Networking link, then click the Properties... link next to the virtual switch. Module 4 Networking: Modify Virtual Switch Configurations 95 Virtual Switch Properties: Network Adapters • For each physical adapter, speed and duplex can be changed (default is autonegotiate) • Might need to set with certain NIC/switch combinations To change the speed and duplex of a network adapter in any of your virtual switches, use the VI Client. Select your ESX Server from the inventory, then click its Configuration tab. Then, click the Networking link. Click on the Properties... link of the virtual switch that you would like to modify. Select the Network Adapters tab in the Properties window. Click the Edit button to change the speed and duplex. If you are using Gigabit Ethernet adapter, leave it at autonegotiate because it is part of the gigabit standard. If you are using a 10/100 adapter, you might need to manually set speed and duplex settings. These days, Gigabit Ethernet adapters are common, therefore, it is less frequent that we have to modify this setting. Additional Information ==> When we attach a NIC to a virtual switch in the VI Client, the virtual switch properties displays a range of IP addresses. Where does the ESX sever get these from? • The VMkernel learns the IP addresses by snooping the traffic on the network. As various computers send broadcast packets with their own IP address as the sender, the VMkernel remembers them and presents them in this user interface. This is not a security issue. The VMkernel does not need to snoop on all packets, and of course it's connected externally to a physical switch--probably not a hub! So the only packets that come in from the outside world are broadcast packets plus unicast packets that are addressed to some particular VM. The purpose of this behavior is to help administrators get things connected to the right networks. Just as with physical LANs, you have to plug the Ethernet cables in correctly! Sure, the cables are virtual in this case, but it's still important. 96 VMware Infrastructure 3: Install and Configure Virtual Switch and Connection Policies • There are three network policies: • Security • Traffic shaping • NIC teaming • Policies are defined • At the virtual switch level •Default policies for all the ports on the virtual switch 4 • At the port or port group level •Effective policies: Policies defined at this level override the default policies set at the virtual switch level Networking There are three network policies: Security, traffic shaping and NIC teaming. These policies are defined for the entire virtual switch and they can also be defined for the service console port, the VMkernel port or a VM port group. When a policy is defined for an individual port or port group, the policy at this level overrides the default policies defined for the virtual switch. More than one policy can be assigned to a port group. Examples of this are provided in the VI3: Deploy, Secure and Analyze course. Module 4 Networking: Modify Virtual Switch Configurations 97 Connection Policies: VLANs (1 of 2) • Virtual LANs (VLANs) allow the creation of multiple logical LANs within or across physical network segments • VLANs free network administrators from the limitations of physical network configuration • VLANs provide several important benefits • Improved security: the switch only presents frames to those stations in the right VLANs • Improved performance: each VLAN is its own broadcast domain • Lower cost: less hardware required for multiple LANs • ESX Server includes support for IEEE 802.1Q VLAN Tagging Further information on understanding the tag: The last two bytes of the 4byte VLAN tag frame contain 3 802.1p priority bits (that ESX 3 does not use), the Canonical Format Indicator (CFI), and then the 12-bit VLAN ID number. A 12 bit number yields VLAN IDs in the range of 0-4095. VLANs are not a VMware invention, but an entirely independent networking concept, for which ESX Server provides support. VLANs were originally designed to segment switches into multiple collision domains. It was quickly recognized that VLANs could simplify a number of common networking tasks such as moves, adds and changes, and could provide segmentation in an otherwise flat network. Early VLAN solutions were proprietary and distinctly non-standard. The IEEE 802.1Q is the industry standard for VLAN implementation. ESX Server has provided support for IEEE 802.1Q VLANs since version 2.1. VLANs are a network layer 2 concept (the same layer at which MAC addresses and Ethernet live, one layer below IP addressing and routing.) Physical VLAN compatible switches that can keep track of which ports belong to which VLANs are required. In order to extend VLANs across physical switches, a "trunk link" must interconnect the physical switches. Frames on the trunk are encapsulated in the IEEE 802.1Q format and contain an extra four bytes inserted after the source and destination MAC address. In the four-byte 802.1Q tag, the first two bytes (0x8100) are an indicator that the following frame is an 802.1Q frame and the next two bytes contain the 12-bit VLAN ID number. 98 VMware Infrastructure 3: Install and Configure Connection Policies: VLANs (2 of 2) • Virtual switch tagging • Packets from a VM are tagged as they exit the virtual switch • Packets are cleared (untagged) as they return to the VM • Little impact on performance 4 Networking ESX Server provides VLAN support through virtual switch tagging, which is provided by giving a port group a VLAN ID (by default, a VLAN ID is optional.) The VMkernel then takes care of all tagging and untagging as the packets pass through the virtual switch. To define a VLAN ID for a port group, use the VI Client. Select your ESX Server from the inventory, then click its Configuration tab. Click the Networking link, then click the Properties... link next to the virtual switch. Select the port group listed in the Ports tab, then click the Edit button. Enter a VLAN ID in the field provided. A switch port on the physical ESX Server must be defined as a static trunk port. A trunk port is a port on a physical Ethernet switch configured to send and receive packets tagged with a VLAN ID. No VLAN configuration is required in the virtual machine. In fact, the virtual machine does not know it is connected to a VLAN. For more information on how ESX Server has implemented VLANs, consult the white paper, VMware ESX Server 3 802.1Q VLAN Solutions, available on the VMware Web site at http://www.vmware.com/pdf/ esx3_vlan_wp.pdf. Module 4 Networking: Modify Virtual Switch Configurations 99 Network Policy: Security • Administrators can configure Layer 2 Ethernet security options at the virtual switch and at the port groups These options are discussed in detail in the VI3: Deploy, Secure and Analyze course, so do not spend too much time on this slide. Network policies are defined at either the virtual switch level or at the port group level. In the example above, the network security policy is being defined for the port group named Production. The network security policy contains the following exceptions: • Promiscuous Mode: When set to "Reject", placing a guest adapter in promiscuous mode has no effect on which frames are received by the adapter (default is "Reject") • MAC Address Changes: When set to "Reject", if the guest attempts to change the MAC address assigned to the virtual NIC, it stops receiving frames (default is "Accept") • Forged Transmits: When set to "Reject", drop any frames which the guest sends where the source address field contains a MAC address other than the assigned virtual NIC MAC address (default is "Accept") To accept or reject the security policy exceptions, use the VI Client. Select your ESX Server from the inventory, then click its Configuration tab. Click the Networking link, then click the Properties... link next to the virtual switch. Select the port group listed in the Ports tab, then click the Edit button. Click the Security tab and make the desired changes. In general, these policies give you the option of disallowing certain behavior that could compromise security. For example, A hacker might use a promiscuous mode device to capture network traffic for unscrupulous activities. Or someone could impersonate a node and gain unauthorized access by spoofing its MAC address. 100 VMware Infrastructure 3: Install and Configure Set promiscuous mode to Accept if you want to use an application in a virtual machine that sniffs packets, such as a network-based Intrusion Detection System (IDS). Set MAC Address Changes and Forged Transmits to Reject to help protect against certain attacks launched by a rogue guest OS. Leave MAC Address Changes and Forged Transmits at its default value, accept, because it keeps certain guest applications functions if these applications normally change the mapped MAC address, such as some guest OS-based firewalls. A potential scenario for wanting to set these policies would be for a host which has "public exposure", such as a web server. One might be concerned with the potential of its being compromised and subsequently used as a "launching point" for attacks either on other hosts owned/operated by the owner or possibly against other hosts owned by others. By changing the originator information they could either intend to spoof another system into allowing unauthorized access and/or they might wish to not easily bring attention to their intrusion. In general, most people will not change these security options, and leave them at their defaults. The important thing to note about these options, though, is that these options do not exist in the physical world. You cannot control these behaviors on physical machines on the network. For example, if someone has root access to your physical machine, like everyone usually does to their desktop, you cannot stop someone from doing any of these things. And, none of the physical NICs used today allow you to disable these behaviors. But with virtual switches, you can stop the person with administrative control to the machine from performing these types of insecure behavior.The security policy gives administrators a level of control beyond what is usually possible in most physical environments. Why did VMware add these security options to ESX 3? --> These security options were developed as a response to a 3rd party security audit that was performed on ESX 2.x. Neohapsis is the company VMware hired to audit the ESX Server code in the 2.x timeframe. These security options were added due to a weakness that was pointed out by them in the "disallow promiscuous" option implemented in ESX 2.x. The ESX 3 security options here are our response to that audit. 4 Networking Module 4 Networking: Modify Virtual Switch Configurations 101 Network Policy: Traffic Shaping (1 of 2) • Network traffic shaping is a mechanism for controlling a VM’s outbound network bandwidth • Average rate, peak rate, and burst size are configurable A VM's network bandwidth can be controlled by enabling the Network Traffic Shaper. The Network Traffic Shaper shapes outbound network traffic only. To control inbound traffic, use a load-balancing system, or turn on rate-limiting features of your router. 102 VMware Infrastructure 3: Install and Configure Network Policy: Traffic Shaping (2 of 2) • Disabled by default • Shaping parameters apply to each virtual NIC in the virtual switch 4 Networking These options are discussed in detail in the VI3: Deploy, Secure and Analyze course, so do not spend too much time on this slide Although traffic shaping is no longer DEFINED perVM, emphasize to the student that traffic shaping is still APPLIED on a perVM basis. For example, if I set the average bandwidth at 32000 Kbps, then any VM connected to the port group can use an average bandwidth of 32000 Kbps. ESX Server shapes traffic by establishing parameters for three outbound traffic characteristics: average bandwidth, burst size, and peak bandwidth. You can set values for these characteristics through the VI Client, establishing a traffic shaping policy for each uplink adapter. • Average Bandwidth establishes the number of bits per second to allow across the vSwitch averaged over time--the allowed average load. • Peak Bandwidth is the maximum bandwidth the vSwitch can absorb without dropping packets. If traffic exceeds the peak bandwidth you establish, excess packets are queued for later transmission after traffic on the connection has returned to the average and there are enough spare cycles to handle the queued packets. If the queue is full, the packets are dropped. Even if you have spare bandwidth because the connection has been idle, the peak bandwidth parameter limits transmission to no more than peak until traffic returns to the allowed average load. • Burst Size establishes the maximum number of bytes to allow in a burst. If a burst exceeds the burst size parameter, excess packets are queued for later transmission. If the queue is full, the packets are dropped. When you specify values for these two characteristics, you indicate what you expect the vSwitch to handle during normal operation. Average bandwidth and peak bandwidth are specified in Kbps (kilobits per second), and the burst size is specified in KB (kilobytes). Network traffic shaping is off by default. Module 4 Networking: Modify Virtual Switch Configurations 103 In the example above, the network traffic shaping policy is not defined at the virtual switch. Rather, it is being defined for the port group named Production. There is no reason to modify with the traffic shaping parameters unless you know more about the behavior of the application in the VM you're trying to control the bandwidth usage of. If you know from analysis that this VM, every so often, needs to transmit 1 MB of data now and then, you can let it do so faster. Set the burst size to 1 MB, and specify some higher bandwidth. How much higher? Well, what do you want to achieve? You're trying to reserve bandwidth for other VMs, right? So just how much bandwidth are you willing to steal from them now and then? The answer to this question is determined by analysis of actual applications and traffic loads. 104 VMware Infrastructure 3: Install and Configure Network Policy: NIC Teaming • NIC Teaming settings: • Load Balancing (outbound only) • Network Failure Detection • Notify Switches • Rolling Failover • Failover Order 4 Networking NIC teaming policies, which include load balancing and failover settings, allow you to determine how network traffic is distributed between adapters and how to re-route traffic in the event of an adapter failure. Default NIC teaming policies are set for the entire virtual switch. These default settings can be overidden at the port group level. To modify NIC teaming policies of a port group, click your ESX Server's Configuration tab, then click the Networking link. Click the Properties... link next to the virtual switch on which the port group is located. Select the port group in the list of ports, then click the Edit... In the port group properties window, click the NIC Teaming tab. In the example above, the network NIC teaming policy is not defined at the virtual switch. Rather, it is being defined for the port group named Production. Module 4 Networking: Modify Virtual Switch Configurations 105 Load Balancing Method: vSwitch Port-Based (Default) Point out that VMkernel load balancing affects outbound load distribution only. An alternative for presenting this slide is to illustrate the diagram using the whiteboard instead of presenting the slide as is. One reason for this is that students might ask, "Why are the ports not accessed sequentially, why are there some spare ports between the two adapters?" This diagram might work better for you if it is drawn on a whiteboard by developing it while explaining the different mechanisms. As each IP packet leaves its virtual NIC the VMkernel must decide which uplink (physical NIC) will carry that packet to the outside world. The load balancing options are: • Route based on the originating port ID (the default): Choose an uplink based on the virtual port where the traffic entered the virtual switch • Route based on source MAC hash: Choose an uplink based on a hash of the source Ethernet address • Route based on IP hash: Choose an uplink based on a hash of the source and destination IP addresses of each packet The example above shows routing based on the originating port ID, also known as the vSwitch port-based load balancing method. With this method, a virtual machine's outbound traffic is mapped to a specific physical NIC based on ID of the virtual port to which this virtual machine is connected. This method is simple and fast and does not require the VMkernel to examine the frame for necessary information. 106 VMware Infrastructure 3: Install and Configure When the load is distributed in the NIC team using the port-based method, no single-NIC VM will ever get more bandwidth than can be provided by a single physical adapter. This slide builds: • 1st screen: This slide illustrates how the VMkernel uses the default port-based method to balance network traffic across NICs. • Press Enter • Red line represents traffic from VM0, using the first NIC • Press Enter • Green line represents traffic from first VNIC in VM1, using the second NIC • Press Enter • Blue line represents traffic from second VNIC in VM1, using the third NIC 4 • Press Enter • Purple line represents traffic from VM2, using the first NIC • Press Enter • Black line represents traffic from VM3, using the second NIC How does the VMkernel balance the load? I.e. how does it determine what NIC to use to route the VM's packets? • Depending on the policy you choose, a "load balancing value" (for lack of a better term) is calculated. The load balancing value differs based on algorithm. For example, with the Source MAC based algorithm, the load balancing value is the least significant bit (LSB) of the source MAC address in the frame. With the vswitch port-based algorithm, the load balancing value is the source of the port ID. • The VMkernel performs the calculation, which is "load balancing value" MOD "number of NICs that are up" = The NIC to use • For example, let's say you have 4 active NICs (NIC 0, NIC 1, NIC 2 and NIC 3). And let's say that you are using the vswitch port-based algorithm (or any policy for that matter). The VMkernel runs through the calculation to determine what NIC to use. So if the load balancing value is 1, then 1 MOD 4 (number of NICs UP) = 1, which means NIC 1 is used. 2 MOD 4 = 2, so NIC 2 is used. 3 MOD 4 = 3, so NIC 3 is used, and so on. Networking • Module 4 Networking: Modify Virtual Switch Configurations 107 Load Balancing Method: Source MAC-Based The example above shows routing based on source MAC hash. In this load balancing method, each virtual machine's outbound traffic is mapped to a specific physical NIC based on the virtual NIC's MAC address. This method has low overhead, is compatible with all switches, but might not spread traffic out evenly across the physical NICs. When the load is distributed in the NIC team using the MAC-based method, no single-NIC VM will ever get more bandwidth than can be provided by a single physical adapter. This slide builds: • 1st screen: This slide illustrates how the VMkernel uses the source MAC-based method to balance network traffic across NICs. • Press Enter • Blue line drawn to first NIC • Press Enter • Purple line drawn to first NIC. Collisions occur. • Press Enter • Green line drawn to second NIC • Press Enter • Red line drawn to second NIC. The third NIC never gets used. 108 VMware Infrastructure 3: Install and Configure Load Balancing Method: IP-Based 4 Networking The example above shows routing based on IP hash. In this load balancing method, a NIC for each outbound packet is chosen based on its source and destination IP address. This method has higher CPU overhead, is not compatible with all switches (requires 802.3ad link aggregation support, also known as EtherChannel), but has a better distribution of traffic across physical NICs. When the load is distributed in the NIC team using the IP-based method, a single-NIC VM might use the bandwidth of multiple physical adapters. What if the packet is not an IP packet, but, for example, a Novell packet instead? The VMkernel looks in the place in the packet where the IP address would be if it were an IP packet and uses those bits. When one VM communicates to different clients, it chooses different NICs. On the return traffic, it can come in on multiple paths since more than two NICs might be teamed. That is why link aggregation must be supported on Module 4 Networking: Modify Virtual Switch Configurations 109 the physical switch. Note that none of this deals with any inbound traffic, just the outbound traffic is affected. This slide builds: • 1st screen: This slide illustrates how the VMkernel uses the IP-based method to balance network traffic across NICs. With this method, the VMkernel snoops inside the packets. • Press Enter • Blue line drawn to first NIC • Press Enter • Purple line drawn to second NIC. • Press Enter • Red line drawn to third NIC • Press Enter • Green line drawn to second NIC. 110 VMware Infrastructure 3: Install and Configure Detecting and Handling Network Failure • Network failure is detected by the VMkernel, which monitors • Link state only • Link state + beaconing • Switches can be notified whenever • There is a failover event • A new virtual NIC is connected to the virtual switch • Failover implemented by the VMkernel based on configurable parameters 4 • Load Balancing option: Use explicit failover order • Always use the highest order uplink from the list of Active adapters which passes failover detection criteria Networking • Rolling Failover • Determines how a physical adapter is returned to active duty after recovering from a failure The VMkernel can use link status and/or beaconing to detect a network failure. Monitoring the link status provided by the network adapter will detect failures such as cable pulls and physical switch power failures, but not configuration errors such as a physical switch port being blocked by spanning tree or misconfigured to the wrong VLAN. It will also not detect cable pulls or any type of link failure on the other side of the physical switch. When beaconing is activated, the VMkernel sends out and listens for probe packets on all NICs in the team. This technique can detect failures that link-status monitoring alone cannot. Whenever a virtual NIC is connected to a virtual switch, or whenever a failover event causes a virtual NIC's traffic to be routed over a different physical NIC, a notification is sent out over the network to update the lookup tables on physical switches. In most cases, this is desirable, because otherwise virtual machines would experience greater latency after failovers and VMotion. However, do not use this option when the virtual machines using the port group are using unicast-mode Microsoft Network Load Balancing. (NLB in multicast mode is unaffected). For more details on the NLB issue, see Knowledge Base article 1556 (http://kb.vmware.com/kb/ 1556). When using explicit failover order, always use the highest order uplink from the list of Active adapters which passes failover detection criteria. Rolling failover determines how a physical adapter is returned to active duty after recovering from a failure. If rolling is set to No, the adapter is Cable pulls or the loss of link status on the other side of the switch might be forwarded to the ESX Server if the physical switch supports port groups. This is covered in the DSA course. Beaconing introduces a load of a 62-byte packet every ~10 seconds per physical NIC. Module 4 Networking: Modify Virtual Switch Configurations 111 returned to active duty immediately upon recovery, displacing the standby adapter that took over its slot. If rolling is set to Yes, a failed adapter is left inactive even after recovery until another currently active adapter fails, requiring its placement. Additional Information ==> Enabling the Notify Switches option causes the VMkernel to send a Reverse ARP (RARP) request for each virtual NIC's MAC over it's currently favored uplink whenever there is a failover event, or whenever there is a (re)connect of a virtual NIC to the virtual switch. The purpose of this is to update the MAC->port lookup tables on the physical switches. If this is not done, these MAC->port entries can take some time to expire on their own and update after a link failover (or VMotion) event. One reason not to do this is if you are using Microsoft's unicast Network Load Balancing, which works by hiding the adapter's MAC address. Therefore, doing the RARPs will break it. To my knowledge, MS unicast NLB is the only thing that has a problem with our switch notification mechanism. MS multicast NLB works fine with switch notification. For more information on unicast NLB, see the forum thread, http://www.vmware.com/community/ thread.jspa?messageID=346965. 112 VMware Infrastructure 3: Install and Configure Lab for Lesson 2 • Design networking • In this lab, you will perform the following task: •Based on a given scenario, design the network configuration for an ESX Server system, specifying virtual switches, ports and port groups, port group policies, and physical connections 4 Networking Module 4 Networking: Modify Virtual Switch Configurations 113 Lesson Summary • Network adapter properties • Port group policies • VLAN tagging • Security • Traffic shaping • NIC teaming 114 VMware Infrastructure 3: Install and Configure Module Review • What are the three virtual switch connection types? Describe the purpose of each type. • What is an "internal-only" virtual switch? • What are the uses for a VMkernel port? • Name the different load-balancing algorithms that can be used by a NIC team. 4 Networking Module 4 Networking: Modify Virtual Switch Configurations 115 Questions? Questions? 116 VMware Infrastructure 3: Install and Configure MODULE 5 Storage 5 5 Storage Importance • Storage options give you the flexibility to set up your storage based on your cost, performance, and manageability requirements • Shared storage is useful for disaster recovery, high availability and moving VMs between ESX Servers Objectives for the Learner • Understand the purpose and configuration of virtual switches • Create virtual switches and connections • Understand virtual switch settings and policies Module Lessons • Fibre Channel San Storage • iSCSI SAN Storage • VMFS Datastores • NAS Storage and NFS Datastores VMware Infrastructure 3: Install and Configure 117 Lesson 1 Fibre Channel San Storage Lesson Topics • Fibre Channel SAN components and addressing • Configuring Fibre Channel SAN storage : 118 VMware Infrastructure 3: Install and Configure What is Fibre Channel (FC)? • A high-speed SCSI transport protocol used for Storage Area Networking (SAN) • Fibre Channel switches interconnect multiple nodes to form the “fabric” in a Fibre Channel SAN • Standard first ratified by ANSI in 1988 Block storage Fibre Channel 5 Storage Fibre Channel is a high-speed transport protocol used for Storage Area Networks (SANs). Fibre Channel encapsulates SCSI commands, which are transmitted between Fibre Channel nodes. In general, a Fibre Channel node is a server, storage system or a tape drive. A Fibre Channel switch interconnects multiple nodes, forming the "fabric" in a Fibre Channel network. Transmission speeds in a Fibre Channel SAN can reach up to 4 Gbps.. Fibre Channel is a standard that was first ratified by the American National Standards Institute (ANSI) in 1988. This standard was adopted by storage vendors due to high transfer rates, as well as low latency and overhead. Not covered in this course is N-Port ID Virtualization (NPIV): ESX Server 3.5 introduces support for NPIV for Fibre Channel SANs. Each virtual machine can now have its own World Wide Port Name (WWPN). This is covered in the VI3: Deploy, Secure and Analyze course. Module 5 Storage: Fibre Channel San Storage 119 How is Fibre Channel Used with ESX Server? • Boot ESX Server from Fibre Channel SAN LUN • Create a VMFS on a Fibre Channel SAN LUN • To hold VMs’ files, ISO images, and templates • Allow VM access to a raw Fibre Channel SAN LUN • Allow VMotion migration of a VM whose files reside on a Fibre Channel SAN LUN All of these points will be covered later on in the course, so please refrain from discussing them in detail at this time. VMs’ files are the virtual disks, VM's swap file, nvram, snapshot files, configuration file and log files. Installing and booting the ESX Server on Fibre Channel SAN storage is supported. To boot from SAN, the BIOS of the Fibre Channel adapter must be configured with the WWN and LUN number of the boot device and the system BIOS must designate the Fibre Channel adapter as a boot controller. It is common to use Fibre Channel SAN storage for VMFS datastores. VMFS datastores are used to hold virtual machines’ files, ISO images, and templates. It is also possible to assign a raw Fibre Channel SAN LUN to a virtual machine, for example, to hold an application’s data. VMotion is supported with virtual machines on Fibre Channel SAN storage. NOTE Using raw LUNs and VMotion is covered later in the course. 120 VMware Infrastructure 3: Install and Configure Fibre Channel SAN Components 5 Storage ESX Server requires the use of a Fibre Channel switch for connection to storage; the use of more than one allows for redundancy. A Fibre Channel SAN consists of the following: • Storage System: This is the hardware that consists of a set of physical hard disks, or disk array, and one or more intelligent controllers. The storage system supports the creation of LUNs. Disk arrays' storage processors aggregate physical disks into logical volumes, or LUNs, each with a LUN number identifier. • LUN: Logical Unit Number, it is the address of a Logical Unit (LU). An LU is a unit of storage access. An LU can be a JBOD (just a bunch of disks) or a part of a JBOD, a RAID set, also referred to as a "storage container", or a part of a storage container. Both a JBOD and a storage container can be partitioned into multiple LUNs. An LU can also be a control function like an array gatekeeper LUN or tape controller. • SP: Storage Processor, it can partition a JBOD or RAID set into one or more than one Logical Units (LUNs). It can restrict access of a particular LUN to one or more server connections. Each connection is referenced by the server HBA's WWN (World-Wide Name), and might also require defining the operating system in the connection tables to adjust how the storage array controller presents Fibre Channel and SCSI commands to a particular server. Module 5 Storage: Fibre Channel San Storage 121 The SP can define read, read-ahead, and writeback cache, stripe size, controller redundancy, multi-pathing and a variety of other options. You might have to update the firmware version to get the system to work, or to enable new features, or to resolve compatibility issues. • HBA: Host Bus Adapter connects the ESX Server to the Fibre Channel network and is required along with cables attached to the Fibre Channel switch ports. A minimum of two HBA adapters is used for fault tolerant configurations. Virtual machines see standard SCSI connections and are not aware of the underlying storage area network being accessed. • FC Switches: One or more Fibre Channel (FC) switches form the Fibre Channel fabric. The Fibre Channel fabric interconnects multiple nodes. The FC switches form packets from the FC messages and add the source and destination addresses to each packet. The Fibre Channel switch might have to be updated by flash upgrade to firmware to resolve interoperability issues and to add new features. Additional Information: Theoretically, you can have up to 239 switches in the fabric, but vendor certified solutions are considerably less (typically 5-30)! Different vendors' switches will provide basic interoperability although vendor specific enhancements often cannot be shared. Why only 239 switches? • Internally generated N-Port ID addresses are used to route packets within the FC network. The 24 bit N-Port ID address is broken into three 8-bit parts known as the Domain, Area, and Port. The Domain field is the address of the switch. The Area field is the port number on the switch. The Port field contains the Fibre Channel Arbitrated Loop (FCAL) address of any loop devices attached to the fabric. Since we do not support FCAL, the address in the Port field will be 00. Vendors limit the number of switches to less than 239 due to switch Inter-Switch Links (ISL) traversals (AKA "Hops"), convergence traffic, latencies introduced, etc. A "node" (mentioned in the descriptions above) generally is a server, storage or a tape drive. A switch COULD be a node, but only from a fabric management perspective, as that is about all their WWN can be used for. Most storage folks do not consider them to be nodes. The 24 bit N-Port ID address is broken into 3 8-bit parts known as Domain, Area, and Port. The Domain field is the address of the switch. The Area field is the port number on the switch. The Port field contains the Fibre Channel Arbitrated Loop (FCAL) address of any loop devices attached to the fabric. Since we do not support FCAL, the address in the Port field will be 00. 122 VMware Infrastructure 3: Install and Configure Addressing and Access Control in a Fibre Channel SAN 5 Storage There are several mechanisms for controlling hosts' access to LUNs. Soft zoning, which is done on a Fibre Channel switch, controls LUN visibility on a per-WWN basis. The Fibre Channel switch might also implement hard zoning, which is the control of storage-processor visibility on a per-switchport basis. Fabric zoning controls target presentation, and tells an ESX Server that a target exists or not. If the host can't get to the target, it can't see the LUNs World Wide Names (WWNs) are assigned by the manufacturer of the SAN equipment. HBAs and SPs have WWNs. WWNs are used by SAN administrators to identify your equipment for zoning purposes. In many well-managed SAN environments, both soft and hard zoning are in use. The purpose of using both is to make accidental access to volumes by servers very unlikely. Zoning is especially important in environments where physical Windows servers are accessing the SAN, because Windows operating systems typically write a disk signature on any storage volumes they see. These volumes might in fact be in use by non-Windows systems. The storage processor or the hosts themselves might also implement LUN masking, which controls LUN visibility on a per-host basis. ESX Server offers a mechanism for LUN masking. Although LUN masking can be done within the ESX Server, LUN masking is normally performed at the storage processor (SP) level, and, with newer switches, can also be done at a switch/ Module 5 Storage: Fibre Channel San Storage 123 fabric level. Though it could be done at the host level, it normally is not, for security and data integrity sake. If a LUN is masked, the SP does not tell the host the LUN exists nor does it allow any communication with it. ESX Server doesn't have to obey it. It has no choice. It can't "see" it (as it is not presented, and is barred from any communication with it. It's important to stress that WWNs are both WWNNs (world-wide node names) and WWPNs (world-wide port names). The VI class (and this) don't make the distinction. t is good to reinforce to students that the storage processor "presents" LUNS to the servers on the SAN according to the configuration made by the SAN administrator. The SAN administrator gets the WWN of a particular HBA on the SAN and then uses a configuration utility to present a particular LUN# to this HBA. The net result is that the server that contains the HBA is the server that the storage is being presented to. Once the LUN has been presented to the server by the SAN administrator, it is now up to the server to scan for storage on the SAN so that it will see only those LUNS that have been presented to it. This scanning for storage will be seen later in the module. Additional Information: How to mask LUNs on an ESX Server: • In the VI Client, in the Configuration tab of the ESX Server, select Advanced Settings, then select Disk. Disk.MaskLUNs is the fourth parameter in the list. To set the value, you may list one or more ranges of LUNs for the VMkernel to ignore on boot. Use the following syntax: adapter:target:comma_separated_LUN_range_list, for example, vmhba0:0:0-6; • If you wish to mask several ranges, separate them with semicolons, for example, vmhba0:0:0-6;vmhba1:0:0-4,7; • And, always place a semicolon at the end of the line. Soft zoning requires cooperation by hosts; a few HBAs are "bad citizens" and do not respect it. On the other hand, hard zoning is enforced by the Fibre Channel switch. The debate of hard vs soft zoning is a big deal. Soft zoning is more convenient, in that if you move to a different port on the fabric, it preserves the zoning info, as it is based on the FC-HBA WWN. However, if you change FC-HBA's the zoning info is lost, due to the WWN changing. The biggest danger of soft zoning is that it DOES NOT prevent communications with known targets, that the host already knows about or can otherwise discovers. Hard zoning prevents, through hardware enforcement, ports on different zones from communicating. The example I use for soft zoning is like having an unlisted phone number. It doesn't stop calls IF somebody already has your number or can find it out by other means. The S_ID is assigned to HBAs and SPs and is done at the Fibre Channel switch level. It's mostly important to SAN administrators. The S_ID is the Source ID field in the Fibre Channel packet. What it contains is the 24-bit N-Port ID that is the Domain/Area/Port address. More information on Zoning: • Zoning can be used to segment the fabric by OS, function, responsible group, etc. It is similar in concept to VLANs. • Zoning is used to create barriers between different operating environments. • It is used to deploy logical Fabric subsets by creating defined user groups • It is used to create test and/or maintenance areas that are separate within the Fabric • It allows finer segmentation of Storage Area Networks by creating Logical Subsets of devices within a Server-Storage Area Network 124 VMware Infrastructure 3: Install and Configure Addressing SAN LUNs in the VMkernel VMkernel addresses disk partitions as follows: Examples: LUN addresses Partition addresses vmhba0:0:11 vmhba0:0:11:3 vmhba1:1:12 vmhba1:1:12:1 5 Storage The VMkernel disk partition addressing scheme is as follows: • vmhba: Standard label that identifies a physical host bus adapter • Adapter: Adapter ID, assigned to each HBA • Target: Represents the SCSI target that the Storage Processor presents • LUN: Logical Unit Number • Partition: Partition on the LUN, identified by a number If you have multiple disk arrays in your SAN fabric, each must be configured with a different target ID, and each will appear to ESX Server as a different target number. If one of your disk arrays has multiple storage processors, each will also have a different target ID. For any given “disk” the same LUN number must be presented to all ESX Servers accessing it. It is a good idea to help students remember this physical scsi addressing scheme with the expression "c-t-l-p" as in, "The vmkernel addresses the physical storage it sees as 'Control-TargetLun-Partition'" Furthermore it is good to reinforce that just because the vmkernel can see a LUN does not mean there is a VMFS on it. One must format the LUN with a VMFS if there is none. This will be seen later in the module. Module 5 Storage: Fibre Channel San Storage 125 The terms Target ID and SCSI ID can, traditionally, be used interchangeably, but there is a danger: The Target ID (i.e. VMHBA0:1), represents the SAN target that the Storage Processor presents, and the LUN's are underneath it. The SCSI ID (i.e. VMHBA0:1), represents the SCSI device number on a SCSI bus (0-15), when the HBA is a SCSI Host Bus Adapter. This would be if you had a JBOD on a SCSI Adapter, and normally, the LUN number would be 0. What is the danger in using the term SCSI ID? The term SCSI ID, COULD also refer to the SCSI ID number the disk presents (physical or virtualized disk) that is used to insure uniqueness, though the better term is SCSI Serial Number, though in proc, I think it is referred to as a SCSI ID, and is a hex number. Why do we care about the SCSI Serial number (SCSI ID) ??? It is used when the VMkernel sees a LUN and mounts it. It uses that to determine whether it is seeing a unique LUN, or a different LUN, and is critical to RDM and multipath operations. If it sees a given LUN SCSI Serial Number, RDM uses it to determine what metadata file to use, as part of dynamic name resolution. Path management uses it to figure out that it is seeing the same LUN through multiple paths, regardless of the volume name, as opposed to thinking it is the same LUN. Original SCSI had 8 devices on the bus (0-7, with 7 being the controller itself). This was inadequate, so they added a "Logical Unit Number" below the SCSI ID to increase the number of possible addressable devices, originally for disk libraries. With SCSI-2, it increased the number of devices on the bus (0-15, with 7 STILL being reserved for the controller itself). The LUN Number remained in the standard, though rarely used. Port ID is a very different thing from the Target ID, as you have to factor in path management, as there could be multiple paths to the same target, depending on the switch configuration, and that could get VERY confusing. 126 VMware Infrastructure 3: Install and Configure Making SAN Storage Available to ESX Server • The Fibre Channel storage adapter is recognized by the VMkernel during the boot sequence • At boot up, the VMkernel scans up to 256 LUNs • The Rescan link allows the VMkernel to scan the Fibre Channel SAN for additional LUNs 5 Storage The VMkernel parameter, Disk.MaxLUN, is now 255 by default. Disk.SupportSparseLUNs still sets noncontiguous order by default. All supported PCI devices (SCSI, FC, Ethernet, iSCSI, etc.) are assigned to the VMkernel, and are recognized by the VMkernel when the ESX Server boots. ESX 3 supports 256 LUNs found in the range of 0-255. However, during installation, the ESX installer can only see the first 128 LUNs. This is a display from the VI Client interface. To get to this display, select the ESX Server, click its Configuration tab, then select the Storage Adapters link. Module 5 Storage: Fibre Channel San Storage 127 Where to Find Information on SAN Troubleshooting • Keys to successfully troubleshooting SAN and storage subsystems • Understanding the components of the SAN •Fibre Channel, SCSI protocol, Fibre Channel fabric switch commands • Having a working knowledge of the SAN components’ specifications and limitations • Keeping accurate documentation of system architecture and configuration • Consult the SAN System Design and Deployment Guide, Chapter 10, Common Problems and Troubleshooting • http://www.vmware.com/pdf/vi3_san_design_deploy.pdf Troubleshooting SAN and storage subsystems is both a science and an art. The science of troubleshooting relies on understanding the components of your SAN or storage subsystems and obtaining a working knowledge of component specifications and limitations. Using your experience to troubleshoot a problem, and more specifically, identify where in the overall system to focus your investigation first, is the art. It is also extremely helpful to have a record of your SAN fabric infrastructure architecture and component configuration. For a detailed discussion on how to troubleshoot and resolve issues in systems using the VMware Infrastructure with SAN, consult the SAN System Design and Deployment Guide, Chapter 10, Common Problems and Troubleshooting, available on the VMware Web site at http:// www.vmware.com/pdf/vi3_san_design_deploy.pdf. 128 VMware Infrastructure 3: Install and Configure Lesson Summary • The worldwide name (WWN) uniquely identifies a node in the Fibre Channel network • LUN masking and zoning make a LUN invisible when a target is scanned, and is usually set at the SP level • The VMkernel addresses LUNs using the following syntax: vmhbaadapter#:target#:LUN#:partition# • Fibre Channel networks can be dynamically rescanned to find newly added LUNs 5 Storage Module 5 Storage: Fibre Channel San Storage 129 Lesson 2 iSCSI SAN Storage Lesson Topics • iSCSI components and addressing • iSCSI hardware and software initiators • Configuring the iSCSI software initiator • Configure access to iSCSI storage : 130 VMware Infrastructure 3: Install and Configure What is iSCSI? • A SCSI transport protocol, enabling access to storage devices over standard TCP/IP networks • Maps SCSI block-oriented storage over TCP/IP • Similar to mapping SCSI over Fibre Channel • “Initiators”, such as an iSCSI HBA in an ESX Server, send SCSI commands to “targets”, located in iSCSI storage systems Block storage IP 5 Storage iSCSI (Small Computer System Interface over IP) provides alternatives to Fibre Channel SANs: • Cost: iSCSI is less expensive than Fibre Channel and you can use the NICs that already exist in your system. And, Ethernet switches cost less than Fibre Channel switches. • Infrastructure: Use your existing infrastructure and existing network knowledge as well; network administrators know about iSCSI routing and switching since it uses the same methods as regular office Ethernet traffic. • Routing: IP routing is mature and well understood. • Internet: iSCSI is Internet ready. Since iSCSI is based on IP, transfers of information can more easily take place over WAN architectures in addition to LAN environments. Module 5 Storage: iSCSI SAN Storage 131 How is iSCSI Used with ESX Server? • Boot ESX Server from iSCSI storage • Using hardware initiator only • Create a VMFS on an iSCSI LUN • To hold VMs’ files, ISO images, and templates • Allow VM access to a raw iSCSI LUN • Allow VMotion migration of a VM whose files reside on an iSCSI LUN Installing and booting the ESX Server from iSCSI storage is supported. To boot from SAN, the BIOS of the iSCSI adapter must be configured with the WWN and LUN number of the boot device and the system BIOS must designate the iSCSI adapter as a boot controller. VMs’ files are the virtual disks, VM's swap file, nvram, snapshot files, configuration file and log files. Use iSCSI storage for VMFS datastores. VMFS datastores are used to hold virtual machines’ files, ISO images, and templates. It is also possible to assign an iSCSI LUN to a virtual machine, for example, to hold an application’s data. VMotion is supported with virtual machines on iSCSI storage. NOTE Using raw LUNs and VMotion is covered later in the course. 132 VMware Infrastructure 3: Install and Configure Components of an iSCSI SAN Targets IP Network Initiators * Software initiator 5 Storage An initiator transmits SCSI commands over the IP network. A target receives SCSI commands from the IP network. You can have multiple initiators and targets in your iSCSI network. iSCSI is SAN-oriented in that the initiator finds one or more targets, a target presents LUNs to the initiator, and the initiator sends it SCSI commands. An initiator resides in the ESX Server while targets reside in the storage arrays supported by the ESX Server. LUN masking is also available in iSCSI and works like it does in Fibre Channel. Ethernet switches do not implement zoning like Fibre Channel switches. Instead, you can create zones using VLANs. Module 5 Storage: iSCSI SAN Storage 133 Addressing in an iSCSI SAN iSCSI target name iqn.1992-08.com.netapp:stor1 iSCSI alias stor1 IP address 192.168.36.101 iSCSI initiator name iqn.1998-01.com.vmware:train1 iSCSI alias train1 IP address 192.168.36.88 * Software initiator IP Network The main addressable, discoverable entity in iSCSI is an iSCSI Node. An iSCSI node can be either an initiator, a target, or both. Both targets and initiators require names for the purpose of identification, so that iSCSI storage resources can be managed regardless of location (address). The rules for constructing an iSCSI name are specified in RFC 3720 (see http:// www.faqs.org/rfcs/rfc3720.html). The IQN (iSCSI Qualified Name) naming convention is as follows: • The string "iqn." • A date code specifying the year and month in which the organization registered the domain or sub-domain name used as the naming authority string • The organizational naming authority string, which consists of a valid, reversed domain or subdomain name • Optionally, a ':', followed by a string of the assigning organization's choosing, which must make each assigned iSCSI name unique 134 VMware Infrastructure 3: Install and Configure Additional Information: An iSCSI node also has one or more addresses. An iSCSI address specifies a single path to an iSCSI node and consists of the iSCSI name, plus a transport (TCP) address which uses the following format: [:] The default port 3260, assigned by IANA, will be assumed. An iSCSI Name is a location-independent, permanent identifier for an iSCSI node. An iSCSI node has one iSCSI name, which stays constant for the life of the node. An iSCSI Address specifies not only the iSCSI name of an iSCSI node, but also a location of that node. The address consists of a host name or IP address, a TCP port number (for the target), and the iSCSI Name of the node. The alias strings are communicated between the initiator and target at login, and can be displayed by a user interface on either end, helping the user tell at a glance whether the initiators and/or targets at the other end appear to be correct. The alias is a variable length string, between 0 and 255 characters. An iSCSI node can have any number of addresses, which can change at any time. To assist in providing a more human-readable user interface for devices that contain iSCSI targets and initiators, a target or initiator may also provide an alias. Note that this means iSCSI names are independent of location. Furthermore, iSCSI names are associated with iSCSI nodes instead of with network adapter cards to ensure the free movement of network HBAs between hosts without loss of SCSI state information (reservations, mode page settings etc) and authorization configuration. The following is an example of an iSCSI qualified name from an equipment vendor: iqn.2001-04.com.example:diskarrays-sn-a8675309 The following is an example of an iSCSI name string from a storage service provider: iqn.1995-11.com.example.ssp:customers.4567.disks.107 Note that when reversing these domain names, the first component (after the "iqn.") will always be a top-level domain name, which includes "com", "edu", "gov", "org", "net", "mil", or one of the two-letter country codes. The use of anything else as the first component of these names is not allowed. What if you do not know the year and month in which the organization registered the domain? • Then, make one up. The reason why the year and month is part of the IQN is for uniqueness. Of course, it is always preferable that you try to use the correct year and month when possible. Another iSCSI naming convention is the EUI. format: • The iSCSI EUI. naming format allows a naming authority to use IEEE EUI-64 identifiers in constructing iSCSI names. The details of constructing EUI-64 identifiers are specified by the IEEE Registration Authority (see [EUI64]). • Example iSCSI name: eui.02004567A425678D The EUI naming convention should not be used because it is not supported in the current iSCSI implementation. This slide (iSCSI Name/Alias) and the next two slides (Discovery Methods and CHAP Authentication) should be used to explain iSCSI concepts to the student. After the concepts are explained, then the next sequence of slides describes how to configure the iSCSI software initiator. Present this sequence of slides with this thought in mind: Concepts first, then "How To" next. 5 Storage Module 5 Storage: iSCSI SAN Storage 135 How iSCSI Targets are Discovered • Two discovery methods are supported: • Static Configuration • SendTargets 192.168.36.101:3260 IP Network SendTargets request SendTargets response • SendTargets response returns IQN and all available IP addresses iSCSI target 192.168.36.101:3260 In order for an iSCSI initiator to establish an iSCSI session with an iSCSI target, the initiator needs the IP address, TCP port number and iSCSI target name information. The goal of iSCSI discovery is to allow an initiator to find the targets to which it has access, and at least one address at which each target may be accessed. This should generally be done using as little configuration as possible. The iSCSI discovery mechanisms listed here only deal with target discovery and one still needs to use the SCSI protocol for LUN discovery. The ESX Server implementation of iSCSI supports the following discovery mechanisms: • Static Configuration: IP address, TCP port and the iSCSI target name are already available to the initiator. No target discovery is necessary. This discovery option is convenient for small iSCSI setups. • SendTargets: Initiator uses target's IP address and TCP port information to establish a discovery session to the IP address. The initiator then issues the SCSI SendTargets command to query information about the iSCSI targets available at the particular IP address. Hardware initiators suppport both the static and SendTargets configuration, whereas software initiators support only SendTargets. 136 VMware Infrastructure 3: Install and Configure An iSCSI discovery method not supported in ESX 3: • Zero-Configuration: This mechanism assumes that the initiator does not have any information about the target. In this option, the initiator can either multicast discovery messages directly to the targets or it can send discovery messages to storage name servers. Currently, the main discovery frameworks available are SLP and iSNS. The administrator must manually provide the IP address of one or more iSCSI targets, then SendTargets does the rest by discovering that target's information. 5 Storage Module 5 Storage: iSCSI SAN Storage 137 How iSCSI Storage Authenticates the ESX Server • CHAP (Challenge-Handshake Authentication Protocol) • Allows a password to be verified without sending the password (in cleartext) over the network CHAP password: K CHAP password: K Log into target Computes using formula against K using C and sends result R C R Accept or Reject Create random hash/computation value “C” and send value back to ESX Server; perform local computation against K using C to come up with R. Compares local R against returned R; If match, then ESX Server is authenticated. CHAP authentication is a mechanism in which the target (the storage resource) authenticates the initiator trying to access it (in this case, the ESX Server.) CHAP can be enabled on either a hardware or software initiator. By default, CHAP is disabled. It is a best practice to create a separate, isolated IP network for iSCSI traffic since transmitted data is unencrypted. If the network is to be shared between iSCSI traffic and other traffic, then enable CHAP authentication. Using different CHAP passwords for different storage devices also prevents you or your co-workers from accidentally reformatting the wrong storage. An isolated network is the only way VMware supports iSCSI. ESX Server implements RFC 1994. Security-related functionality not yet supported in ESX Server: • • • Bi-directional CHAP authentication Bi-directional mutual authentication Data encryption (IPSec) 138 VMware Infrastructure 3: Install and Configure iSCSI Software and Hardware Initiators ESX Server provides full support for software and hardware initiators Software Initiator Hardware Initiator e.g. QLogic QLA4050C 5 Storage The software initiator is a port of the Cisco iSCSI Initiator Command Reference implementation. VMware has modified it to work with ESX 3 and the VMkernel networking stack. The software initiator works with the vmkiscsid daemon that runs in the service console. Therefore, the service console and VMkernel NICs both need access to the iSCSI storage since the iSCSI daemon initiates the session and handles login and authentication. The actual I/O goes through the VMkernel. The hardware initiator provides access to storage like other types of SCSI adapters. SCSI LUNs are made available to the ESX Server from the iSCSI adapter. The hardware initiator offloads the iSCSI network traffic load from the VMkernel’s networking stack. For both initiators, hardware and software, the guest OS never specifically sees iSCSI network traffic. Since the guest OS is not aware of the underlying storage, the guest OS sees only virtual disk SCSI I/O traffic. ESX Server does not support both hardware and software initiators running simultaneously. For a list of iSCSI storage arrays supported for iSCSI software and/or hardware initiators, consult the Storage/SAN Compatibility Guide, available on the VMware Web site. Module 5 Storage: iSCSI SAN Storage 139 iSCSI Software Initiator Network Configuration • • Both service console and VMkernel need to access the iSCSI storage Two ways to do this: 1. Have the service console port and VMkernel port share a virtual switch and be in the same subnet 2. Have the service console port and the VMkernel port on different networks, but have routing in place The software initiator works with a daemon called vmkiscsid that runs in the service console. Therefore, the service console and VMkernel NICs both need access to the iSCSI storage since the iSCSI daemon initiates the session and handles login and authentication. The actual I/O goes through the VMkernel. To get to the virtual switch display (shown above), in the VI Client inventory list, select your ESX Server, click its Configuration tab, then click the Networking link. 140 VMware Infrastructure 3: Install and Configure Enable iSCSI Traffic Through the Service Console Firewall 5 Storage In order for the iSCSI software initiator to communicate with its target iSCSI storage, outgoing port 3260 needs to be opened in the service console firewall. Use the VI Client to open the port. Select your ESX Server in the inventory, then click its Configuration tab. Click the Security Profile link, then click the Properties link to display the Firewall Properties window. Locate the Software iSCSI Client service. Select the check box next to this service to open this firewall port in the service console. Module 5 Storage: iSCSI SAN Storage 141 Configure the iSCSI Software Initiator To configure the iSCSI software initiator, use the VI Client. Select your ESX Server, click the Configuration tab, select the Storage Adapters link. A list of available storage adapters is displayed. Select iSCSI Software Adapter, then click the Properties... link. 142 VMware Infrastructure 3: Install and Configure Configure Software Initiator: General Properties (1 of 2) • Enable the iSCSI initiator 5 Storage The iSCSI Initiator Properties window displays. Click the Configure button in the General tab. The General Properties window displays. Select the check box, Enabled, then click OK. Module 5 Storage: iSCSI SAN Storage 143 Configure Software Initiator: General Properties (2 of 2) • The iSCSI name and alias are automatically filled in after initiator is enabled By enabling the software initiator, a default iSCSI name and alias is chosen for you. The iSCSI name follows the IQN naming convention and the iSCSI alias is the fully-qualified domain name of your ESX Server. You can change these defaults if you wish, however, it is recommended to always use the IQN naming convention when defining the iSCSI name. This is because most iSCSI storage arrays know how to recognize that name. If an IQN is not used, it is possible that an iSCSI array might not recognize it. 144 VMware Infrastructure 3: Install and Configure Configure Software Initiator: Dynamic Discovery • In the Dynamic Discovery tab, enter the IP address of each target server for initiator to establish a discovery session 5 Storage The iSCSI Initiator Properties window has two tabs, Dynamic Discovery and Static Discovery. To use the SendTargets method of discovery, enter the address of the target device (referred to as the Send Targets server) in the Dynamic Discovery tab. The initiator will establish a discovery session with this target. The target device responds by forwarding a complete list of additional targets that the initiator is allowed to access. The target device responds with a list of available targets, which is displayed in the Dynamic Discovery tab. The Static Discovery tab allows you to manually add IP addresses of any targets you identify as accessible to your ESX Server. However, adding static target IP addresses is only available with the hardware initiator, not the software initiator. To define the Send Targets server, click the Dynamic Discovery tab in the iSCSI Initiator Properties window and create an entry for each target server to discover. Module 5 Storage: iSCSI SAN Storage 145 Configure Software Initiator: CHAP Authentication • By default, CHAP is disabled • Enable CHAP and enter CHAP name and secret To set a CHAP login name and password, click the CHAP Authentication tab, then click the Configure... button. Type in a CHAP name and a CHAP secret. You can choose to use the name of the initiator as the CHAP login name. The CHAP secret must match the CHAP secret set at the target you wish to establish communication with. CHAP secrets (or shared secrets) are pre-shared keys (PSKs) that have been allocated to the communicating parties prior to the communication process starting. A shared secret is a string of text that a VPN service expects to get before it receives any other credentials (such as a username and password). Windows XP calls this string the "pre-shared key for authentication", but in most operating systems it is known as a "shared secret". The VPN server will not allow the authentication process to continue until the correct string of text is given. Unless the VPN server receives the shared secret, a username and password cannot be sent, and the connection will be refused. In a sense, a shared secret is sort of a password, albeit a weak one known by a large number of people. 146 VMware Infrastructure 3: Install and Configure Discover iSCSI LUNs • Rescan to find new LUNs 5 Storage After configuring the properties of the iSCSI software adapter, you are ready to scan for iSCSI target LUNs. Click the Rescan link to start the rescan. The iSCSI software adapter is identified as vmhba32. An iSCSI hardware adapter is identified using an available vmhba. For example, if an iSCSI hardware adapter were added to the system above, the VMkernel would name that adapter vmhba2. Module 5 Storage: iSCSI SAN Storage 147 Problem: Cannot Access iSCSI Storage • Is the VMKernel port configured and on the same LAN as the storage array? • Is a second Service Console port required? • Is there an error in network configuration (storage array IP address, routing, etc.)? • Is Send Targets correctly configured? • Is CHAP authentication required? • If CHAP is required, has the firewall port been opened on the Service Console? If you are having problems accessing your iSCSI storage, check your network configuration and iSCSI configuration. Is the VMKernel port configured and on the same LAN as the storage array? • Examine the network configuration of your ESX Server. • Make sure that you have a VMKernel port on a switch that is connected to the same LAN as the iSCSI or NAS storage array. • Make sure the IP address and subnet mask of the VMKernel port is correct for the storage LAN. • You should be able to ping the VMKernel address from the network storage device. Is a second service console port required? • Examine the network configuration of your ESX Server. • Is it possible for some other service console port on this ESX Server to connect to the storage array (possibly via a router)? You should be able to connect to your service console via SSH and ping the address of the network storage device. If you cannot reach it with the ping command, then you will need a second service console port. • Make sure your storage device will respond to ping requests • Make sure that no firewalls between the ESX Server service console NIC and the storage array are blocking the ping requests. 148 VMware Infrastructure 3: Install and Configure • Make sure that any ports required by the service console (such as iSCSI 3260) are not blocked by firewalls if they are needed. Is there an error in the network configuration (storage array IP address, routing, etc.)? • Your network storage device configuration under ESX Server requires that you enter either an IP address or an FQDN for the storage array. Make sure this address and/or FQDN is correct. • If you connect to your storage via a routed network (not recommended), is your default router set for that network? Each ESX Server has a default router gateway set for all VMKernel and service console traffic. The gateway set for VMKernel does not have to be the same as the one for the service console, but there can only be one for each. All VMKernel ports use the same router and all service console ports use the same router. If you are adding a new VMKernel or service console port that is on a new subnet and the defined gateway address is not on that subnet, you must already have a defined VMKernel and/or service console port on the subnet on which the gateway is defined. For example, if your new VMKernel port is on subnet 10.1.161.x and your gateway address for VMKernel traffic is 192.168.161.1, you must already have a VMKernel port defined somewhere on 192.168.161.x. • You must also make sure that the designated VMKernel and service console routers are online and functioning properly. Is Send Targets correctly configured for the Software Initiator? • Your iSCSI storage device configuration requires that you enter either an IP address or an FQDN for the storage array. If you are using a hardware adapter, consult your vendor documentation on the correct procedure to configure the iSCSI storage adapter. If you are using the ESX Server's software initiator, you must correctly configure Send Targets. Is CHAP authentication required? • Some iSCSI storage devices are configured to require Challenge Handshake Authentication Protocol (CHAP) authentication. If you are using a hardware iSCSI adapter, consult your vendor documentation to determine how to configure CHAP authentication. If CHAP is required, has the Software iSCSI Client firewall port been opened on the service console? • An open service console firewall port is required when you are using the ESX Server's built-in iSCSI software initiator and CHAP authentication is required by the storage array • An open service console firewall port is NOT required when you are using a hardware iSCSI initiator card, or you are using the ESX Server's Module 5 Storage: iSCSI SAN Storage 149 5 Storage built-in iSCSI software initiator, but CHAP authentication is NOT required by the storage array. 150 VMware Infrastructure 3: Install and Configure Lab for Lesson 2 • Configure iSCSI storage • In this lab, you will perform the following tasks: •Configure a VMkernel port to access iSCSI storage •Configure the iSCSI software adapter •Rescan the storage adapter to detect the iSCSI storage 5 Storage Module 5 Storage: iSCSI SAN Storage 151 Lesson Summary • ESX Server provides full support for the iSCSI software initiator and hardware initiator (qualified iSCSI HBAs only) • iSCSI targets and initiators are identified by an IQN (iSCSI qualified name) • The sendTargets discovery method is the only method for a software initiator to discover LUNs in an iSCSI target 152 VMware Infrastructure 3: Install and Configure Lesson 3 VMFS Datastores Lesson Topics • Creating a VMFS datastore • Extending a VMFS datastore • Multipathing : 5 Storage Module 5 Storage: VMFS Datastores 153 What is a VMFS? • Repository of virtual machines and virtual machine state • Each virtual machine’s files are located in its own subdirectory • Repository for other files • Templates • ISO images • VMFS volumes are addressed by a volume label, a datastore name and physical address (e.g. vmhba1:0:0:1) • VMFS volumes are accessible in the service console underneath /vmfs/volumes The VMware File System (VMFS) is a file systemfile system optimized for storing ESX Server virtual machines. VMFS can be deployed on a variety of SCSI-based storage devices, including Fibre Channel and iSCSI SAN equipment. A virtual disk stored on a VMFS always appears to the virtual machine as a mounted SCSI device. The virtual disk hides a physical storage layer from the virtual machine's operating system. This allows you to run even operating systems not certified for SAN inside the virtual machine. Specific features of VMFS-3: • Distributed journaling • Faster file system recovery, independent of volume size or number of hosts connected • Scalable distributed locking-survives short and long SAN interruptions much better • Support for small files-small files allocated from sub-block resource pool volumes VMFS volumes are accessible in the service console underneath the /vmfs/ directory. This directory contains a subdirectory for each VMFS. The serial number of the disk on which the VMFS resides is used as the name of the subdirectory. The maximum number of hosts allowed to access a single VMFS at the same time is 32. This is a soft limit and a general recommendation. 154 VMware Infrastructure 3: Install and Configure VMFS-3 supports a maximum of 30720 files per-VMFS-3 directory and 30720 files per-VMFS-3 volume. Additional Information: Locking Contention in VMFS-3: For those of you familiar with the locking contention issues experienced in ESX 2/VMFS-2, here is the scoop on locking in VMFS-3: Locking contention has in fact increased with VMFS-3 because VMFS-3 stores many more virtual machine files than VMFS-2, such as log files, swap file, config file, snapshot file(s), etc. However, VMFS3 locking is scalable for a large number of files, so the behavior has improved because locking overhead has decreased. VMware cannot disclose exactly how scalability has been achieved in VMFS-3. Does locking contention still exist when snapshot files of multiple VMs exist in the same VMFS? Yes, that is still true, but since locking is better now, we can possibly host a bunch of snapshots on the same LUN. How many? We don't know yet. We have yet to conduct the requisite experiments to figure out a number. The command vmkfstools -R can be used to release SCSI locks. 5 Storage Module 5 Storage: VMFS Datastores 155 Creating a VMFS • Select device location (iSCSI or Fibre Channel LUN) • Specify datastore name • Change maximum file size/disk capacity, if desired When you create a VMFS, you can edit its properties. To create a VMFS datastore, go to the ESX Server's Configuration tab, then select the Storage link under the Hardware section. The screenshots in the slide are screens from the Add Storage wizard, which is launched when you click the Add Storage... link to create a VMFS. VMware only supports a single VMFS on a single partition on a LUN. A single-LUN VMFS must be at least 1.2 GB in size, but due to a limitation of the SCSI-2 protocol, a VMFS cannot exceed 2 TBs in size. 156 VMware Infrastructure 3: Install and Configure VMFS Properties 5 Storage In terms of the file system block size, the VMFS-3 file system does automatic sub-block to file block conversion. Small files start by using sub-blocks, and as they grow larger, VMFS changes them to use file blocks. However, the file block size doesn't change. The Storage display lists all datastores currently configured for the ESX Server. Selecting a datastore from the list allows you to view its Details. To get to this display, go to ESX Server's Configuration tab, then click the Storage link. Module 5 Storage: VMFS Datastores 157 Question How can you tell if the VMFS is on a Fibre Channel device or an iSCSI device? In answer to this question, there are a couple of ways to determine this: • By looking at the vmhba# in the device address. This screenshot was taken from an ESX Server where vmhba0 is the local HBA and vmhba1 is the Fibre Channel HBA. On any ESX Server, go to the Storage Adapters link in the Configuration tab to see the vmhba references. • By the datastore name. If the administrator uses a naming convention descriptive of storage type/location, then it will be quite easy to determine whether a datastore is on a Fibre Channel SAN or an iSCSI SAN. The student may or may not know this at this stage. However, we already discussed the device address (vmhba#:#:#:#) as well as datastore names, so chances are good that the student might take this info and apply it to this question. By the way, the answer is obvious if the software iSCSI initiator is being used because the vmhba will always be vmhba32. The answer is not so obvious if the hardware iSCSI initiator is used because you cannot readily tell if the vmhba is referring to a Fibre Channel adapter or an iSCSI adapter. Since the screenshot shows that vmhba32 is not being used, then the answer to this question is "It depends what vmhba0 and vmhba1 refer to." 158 VMware Infrastructure 3: Install and Configure Extend a VMFS • The size of a VMFS can be extended dynamically • Why extend a VMFS? • To give a VMFS more space without taking it offline • To create a VMFS > 2TB • In some cases, to improve overall I/O performance of the VMFS • If the master extent is lost, it could cause data loss on the entire VMFS Before spanning After spanning 5 vmhba0:0:3:1 vmhba0:0:6:1 vmhba0:0:3:1 vmhba0:0:6:1 Extent Storage In the ESX Server context, an extent is a hard disk partition on a physical storage device that can be added to an existing VMFS-based datastore dynamically, while the VMFS is in use. The datastore can stretch over multiple extents, yet appear as a single volume (analogous to a spanned volume.) One reason for extending a VMFS is to give it more space. It is also used to create a VMFS greater than 2TB in size. An extent must be added because the maximum size of a VMFS extent, including the original LUN, is 2TB. For example, to create a VMFS that is 6TB in size, create a VMFS 2TB in size, then add 2 extents (each extent is 2TB) to make a 6TB VMFS. A physical extent can be a maximum of 2 TB. A VMFS can have up to 32 physical extents for a maximum VMFS size of approximately 64 TB. See the ESX Server Requirements section, "Maximum Configuration for ESX Server" in the VMware Virtual Infrastructure Installation and Upgrade Guide for more information on the maximum configuration. Another reason for using extents is to improve performance. In some cases, by having multiple VMkernel queues (one per LUN) and setting manual load distribution to divide the traffic between multiple paths going to multiple LUNs, overall I/O performance could improve for large VMFS volumes with a single Virtual Disk file. Be aware that when using extents with multiple LUNs, the master extent member, which is the first LUN in the set, contains the metadata for the Module 5 Storage: VMFS Datastores 159 entire extent set. If that master LUN is lost, it could cause a loss of all data on the entire extent set! 160 VMware Infrastructure 3: Install and Configure Add Extent Candidate to VMFS • The list of possible extent candidates will not include LUNs with existing VMFSes • If you choose a candidate with existing data (e.g. an NTFS), you are warned that data will be permanently lost if you use it 5 Storage To add an extent to a VMFS, go to the ESX Server's Configuration tab, then select the Storage link. Select the VMFS to extend from the displayed list, then click on the Properties... link. In the VMFS's Properties window, click the Add Extent... button to launch the Add Extent wizard. The only way to increase the size of an existing VMFS volume is to span it to another LUN. If you enlarge the disk partition that contains the VMFS volume, you will not be able to grow the volume to use the new space in the partition. Module 5 Storage: VMFS Datastores 161 VMFS Extent List Updated “Test_Dev_22” has two extents A VMFS consists of one or more extents. These extents form an extent group. The first extent refers to the original VMFS partition and is known as the extent master. The rest of the extents in the extent group are known as extent members. In the example above, the VMFS named Test_Dev_22 has two extents, an extent master and one extent member. A VMFS that has not been extended is simply a VMFS with one extent. To remove an extent, you must remove the entire VMFS. To remove a VMFS, select the VMFS from the list of storage devices, then click the Remove button. The entire VMFS will be deleted, including all of its data. 162 VMware Infrastructure 3: Install and Configure Multipathing with Fibre Channel • Multipathing allows continued access to SAN LUNs in the event of hardware failure • Exactly one path is active (in use) to any LUN at any time • Two multipathing policies exist: • MRU (Most Recently Used) • Fixed (Preferred path) The following multipathing policies are currently supported: • Fixed: The ESX Server always uses the preferred path to the disk when that path is available. If it cannot access the disk through the preferred path, then it tries the alternate paths. Fixed is the default policy for active/active storage devices. • MRU (Most Recently Used): The ESX Server uses the most recent path to the disk until this path becomes unavailable. That is, the ESX Server does not automatically revert back to the original path. MRU is the default policy for active/passive storage devices and is required for those devices. Manually changing Most Recently Used to Fixed is not recommended. The VMkernel sets this policy for those arrays that require it. The ESX Server automatically sets the multipathing policy according to the make and model of the array it detects. If the detected array is not supported, it is treated as active/active. For a list of supported arrays, see the SAN Compatibility Guide. ESX Server supports failover with any supported Fibre Channel adapter. The BIOS of the Fibre Channel adapter allows you to configure the failover delay. ESX Server multipathing is only supported for failover, not automatic load balancing. However, manual load balancing can also be achieved. The links between the HBAs and the SPs are intended to show the physical paths that ESX Server has a choice among. Make clear to the students that ESX Server uses exactly one path at a time to any given LUN. ESX Server 3.5 enhances native load balancing by providing experimental support for round-robin load balancing of HBAs. MRU is needed to prevent ping-pong’ing LUN ownership in the Storage Array when a given host experiences path failure. 5 Storage Module 5 Storage: VMFS Datastores 163 Multipathing with iSCSI • SendTargets advertises multiple routes • It reports different IP addresses to allow different paths to the iSCSI LUNs • Routing done via IP network • For the software initiator • Counts as one network interface • NIC teaming and multiple SPs allow for multiple paths IP networking already has multipath support built in (e.g. IP networking does routing, if you're using dynamic routing protocols). Therefore, it provides a simpler multipath structure than Fibre Channel networks. iSCSI initiators recognize multiple paths from a SendTargets discovery. Like our support with SANs, ESX uses multipathing for failover purposes only. The failover polices of fixed and MRU (most recently used) are the same policies used with SAN multipathing. ESX Server supports supports an active/passive configuration only. It also supports only one type of multipathing at a time, either software initiator multipathing or hardware initiator multipathing, but not both at the same time. Since the software initiator counts as only one "HBA", it relies on the underlying network to provide it with multiple paths to the iSCSI LUNs. This is accomplished by placing the VMkernel port used for iSCSI storage access on a virtual switch that has NIC teaming in place. There is no heterogeneous multipathing. In other words, you cannot use a NIC and an iSCSI adapter to access the same iSCSI storage. The software initiator only supports a single storage interface, in other words, the software initiator looks like a single iSCSI HBA. However, keep in mind that the software initiator sits on top of multiple NICs and therefore, multipathing can be performed through the networking layer in the VMkernel via NIC teaming. IP Network 164 VMware Infrastructure 3: Install and Configure It is possible to have both Fibre Channel and iSCSI HBAs in the same ESX Server. However, having the Fibre Channel and iSCSI HBAs point to the same LUN is not a supported configuration. 5 Storage Module 5 Storage: VMFS Datastores 165 Manage Multiple Paths Manage paths using the Storage Properties window Pathing information can be managed, for example, you can set a preferred path to be used for a particular LUN. You can also enable or disable a path to a particular LUN. To manage paths, use the VI Client. Select your ESX Server in the inventory, then click its Configuration tab. Select the Storage link, select the desired storage from the Storage list, then click its Properties... link. In the Storage Properties window, click the Manage Paths button to change your path configuration. If a LUN is not formatted, you can set the preferred path; however, the active path will not be switched to the preferred path until the LUN is formatted. Preferred paths can only be used with a Fixed policy. 166 VMware Infrastructure 3: Install and Configure Labs for Lesson 3 1. Create VMFS Datastore • In this lab, you will perform the following tasks: • • • Display information about your fibre channel adapter Create a VMFS datastore on a fibre channel SAN LUN Change the name of your local VMFS 2. (OPTIONAL) Extend a VMFS • In this lab, you will perform the following tasks: • • Add an extent to a VMFS datastore Remove an extent from a VMFS datastore 5 Storage Module 5 Storage: VMFS Datastores 167 Lesson Summary • A VMFS datastore is used to hold templates, ISO images and the files that make up a VM • The size of a VMFS can be dynamically extended • Multipathing in either a Fibre Channel or iSCSI LUN is used for path failover 168 VMware Infrastructure 3: Install and Configure Lesson 4 NAS Storage and NFS Datastores Lesson Topics • NAS storage • NFS components and addressing • Configuring an NFS datastore 5 : Storage Module 5 Storage: NAS Storage and NFS Datastores 169 What is NAS and NFS? • What is NAS (Network-Attached Storage? • Storage shared over the network at a filesystem level • Why use NAS? • Lower cost, lesser infrastructure investment required than Fibre Channel • There are two key NAS protocols: • NFS (Network File System) • SMB (Windows networking, also known as CIFS) • Major NAS appliances and server OSes support both NFS and SMB • ESX Server supports NFS only • Specifically, NFS version 3 carried over TCP NAS (network-attached storage) is a specialized storage device that connects to a network and can provide file access services to an ESX Server. ESX Servers use the NFS protocol to communicate with NAS servers. ESX Server supports NFS Version 3 over TCP only. CIFS stands for Common Internet File System. 170 VMware Infrastructure 3: Install and Configure How is NAS/NFS Used with ESX Server? • NFS volumes are treated just like VMFS volumes in Fibre Channel or iSCSI storage • Any can hold VMs’ files • Any can hold ISO images • Any can hold VM templates • Allow VMotion migration of a VM whose files reside on an NFS datastore 5 Storage ESX Server supports the following shared storage capabilities on NFS volumes: • Use VMotion • Create virtual machines • Boot virtual machines • Mount ISO files, which are presented as CD-ROMs to virtual machines The NFS client built into ESX Server lets you access the NFS server and use NFS volumes to store virtual machine disks. Module 5 Storage: NAS Storage and NFS Datastores 171 NFS Components The ESX Server must be configured with a VMkernel port defined on a virtual switch. The VMkernel port must be able to access the NFS Server over the network. The NFS Server contains the directory to share with the ESX Server. 172 VMware Infrastructure 3: Install and Configure Addressing and Access Control with NFS 5 Storage defines the systems allowed to access the shared directory. The options used in this file are: /etc/exports • Name of directory to be shared • Subnet(s) allowed to access the share • rw: Allows both read and write requests on this NFS volume. • no_root_squash: By default, the root user (whose UID is 0) is given the least amount of access to an NFS volume. This option turns off this behavior because the VMkernel needs to access the NFS volume using UID 0. • sync: All file writes must be committed to the disk before the write request by the client is actually completed. Module 5 Storage: NAS Storage and NFS Datastores 173 Configure Networking for an NFS Datastore Create a VMkernel port on a virtual switch You must define a new IP address for NAS use, different from the Service Console’s IP address For the ESX Server to access the NFS datastore over the network, a VMkernel port must be configured manually. The name of this port can be anything you want. In the example above, it is named "NFS Access." The VMkernel port can be created as either another connection on an existing virtual switch or as a new connection on a new virtual switch. 174 VMware Infrastructure 3: Install and Configure Configure an NFS Datastore (1 of 2) Describe the NFS share 5 Storage To configure an NFS datastore, select your ESX Server in the inventory, then click its Configuration tab. Select the Storage link. Click the Add Storage... link, then select Network File System as the storage type. Enter the properties of your NFS datastore, as shown above. There are various reasons for mounting an NFS as a read-only file system: • You want the NFS to be a library of files, such as ISO images • You do not want this file system to be space for users to place their personal files • You have a limited amount of space in the NFS and you do not want users accidentally filling up the NFS file system Module 5 Storage: NAS Storage and NFS Datastores 175 Configure an NFS Datastore (2 of 2) Verify that the NFS datastore has been added and display the datastore contents, if desired After creation, the NFS datastore shows up in the Storage display of the server's Configuration tab. From this screen, you can also display the contents of the datastore: right-click the datastore, then select Browse Datastore... from the menu. 176 VMware Infrastructure 3: Install and Configure Problem: Cannot Access NFS Datastore • Is the VMKernel port configured and on the same LAN as the storage array? • Is a second Service Console port required? • Is there an error in network configuration (storage array IP address, routing, etc.)? • Is the NAS Share name correct? • Has NFS been correctly configured (rw, no_root_squash, sync)? • Are network problems preventing access to NAS storage? 5 Storage If you are having problems accessing your NFS datastore, check your network configuration and NFS configuration. Is the VMKernel port configured and on the same LAN as the storage array? • Examine the network configuration of your ESX Server. • Make sure that you have a VMKernel port on a switch that is connected to the same LAN as the NAS storage array. • Make sure the IP address and subnet mask of the VMKernel port is correct for the storage LAN. • You should be able to ping the VMKernel address from the network storage device. Is there an error in the network configuration (storage array IP address, routing, etc.)? • Your network storage device configuration under ESX Server requires that you enter either an IP address or an FQDN for the storage array. Make sure this address and/or FQDN is correct. • If you connect to your storage via a routed network (not recommended), is your default router set for that network? Each ESX Server has a default router gateway set for all VMKernel and service console traffic. The gateway set for VMKernel does not have to be the same as the one for the service console, but there can only be one for each. All VMKernel ports use the same router and all service console ports use the same router. If you are adding a new VMKernel or service console Module 5 Storage: NAS Storage and NFS Datastores 177 port that is on a new subnet and the defined gateway address is not on that subnet, you must already have a defined VMKernel and/or service console port on the subnet on which the gateway is defined. For example, if your new VMKernel port is on subnet 10.1.161.x and your gateway address for VMKernel traffic is 192.168.161.1, you must already have a VMKernel port defined somewhere on 192.168.161.x. • You must also make sure that the designated VMKernel and service console routers are online and functioning properly. Is the NFS share name correct? • If you do not correctly configure the share name that the NFS server is using, you will not be able to connect to NFS storage. ESX Server will not allow you to create NFS storage if you enter an incorrect share name. Make sure your share name is correct, including the direction of the slash mark. Many Windows administrators are used to using the backwards slash ("\") in storage and directory names, but most NFS storage arrays require the UNIX forward slash ("/"). The NFS share name appears as Folder in the NFS add storage wizard. Has NFS been correctly configured (rw, no_root_squash, sync)? • The NFS storage array must be configured correctly. Many of these arrays are UNIX or Linux servers. The NFS storage must be shared with the rw, no_root_squash, and sync flags in order for the ESX Server to utilize it. If the storage administrator has left one of these flags off, NFS storage may work temporarily, but it will not be stable. Are network problems preventing access to NAS storage? • Network problems can prevent access to any storage array. A common problem is that other systems on the network may accidentally configure TCP/IP addresses that are identical to the NFS storage array, which will take the storage array off-line. Most NFS storage servers will answer a ping request (check with the NFS system administrator to verify this). If the NFS server answers ping requests, then you should be able to ping it from the service console. Even more importantly, the NFS storage system should be able to ping both your service console address and the VMKernel port address. 178 VMware Infrastructure 3: Install and Configure Lab for Lesson 4 • Create an NFS datastore • In this lab, you will perform the following tasks: •Create a VMkernel port to access NFS storage •Create an NFS datastore 5 Storage After students create their NFS datastore, encourage them to view the contents of the NFS datastore. To do this, right-click the NFS datastore, then select Browse Datastore. Module 5 Storage: NAS Storage and NFS Datastores 179 Lesson Summary • An NFS datastore can be used to hold templates, ISO images and the files that make up a VM • ESX Server supports NFS version 3 over TCP • A VMkernel port must be configured in order for the VMkernel to access the IP-based storage 180 VMware Infrastructure 3: Install and Configure Module Summary • A LUN is addressed by the VMkernel using the following syntax: vmhbaadapterID:targetID:LUN:partition • A VMFS datastore holds virtual machines’ files, ISO images and templates • A VMkernel port must be configured for IP storage networking, needed to access iSCSI and NAS storage • An IQN and a WWN uniquely identify nodes in an iSCSI SAN and Fibre Channel SAN, respectively 5 Storage Module 5 Storage: NAS Storage and NFS Datastores 181 Questions? Questions? 182 VMware Infrastructure 3: Install and Configure MODULE 6 Virtual Center Installation 6 6 Virtual Center Installation Importance • VirtualCenter Server allows you to centrally manage multiple ESX Servers and VMs. VirtualCenter Server also gives large-scale environments added functionality in the areas of resource balancing and high availability. Objectives for the Learner • To install VirtualCenter components • To manage an ESX Server using VirtualCenter Module Lessons • VirtualCenter Software Installation • VirtualCenter Inventory Hierarchy • Using VirtualCenter to Manage Hosts and VMs VMware Infrastructure 3: Install and Configure 183 Lesson 1 VirtualCenter Software Installation Lesson Topics • VirtualCenter Components • VirtualCenter Architecture • VirtualCenter Database • VMware License Server • VirtualCenter Server • VMware Infrastructure (VI) Client : 184 VMware Infrastructure 3: Install and Configure VirtualCenter Components The VMware VirtualCenter Server is the management server for ESX Servers and virtual machines. It is a software product consisting of numerous services and modules that is installed on a Windows server. The Windows server may be either a physical machine or a virtual machine. The same VI Client that is used as a graphical interface to manage ESX Servers is what is used to connect to VirtualCenter. Once an ESX Server is being managed by a VirtualCenter server, administrators should always use the VirtualCenter server to manage the ESX Server. The VI Client should be used to directly connect to the ESX Server in unusual circumstances, for example, the VirtualCenter Server is down or for command-line troubleshooting. It is highly recommended that the VMware License Server be installed on the same Windows server as the VirtualCenter Server. The license server provides licenses for all ESX Servers being managed by the VirtualCenter Server. The most critical component of VirtualCenter is the database. The actual VirtualCenter Server software can be installed fairly quickly. It is the database that stores the inventory items, security roles, resource pools, performance data and other critical information. Also shown is an ActiveDirectory (AD) domain. The VirtualCenter Server must be installed on a Windows platform. This means that security for the VirtualCenter Server is built on Windows security. The VirtualCenter Server is not required to belong to an ActiveDirectory domain. However, if the server VirtualCenter is running on is a member of an ActiveDirectory Module 6 Virtual Center Installation: VirtualCenter Software Installation 185 6 Virtual Center Installation domain, then user accounts and groups from the domain will be available on the VirtualCenter Server. If the VirtualCenter server is NOT a member of a domain, the Windows security architecture is still in force. Only now the users and groups that can use VirtualCenter will be local Windows users and groups. This has profound security implications for administration of the Virtual Infrastructure. For example, by default, anyone with Domain Administrator privileges in the AD domain will have full administrative powers over all ESX Servers and virtual machines that are being managed by VirtualCenter. Virtual Infrastructure administrators will need to plan and coordinate security carefully with Windows ActiveDirectory administrators. During normal operations, VirtualCenter is listening for data from its managed hosts and clients on designated ports. The VirtualCenter Server must communicate with the database server and the license server. If there is a firewall between any of these elements, ports must be opened to allow communications. Some default ports include: • VirtualCenter to License Server: 27000 and 27010 • VirtualCenter to Database Server: Oracle, use 1521; SQL, use 1433 • WebAccess and SDK Clients to VirtualCenter: 443 and 80 • VirtualCenter to managed hosts: 902 • VI Client to VirtualCenter: 443 Many of these default settings can be changed. Other services such as NFS and iSCSI require other open ports. For more information, consult the Installation and Upgrade Guide and the Basic System Administration Guide, available on the VMware Web site. 186 VMware Infrastructure 3: Install and Configure Optional Features • Included with VirtualCenter Server, but require a separate license to activate: • VMware VMotion •Allows migration of running Virtual Machines • VMware HA •High Availability •Fault-Tolerance • VMware DRS •Dynamic Resource Scheduler •Load Balancing VirtualCenter Server also has some optional features. These are packaged and installed with the base product, but require a separate license. Optional features include: • VMware VMotion – A feature that enables you to move running virtual machines from one ESX Server to another without service interruption. It requires licensing on both the source and target host. The VirtualCenter Server centrally coordinates all VMotion activities. • VMware HA – A feature that enables a cluster with high availability. If a host goes down, all virtual machines that were running on the host are promptly restarted on different hosts in the same cluster. • VMware DRS – A feature that helps improve resource allocation across all hosts and resource pools. VMware DRS collects resource usage information for all hosts and virtual machines in the cluster and gives recommendations for VM migration or actually migrates the virtual machines (depending on automation settings). 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 187 VirtualCenter Architecture Distributed Services Additional Services Core Services Modules Third-party Applications Active Directory Interface ActiveDirectory Domain User Access Control VMware Web SDK Third-party Applications Database Interface ESX Server Management VirtualCenter Database ESX Hosts Third-party Applications The VirtualCenter architecture consists of the following services and interfaces: • Core services: The core functionality of the VirtualCenter server, such as management of resources and virtual machines, task scheduler, statistics logging, management of alarms and events, VM provisioning and host and VM configuration • Distributed services: Additional functionality of the VirtualCenter server, for example, VMotion, VMware DRS and VMware HA. They are installed with the VirtualCenter Server, but require a separate license to activate. • Additional services: Additional functionality, packaged separately from the base product and require separate installation. No additional license is necessary. • Database interface: Provides access to the VirtualCenter database • ESX Server management: The VirtualCenter Server provides access to the ESX Server using a VirtualCenter Agent, which is installed on the ESX Server when it is added to VirtualCenter’s inventory. The VirtualCenter Agent communicates with the Host Agent to relay the tasks to perform on the ESX Server. The Host Agent, like the VirtualCenter Agent, reside on the ESX Server. • Active Directory interface: Provides access to domain user accounts • VI API: Along with VI SDK, provides an interface for writing custom applications that access VirtualCenter functionality. 188 VMware Infrastructure 3: Install and Configure VirtualCenter Modules • Applications that provide additional features and functionality to VirtualCenter • Examples • VMware Update Manager • VMware Converter Enterprise for VirtualCenter • Include a server component and a client component. • Client component is available to VirtualCenter clients for download after module installed on VirtualCenter Server • Alters the interface by adding items related to the enhanced functionality • Can be upgraded independently VirtualCenter modules are applications that provide additional features and functionality to VirtualCenter. The VirtualCenter modules are: • VMware Update Manager – Enables security administrators to enforce security standards across ESX Servers and all managed virtual machines. This module provides the ability to create user-defined security baselines which represent a set of security standards. Security administrators can compare hosts and virtual machines against these baselines to identify and remediate systems that are not in compliance. • VMware Converter Enterprise for VirtualCenter – Enables users to convert physical machines, and virtual machines in a variety of formats, to ESX Server virtual machines. Converted systems can be imported into any location in the VirtualCenter inventory. Typically, modules are comprised of a server component and a client component. After the server component of a module is installed, it is registered with the VirtualCenter Server and the client component is available to VirtualCenter clients for download. The client component is also known as a “plug-in”. After a plug-in is installed on a VirtualCenter client, it might alter the interface by adding views, tabs, toolbar buttons, or menu options related to the enhanced functionality. Modules leverage core VirtualCenter capabilities, such as authentication and permission management, but can have their own types of events, tasks, metadata, and privileges. Modules require VirtualCenter and they can be installed anytime after VirtualCenter has been installed. Modules and VirtualCenter can be upgraded independently. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 189 Order of Installation • Check Hardware and Software prerequisites • Database Server • Create database connection to either SQL Server or Oracle database • License Server • VirtualCenter Server • VMware Infrastructure Client This is the recommended order of installation. If you are using either SQL Server or Oracle, the database instance must be created for VirtualCenter before performing the installation. The VirtualCenter installer populates the database with VirtualCenter tables and views. The License Server can be installed before or during the VirtualCenter Server installation (the VirtualCenter Server installer wizard prompts for and will install a license server if one is not already installed.) The Virtual Infrastructure Client can be installed at any time. There is an order you must follow during the VirtualCenter Server installation: 1 2 3 4 5 6 Begin by sure your hardware and software meet the required prerequisites. Create a database on either a supported Microsoft SQL or Oracle Database server. Create a database connection to your database, either SQL Server or Oracle. Install the VMware License Server. Install the VirtualCenter Server. Install the VMware Infrastructure Client. 190 VMware Infrastructure 3: Install and Configure Hardware and Software Prerequisites • Hardware Requirements • Processor – 2.0GHz or higher Intel or AMD x86 processor • Memory – 2GB RAM minimum • Disk storage – 560MB minimum, 2GB recommended • Networking – 10/100 Ethernet adapter minimum (Gigabit recommended) • May be run in a Virtual Machine • Software Requirements • 32-bit version operating system only: • Windows 2000 Server SP4 with Update Rollup 1 • Windows XP Pro SP2 • Windows 2003 Server SP1 • Windows 2003 Server R2 VirtualCenter Server hardware must meet the following requirements: 6 • Processor – 2.0GHz or higher Intel or AMD x86 processor. Processor requirements can be larger if your database is run on the same hardware. • Memory – 2GB RAM minimum. RAM requirements can be larger if your database is run on the same hardware. • Disk storage – 560MB minimum, 2GB recommended. You must have 245MB free on the destination drive for installation of the program, and you must have 315MB free on the drive containing your %temp% directory. • Networking – 10/100 Ethernet adapter minimum (Gigabit recommended). The VirtualCenter Server is supported as a service on the 32-bit versions of these operating systems: • Windows 2000 Server SP4 with Update Rollup 1 (Update Rollup 1 can be downloaded from http://www.microsoft.com/windows2000/server/ evaluation/news/bulletins/rollup.mspx) • Windows XP Pro SP2 • Windows 2003 Server SP1 • Windows 2003 Server R2 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 191 VirtualCenter Database • Storage area for maintaining VirtualCenter inventory as well as the status of each VM and each managed host • Oracle • 9iR2 • 10gR1 (versions 10.1.0.3 and higher) • 10gR2 • Microsoft SQL • SQL Server 2000 (SP4 or Enterprise) • SQL Server 2005 (Enterprise SP1 or SP2) • Microsoft SQL Server 2005 Express • Default evaluation/demo database • Used for demonstration installations • Bundled with VirtualCenter MSDE is no longer Before you install VirtualCenter Server make sure you have your database supported. It have been ready. The following is a list of the supported databases and their replaced with SQL Server requirements: 2005 Express. SQL Server 2005 Express should be used only in a demo environment. When the database size is limit is reached, performance issues and general “weirdness” will occur, such as VMs powering off, stats not being collected, and VMs no longer registered. With that said, SQL Server Express Edition is typically for small environments, for example, <= 5 ESX Servers. This topic comes up occasionally in class. Users are running small environments without a regular database server who do not want to spend money to get a database for VirtualCenter. • Oracle 9iR2, 10gR1 (versions 10.1.0.3 and higher), 10gR2 • Microsoft SQL Server 2000 (Standard SP4 or Enterprise): This will require MDAC 2.8 on the client. The SQL Server driver will be required on the client. • All versions of Microsoft SQL Server 2005 (Enterprise SP1, SP2, and Express) are supported. All require MDAC 2.8 on the client and the SQL native client driver on the client. • Microsoft SQL Server 2005 Express: SQL Server 2005 Express should only be used in demonstration environments, not production environments. The database size limit of SQL Server 2005 Express is 4 GB. When the limit is reached, VirtualCenter will experience performance issues. For more details on the VirtualCenter software and hardware requirements, consult the ESX Server 3 Installation Guide, available on the VMware Web site. 192 VMware Infrastructure 3: Install and Configure Calculating the Database Size •VirtualCenter has a built-in database calculator -> •Administration VirtualCenter Management Server Configuration •Select •No Statistics actual database changes are made •This is a “what-if” calculator The size of the database varies with the number of hosts and virtual machines you manage. To ensure your database can handle the statistics collection you configure, the VI Client provides you with a database estimation calculator in which you enter the number of hosts and virtual machines in your inventory. The calculator uses these numbers to determine how much database space is required for the collection interval configuration you defined. This ensures you have necessary resources. To use the calculator: • Select the Administration from the menu bar. • Select VirtualCenter Management Server Configuration from the menu bar. • Select the Statistics option in the left window. • Make your changes in the right-hand window. The calculator will automatically make an estimate based on your changes. Remember that this is a “what-if” calculator. No actual changes are being made to the size of the VirtualCenter database. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 193 Database Access Requirements • If you are using Microsoft SQL Server • Database user needs either a sysadmin server role or the db_owner fixed database role on the VirtualCenter database • Create an ODBC connection of type, System DSN • Use SQL Server Authentication unless the SQL Server is installed on the same server as VirtualCenter If you are using Microsoft SQL Server the database user must be assigned either a sysadmin server role or the db_owner fixed database role. For Microsoft SQL Server, you will need to create an ODBC connection. This needs to be done prior to starting the VirtualCenter installation process. The ODBC connection should be created as a System DSN connection. If you are using Microsoft SQL Server, always use SQL Server Authentication unless the SQL Server is running on the same system as the VirtualCenter Server. If both your Microsoft SQL database server and your VirtualCenter server are running on the same computer, you may use Windows Authentication. Installing the database on the same system as the VirtualCenter Server is not recommended unless the hardware (either virtual or physical) is sized with enough capacity to handle both applications. For details on setting up an Oracle database for VirtualCenter, consult the ESX Server 3 Installation Guide, available on the VMware Web site. 194 VMware Infrastructure 3: Install and Configure VMware License Server (Centralized Licensing) License Server ESX Servers VirtualCenter Server • Simplifies license management • Licenses are stored on a license server • Makes licenses available to one or more ESX hosts • License Editions: VirtualCenter Foundation and VirtualCenter • Single-host and centralized licensing can be combined • 14-day grace period 6 The VMware License Server is a distributed license system, based on technology licensed from industry-standard FlexNet. A license server can be set up in your datacenter for your VMware software. The VMware License Server can be installed at the same time the VirtualCenter Server is installed. VMware strongly recommends that you follow the default installation and place your license server on the same machine as your VirtualCenter Server. This has the advantage of simplicity of setup, as well as guaranteeing VirtualCenter-to-license server communications. Change this only if you have a good reason, such as an existing FLEXnet license server. It is also possible to run VirtualCenter using a 60-day evaluation license. There are two VirtualCenter editions: • VirtualCenter Foundation: This edition lets you manage up to three ESX Servers. If you need to manage more than three hosts, upgrade to VirtualCenter edition. • VirtualCenter: This enterprise-level edition lets you manage up to the system maximum number of hosts. Using single host licenses for ESX Server features and centralized license server licensing for VirtualCenter features in the same environment is permitted. However, doing so requires changes to the default VirtualCenter configuration settings. If you do not change the VirtualCenter settings, the settings can override single host license files: • When the VirtualCenter Server restarts • When the single host ESX Server machines are added to inventory again Virtual Center Installation There is also a la carte add-ons: DRS, VMotion, Storage VMotion and HA. Transition to the next slide with the last bullet, 14-day grace period. Module 6 Virtual Center Installation: VirtualCenter Software Installation 195 Any single host license file on the ESX Server machine remains unchanged but ignored. For more details on this, consult the Installation Guide, available on the VMware web site. Additional Information on Licensing: For most VMware Infrastructure products, you purchase licenses on a per-processor basis, which means that you need to indicate the total number of processors, not hosts, that will run the products. You can then deploy and redeploy the purchased processor capacity, sold in increments of two processors, on any combination of hosts. Special considerations include: • Dual-core and quad-core processors, such as Intel x86 processors that combine two or four independent central processing units on a single chip, count as one processor. • You cannot partially license a multiprocessor host. For example, a 4-CPU host requires a license with the capacity for four processors. For example, suppose you were to purchase ESX Server licenses for ten processors and VMotion licenses for six processors. You can then deploy those licenses on any of the following combinations of servers: • ESX Server on five 2-processor hosts. Enable VMotion on three hosts. • ESX Server on three 2-processor hosts and a 4-processor host. Enable VMotion on three 2-processor hosts, or on one 2-processor host and a 4-processor host. • ESX Server on two 4-processor hosts and one 2-processor host. Enable VMotion on one 4-processor host and one 2-processor host. • ESX Server on one 8-processor host and one 2-processor host. VMotion between these two hosts cannot be enabled unless you purchase an additional 4-processor VMotion license. 196 VMware Infrastructure 3: Install and Configure License Server 14-day Grace Period Component Attempted Action Grace Period Permitted Permitted Permitted Permitted Permitted Permitted Permitted Permitted Permitted Not Permitted Permitted Permitted Permitted Not Permitted Permitted Permitted Permitted Permitted Not Permitted Not Permitted After Grace Period Not Permitted Permitted Permitted Permitted Permitted Permitted Permitted Permitted Permitted Not Permitted Permitted Permitted Permitted Not Permitted Permitted Not Permitted Not Permitted Not Permitted Not Permitted Not Permitted Power On Create/Delete Virtual Machine Suspend/Resume Configure Virtual Machine with VI Client Continue Operations ESX Server Power On/Power Off Host Configure ESX Server Host with VI Client Modify Host-Based License File Remove an ESX Server Host from Inventory Add an ESX Server Host to Inventory Connect/Reconnect to an ESX Server Host in Inventory Cold Migrate a VM Between Hosts VirtualCenter Move an ESX Server Host Among Folders Server Move an ESX Server Host into/out of Cluster Configure VirtualCenter with VI Client Hot Migrate (VMotion) a VM Between Hosts Continue Load Balancing within VMware DRS Cluster Restart VMs within Failed Host’s VMware HA Cluster Add or Remove License Keys Any Component Upgrade There is a 14-day grace period during which hosts continue operation, relying on a cached version of the license state, even across reboots. After the grace period expires, certain ESX Server operations, such as powering on virtual machines, become unavailable. The table above shows you what operations are permitted during and after the grace period. The VMware License Server is a distributed license system, based on technology licensed from industry-standard FlexNet. A license server can be set up in your datacenter for your VMware software. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 197 VirtualCenter Server Services • VirtualCenter Server is installed on a Windows system • Once installed, services can be managed from the Windows control panel or Administrative Tools / Services Once you have your database setup and your license server configured you may install VirtualCenter Server. Once VirtualCenter Server is installed, a number of new services will appear in the Windows system: • VMware Capacity Planner Service • VMware Converter Enterprise Service • VMware Infrastructure Web Access: Allows users to manage VMs using a web browser • VMware License Server • VMware Mount Service for VirtualCenter: Service used during guest OS customization (during cloning a VM or deploying a VM from a template) • VMware Update Manager Service • VMware VirtualCenter Server: The heart of VirtualCenter, it centrally manages all tasks performed on the ESX Server and virtual machines If the Windows OS that VirtualCenter Server is running on top of is a member of a Windows Domain (either NT4 or Active Directory), it will automatically access all Windows user and group accounts in that (and any trusted) Windows Domains. 198 VMware Infrastructure 3: Install and Configure Additional Information on the VMware Virtual Mount Manager Extended Service: The VMware Virtual Mount Manager Extended service does not get started by default. It is started the first time a guest OS customization is done. The VMware VirtualCenter Server service is the main service. It starts vpxd.exe. If this service is not started, then the management server is considered to be down and unavailable. During the guest OS customization procedure, the VM needs to be configured with unique system information (such as network identity, timezone, ownership, license information, etc.). VirtualCenter uses sysprep to perform the customization. After the VM is cloned or deployed from a template, the virtual machine's virtual disk is remotely mounted onto the VirtualCenter Server to allow file system modification, specifically, the sysprep-related files need to be copied into the guest OS. The VMware Virtual Mount Manager Extended service is responsible for mounting the virtual disk so that it accessible by the VirtualCenter Server. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 199 VMware Infrastructure (VI) Client Overview • The VI Client is a graphical user interface used to access VirtualCenter • Access to VirtualCenter • Full VirtualCenter functionality • Access directly to ESX Server • Single host management only • Connect directly to managed ESX host only in unusual circumstances The VI Client is the interface used to communicate with the VirtualCenter server, just as it was used to communicate directly with an individual ESX Server. The VirtualCenter Server then passes commands to the managed ESX Server. For example, It is possible to corrupt a cluster’s configuration by making changes directly on a host that is being managed by a VirtualCenter server. This will be discussed later on. Once an ESX Server is being managed by a VirtualCenter Server, it is a good practice not to connect the VI Client directly to the ESX Server except in unusual circumstances, for example, the VirtualCenter Server is down or for command-line troubleshooting. 200 VMware Infrastructure 3: Install and Configure ESX Server and VirtualCenter Communication 6 The VI Client and the Web Client are the user interfaces used to access either the VirtualCenter Server or the ESX Server directly. The Web Client provides a browser-based interface for managing VMs. The Web Client connects to Web Access, which is available on both the VirtualCenter Server and the ESX Server. The VirtualCenter Server passes commands to the ESX Servers via the vpxa daemon. A daemon is found on Linux and UNIX systems and is similar to a Windows service. If you are using the VI Client to communicate directly with an ESX Server the vpxa daemon is not used. Instead, communications go directly to the vmware-hostd daemon. vmware-hostd is often referred to as the host agent. NOTE Virtual Center Installation In the example above, hostd represents the host agent and is an abbreviation for vmware-hostd. The following ports are used for communication: • VI Client to ESX Server traffic: 443 • VI Client to VirtualCenter traffic: 443 • VI Web Access Client to either VirtualCenter or ESX Server: 443 • Authentication traffic for the ESX Server: 902 • ESX Server-to-ESX Server access for migration and provisioning: 902 Module 6 Virtual Center Installation: VirtualCenter Software Installation 201 For more details on TCP and UDP ports used for management access, consult for ESX Server 3 Configuration Guide, available on the VMware Web site. 202 VMware Infrastructure 3: Install and Configure Managing Across Geographies A single VirtualCenter Server can manage ESX Servers that are located in different geographical locations but connected by a WAN link or VPN link. The bandwidth required for communications between the VirtualCenter server and the hosts is very small. Likewise, administrators who are using the VI Client do not have to be in the same geographical location that the VirtualCenter server is located in. When you manage ESX Servers separated by WAN and VPN links make sure that any firewalls in-between are configured to allow the required TCP ports to be open. Communication links between the clients and the VirtualCenter Server are encrypted. For more information on the required TCP ports that need to be open, consult the Installation Guide, section “Configuring Communication Between VirtualCenter Components”, available on the VMware Web site. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 203 Backup Strategy for VirtualCenter Server • If primary management server fails: • Power off primary management server • Power on standby management server The standby server must be an exact copy of the primary server. Host name, IP address, and SSL identity must be the same. This is becaue the VirtualCenter configuration file, /etc/vmware/vpxa.cfg contains a hard reference to the IP address of the VirtualCenter Server. If the VirtualCenter Server fails, it will not affect the runtime behavior of the virtual machines and the ESX Servers. The ESX Servers and virtual machines continue to run normally. When the VirtualCenter Server comes back up, it can reconnect to running hosts and re-synchronize the state of the hosts and their virtual machines. One possible strategy for VirtualCenter Server high availability is to create a standby VirtualCenter Server, either on a physical machine or in a virtual machine. The standby VirtualCenter Server is an exact copy of the primary VirtualCenter Server. Leave it powered off, until it needs to take the place of the primary VirtualCenter Server. Multiple VirtualCenter Servers are not allowed to manage the same inventory at the same time. You can also use clustering software so the VirtualCenter server process is automatically restarted on a standby server if the primary server fails. For more information, consult the technical paper, Using MSCS to Cluster VirtualCenter, available on the VMware website at http://www.vmware.com/pdf/VC_MSCS.pdf. Another strategy is to use the clustering capabilities of the database itself, which both SQL Server and Oracle provide. 204 VMware Infrastructure 3: Install and Configure Problem: Cannot Login Using VI Client (1 of 2) • One reason: VMware VirtualCenter Server service is not running 6 If you try to login to your VirtualCenter Server using the VI Client and are unable to, there are several possible reasons: • You are using an incorrect VirtualCenter Server hostname or IP address • You are using an incorrect user account and/or password • Local station has no IP connectivity to the VirtualCenter Server - Try to ping the VirtualCenter Server from the system on which you are running the VI Client • VirtualCenter Server Service (vpxd) is not running - Check that this service is running on the VirtualCenter server Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 205 Problem: Cannot Login Using VI Client (2 of 2) • Check the Windows Services • Start the VirtualCenter Server service if it has stopped VirtualCenter Server service is currently stopped To verify whether or not the VirtualCenter Server service is started, view the Windows services panel on the VirtualCenter Server system. The VMware VirtualCenter Server service should be started. If it is not, then start it at this time by clicking the Start link in the Services window. 206 VMware Infrastructure 3: Install and Configure VirtualCenter Server Service Fails To Start • Use the Windows Event Viewer on the VirtualCenter Server to check the event logs • View the VirtualCenter Server log file, vpxd-#.log • Look for messages of type “error” • Check the VirtualCenter Server database log files (Oracle or SQL Server) • Service failure could be caused by database problems By default, the VirtualCenter Server service (vpxd) starts automatically when the VirtualCenter Server boots up. However, if the VirtualCenter Server service fails to start, this problem must be resolved. Without this service, you cannot manage your ESX Servers and virtual machines using VirtualCenter. However, you will still be able to manage your ESX Servers and their virtual machines if you use the VI Client and log directly into the ESX Server. Use the Windows event viewer to view the application log. VirtualCenter events are always identified with event ID 1000 and they are always prefixed with a warning that this event description is missing in the local computer. The text of the VirtualCenter event message itself follows this warning. Also, check the Windows system log in the event viewer for any messages on starting, stopping or recovering the VirtualCenter Server service The VirtualCenter Server service has log files that you can view using the VI Client. There is one catch - in order to use the VI Client, the VirtualCenter Server service must be started! The VirtualCenter Server log files are named vpxd-#.log, where # is a number from 0 to 9. VirtualCenter keeps a maximum of 10 log files and rotates through them. The file vpxd-index.log always lists the number of the current log. If the Windows administrator account was used to install and run VirtualCenter, then the log files and index file are located in C:\Windows\temp\vpx. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 207 Failure of the VirtualCenter Server service can also be caused by problems accessing the VirtualCenter database. Check the following: • You are using a supported database with VirtualCenter. • The VirtualCenter database is accessible. Network issues, operating system issue, and authentication issues on the database server can prevent VirtualCenter from accessing its database. • The VirtualCenter Server can connect successfully to its database. • If you are using a SQL Server database, the ODBC connection is working properly. • The database’s transaction logs are not full. 208 VMware Infrastructure 3: Install and Configure License Server Will Not Start • Use the VMware License Server Tools • Stop/start the service, apply or re-read license file If the license server does not start, the most likely cause is an incorrect or corrupt license file. Knowledge Base article 1013698 provides the syntax description for your license files, both host- and server-based. The link to this article is http://kb.vmware.com/kb/1013698. To check the configuration and status of your license file, use the VMware License Server Tools utility, also known as LMTOOLS. To launch this utility on the license server, click Start -> All Programs -> VMware -> VMware License Server -> VMware License Server Tools. If you are having license server problems, here are some things to check using LMTOOLS: • Click the Server Status tab to check the status of the license server, and verify the path to the license file being used is correct. • Click the Perform Status Enquiry button and view the messages about your licensing. • If the wrong license file is being used, click the Config Services tab. In the “Path to the license file” field, browse to the correct license file. Click the Save Service button after making the change. Go to the Start/ Stop/Reread tab to re-read the license file. The link to any KB article is http://kb.vmware.com/ kb/#######, where ####### is the KB article ID. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 209 Lab for Lesson 1 • Install VirtualCenter • In this lab, you will perform the following tasks: •Open a Remote Desktop Connection (RDC) to your VirtualCenter Server •Create an ODBC connection to the SQL Server VirtualCenter database •Install the VMware License Server •Install the VirtualCenter Server •Install the VMware Infrastructure Client (Optional) Tips for all lab environments: • If a student is working in their VirtualCenter VM and they cannot use their mouse to select something in the VM's toolbar, then student might need to type CTRL-ALT to release the mouse from the VM's console window. Tips for classes that use a VDC Kit only: • After the lab is over, remind students that they will get better performance if they use the VI Client installed on the Citrix desktop instead of the desktop of their VirtualCenter VM. • Show the students how to access their VirtualCenter VM using a Remote Desktop Connection: In the Citrix desktop, click Start -> Utilities -> Remote Desktop Connection. Provide each student with the IP address of their VirtualCenter VM. • A common error in setting up the ODBC connection is to overlook the step to change the default database from 'master' to the new data source being created (usually 'kitnamexx'). • Another common error is to not switch from the User DSN tab to the System DSN tab when creating the ODBC connection. 210 VMware Infrastructure 3: Install and Configure Lesson Summary • Oracle and SQL Server are the recommended databases to use for VirtualCenter in a production environment • The VMware License Server and VirtualCenter Server typically reside on the same system • If the license server fails, licenses remain unaffected for a 14-day grace period 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Software Installation 211 Lesson 2 VirtualCenter Inventory Hierarchy Lesson Topics • VirtualCenter inventory hierarchy • Adding ESX Server to inventory : 212 VMware Infrastructure 3: Install and Configure VirtualCenter Inventory: Multiple Datacenters Where networks and datastores are configured The VirtualCenter inventory is a hierarchy of objects. These objects are either containers of other objects, such as folders, or objects that you manage, such as hosts and virtual machines. The inventory hierarchy is used to group your hosts and virtual machines in a meaningful way. It also provides a natural structure upon which to apply permissions. The topmost object in the inventory is a folder, also known as the root folder or root. By default, the root folder has the name “Hosts and Clusters” or “Virtual Machines and Templates”, depending upon the current inventory view. Under the root folder, one or more datacenter objects are created. A datacenter is the primary container of inventory objects such as hosts and virtual machines. From the datacenter you can add and organize inventory objects. Typically you will add hosts, folders, and clusters to a datacenter. VirtualCenter Server can contain multiple datacenters. Large companies might use multiple datacenters to represent organizations or business units within the corporation. Inventory objects can interact within datacenters, but have only limited interaction across datacenters. For example, you can VMotion a virtual machine from one host to another within a datacenter, but not to a host in a different datacenter. On the other hand, you can clone a virtual machine within a datacenter and to a different datacenter. In the example above, datacenters are based on their geographical location, where each geographical location might have its own team of IT In 3.5, you can clone a virtual machine from one datacenter to a different datacenter. This is covered in module 6, VM Creation and Management. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Inventory Hierarchy 213 administrators, its own set of customers and its own set of ESX Servers, virtual machines, networks and datastores for which it is responsible. 214 VMware Infrastructure 3: Install and Configure VirtualCenter Inventory: Folders and Subfolders 6 Items within the inventory may be placed into folders. Folders and subfolders may be created to better organize systems. In the example above, datacenters are grouped by areas in the world, such as the Americas and Europe. By doing this, you create a structure upon which appropriate access can be assigned to administrators. Take care when you design your inventory. Too many sub-levels and too complicated a hierarchy can make management harder instead of easier. Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Inventory Hierarchy 215 Organizing Objects in the Inventory Here is a datacenter that contains a collection of virtual machines, templates, and ESX Servers that have been placed into folders for organizational purposes. Note that hosts are being separated by CPU architecture. This is will make organizing DRS cluster configurations easier. It is not possible to VMotion a virtual machine from a host running one CPU architecture to a host with a different architecture. This will be covered in depth during the module on Resource Management. 216 VMware Infrastructure 3: Install and Configure VirtualCenter Inventory: Clusters run nin g Cluster for VMware HA on ng ni n ru on Cluster for VMware DRS An ESX Server serves as the platform on which virtual machines run. Hosts that are not grouped together are known as standalone hosts. Hosts that are grouped together are referred to as a cluster. Even though resources can be pooled together in clusters, bear in mind that a VM can only run on a single host at a time. The example above shows two clusters, one which is a VMware HA cluster and one which is a VMware DRS cluster. In best practice, most clusters are both VMware HA-enabled and VMware DRS-enabled. VirtualCenter Server can support VMware DRS and VMware HA clusters which contain up to 32 ESX Servers. As of version 2.5, 32 hosts are supported. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Inventory Hierarchy 217 View VirtualCenter Inventory Hosts & Clusters View Virtual Machine & Templates View This graphic shows the two most common views used in the VirtualCenter Inventory: the Hosts & Clusters view and the Virtual Machines & Templates view. The other two views are the Networks view and the Datastores view. To display a view in the VI Client, select the desired view in the Inventory panel’s drop-down menu. Note that you cannot see templates in the Hosts & Clusters view. It is possible to see templates in this view by selecting the Hosts & Clusters folder and selecting the Virtual Machines tab. Also note that you cannot see hosts or clusters in Virtual Machines & Templates view. It is possible to see hosts in this view by selecting the Virtual Machines & Templates folder and clicking on the Hosts tab. 218 VMware Infrastructure 3: Install and Configure Problem: Cannot Add Host to Inventory (1 of 2) • ESX Server management agent (vmware-hostd) may not be running 6 If you cannot add an ESX Server to the VirtualCenter inventory, here are some possible reasons: • You are using an incorrect ESX Server IP address or hostname - Try to ping the IP address or host name that you are using. • You are using an incorrect user name and/or password - Make sure you are using the root user account and password. • The ESX Server management agent (vmware-hostd) is not running Check that vmware-hostd is running. Checking to see if vmware-hostd is running is discussed on the next slide. Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Inventory Hierarchy 219 Problem: Cannot Add Host to Inventory (2 of 2) • Check that hostd is running • Run ps –ef | grep hostd from the ESX Server command line • If hostd is not running • Run service mgmt-vmware start to restart hostd To check if vmware-hostd is running, you must use the service console command line. Log into the ESX Server from the ESX Server machine console or a secure shell session. Log in as a normal user account, then use the su - command to switch to user root. Run the command, ps -ef | grep hostd. This command line lists the processes currently running on the ESX Server but only displays entries that contain the word “hostd”. In the example above, there were two entries that contained the word “hostd” (Note these are long entries that span two lines). The second entry shows that vmware-hostd is indeed running. If vmware-hostd is not running, run the command, service mgmt-vmware start to restart vmware-hostd. 220 VMware Infrastructure 3: Install and Configure Problem: ESX Server Not Responding • If you are in the VI Client and the ESX Server’s state changes to “not responding” • The VirtualCenter Server lost connection to the ESX Server •Check service console network connectivity • If you are in the VI Client and you get the error message: • The VirtualCenter Server Service may have stopped If you are working in the VI Client and an ESX Server’s state changes to “not responding”, here are possible reasons: • The VirtualCenter Server lost network connection to the ESX Server. If this is the case, check the service console network connectivity as well as the network connectivity on the VirtualCenter Server. • The VI Client may have lost connection to the VirtualCenter Server. If this is the case, check network connectivity from the system where the VI Client is running to the VirtualCenter Server. • The VirtualCenter Server Service may have stopped. 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Inventory Hierarchy 221 Lab for Lesson 2 • Add ESX Server to VirtualCenter Inventory • In this lab, you will perform the following tasks: •Add an ESX Server to the VirtualCenter inventory •Configure the ESX Server to use the License Server •View general information about the ESX Server 222 VMware Infrastructure 3: Install and Configure Lesson Summary • A datacenter object is the primary organizational structure in the inventory • Folders can be used to organize ESX Servers in the datacenter 6 Virtual Center Installation Module 6 Virtual Center Installation: VirtualCenter Inventory Hierarchy 223 Lesson 3 Using VirtualCenter to Manage Hosts and VMs : Lesson Topics • Lockdown Mode • Scheduled Tasks • Administration • Events • System Logs • Maps • Consolidation • Plugins • Client Settings 224 VMware Infrastructure 3: Install and Configure Lockdown Mode • Prevents administrators from trying to directly manage ESX hosts that are already being managed by a VirtualCenter Server Use the VI Client to directly manage ESX Servers that are under VirtualCenter administration only in case of unusual circumstances, for example, for command-line troubleshooting. Making changes on an individual ESX Server’s configuration when it is under VirtualCenter Server administration can cause major problems. These include causing clusters and resource pools to become internally inconsistent. Once an ESX Server is being managed by a VirtualCenter server, you may prevent anyone from managing it directly by placing it into Lockdown Mode. This can be done when the host is first added to the VirtualCenter Server’s inventory, or later by modifying the Security Profile. To modify the Security Profile, select your ESX Server from the inventory, then click its Configuration tab. In the Software section, click the Security Profile link, then click Edit... next to the Lockdown Mode section. A check box allows you to either enable or disable lockdown mode. 6 Virtual Center Installation Module 6 Virtual Center Installation: Using VirtualCenter to Manage Hosts and VMs 225 Scheduled Tasks • Scheduled Tasks can schedule many key operations on virtual machines and hosts Click on the Scheduled Tasks panel to see the scheduled tasks. If you rightclick anywhere in the Scheduled Tasks window, you may add a new task. Tasks include operations like powering down a virtual machine at a specific time. You can also schedule resource-intensive operations such as the deployment of new virtual machines from templates at off-hours. 226 VMware Infrastructure 3: Install and Configure Administration • Determine who has logged-in to this VirtualCenter Server • Send messages to other administrators The Sessions tab is located in the Administration panel. It is a very convenient way to see who is logged into this VirtualCenter Server. It is also possible to send real-time message broadcasts to all VI Clients that are connected to this VirtualCenter Server. Other tabs available from the Administration button include Roles, Licenses, and System Logs. Roles will be covered later in the discussion on security. 6 Virtual Center Installation Module 6 Virtual Center Installation: Using VirtualCenter to Manage Hosts and VMs 227 Events Event Search Details of selected event An event is the outcome or result of running a VirtualCenter task. The Events panel can show you recent events in VirtualCenter. The search option allows you to search for specific events that are not on the screen. Once you select an event, the details window shows you more detailed information about the event. Another way to display events is to select any object in the inventory, then click its Tasks & Events tab. This view allows you to see the tasks and events related to that specific object. Being able to view VirtualCenter events can be very useful when troubleshooting problems. 228 VMware Infrastructure 3: Install and Configure System Logs vpxd-index identifies the current log file Log Search The System Logs tab is another important tab on the Administration button. Logs can be searched in the same manner as events. Like events, the system logs can be useful when troubleshooting problems. In the drop-down list at the top of the System Logs tab, you will see all the logs that are currently available for viewing. The file named vpxd-index is a file that contains the number, or index, of the most current log. Look at this file first to determine the most current log that VirtualCenter is using. In the example above, vpxd-index contains the number 3. This means that the most current log file is vpxd-3.log. If the VI Client were connected directly to the ESX Server, then local ESX Server logs will be available instead. 6 Virtual Center Installation Module 6 Virtual Center Installation: Using VirtualCenter to Manage Hosts and VMs 229 Maps • The Maps button shows relationships between virtual machines, hosts, datastores, and networks The Maps panel provides a visual understanding of the relationships between the virtual and physical resources available in VirtualCenter inventory. These inventory maps show a selected item's relationships with relevant hosts, virtual machines, networks and datastores. Maps make it easy to understand what items are affected or attached to the item in question. Maps are a visual way of verifying that the VMotion requirements relating to networks and datastores are met by a particular set of hosts. You can zoom in and out of the topology map. In the Overview window, use the mouse to select all or part of the map. You can also hold the mouse button down in that window to move the box cursor over the part of the map that you would like to enlarge. Examples of using maps will be discussed in a later module. 230 VMware Infrastructure 3: Install and Configure Consolidation • The Consolidation button will launch the Guided Consolidation feature The Consolidation panel will launch the Guided Consolidation feature. This feature enables you to consolidate physical systems in your datacenter by converting them to virtual machines and importing them into VirtualCenter. Multiple virtual machines can be hosted on a single physical system, enabling more efficient use of computing resources. Consolidating your datacenter involves the following process: • Discover – In this phase, physical systems in your datacenter are discovered and you select the systems you want analyzed. • Analyze – In this phase, the selected physical systems are analyzed for their potential as candidates to be converted to virtual machines. Performance data on each selected system is collected. The performance data is compared to the resources available on the virtual machine host systems to determine the degree to which a physical system makes a good candidate for consolidation into the virtual environment. Generally, the longer the duration of the analysis phase the higher the confidence in the VirtualCenter’s recommendations. • Consolidate – In this phase, the consolidation plan is executed. The selected physical systems are converted to virtual machines and imported into VirtualCenter onto the recommended hosts where they are managed along with other components of your virtual environment. The VMware Converter Enterprise plug-in is necessary to do the conversion. NOTE 6 Virtual Center Installation Guided Consolidation will be covered later in the course. Module 6 Virtual Center Installation: Using VirtualCenter to Manage Hosts and VMs 231 Plugins Use the Plugins / Manage Plugins… to launch the Plugin Manager A plug-in is an optional application that provides additional capabilities and features, in this case, to VirtualCenter. Examples of plug-ins are VMware Update Manager and VMware Converter Enterprise. Generally, plug-ins are released separately, install on top of VirtualCenter, and can be upgraded independently. You may download and install them or manage them with the Plugin Manager in VirtualCenter. Launch the Plugins Manager by clicking Plugins in the menu bar, then selecting Manage Plugins. After the server component of a plug-in is installed, you can activate the plug-in’s client component, which enhances the VI Client with appropriate UI options. 232 VMware Infrastructure 3: Install and Configure Client Settings • Client Settings Client Settings can set things like timeout values for slow WAN connections You may adjust the VI Client settings by clicking Edit in the menu bar, then selecting Client Settings... from the drop-down menu. This is important for things like adjusting the timeout values for slow WAN connections. Other items are also controlled such as the maximum number of virtual machine consoles, Hint Messages, and Getting Started tabs. The Lists tab controls things like the maximum number of Tasks and Events displayed. 6 Virtual Center Installation Module 6 Virtual Center Installation: Using VirtualCenter to Manage Hosts and VMs 233 Lesson Summary • Use lockdown mode to prevent administrators from trying to directly manage ESX hosts that are already being managed by a VirtualCenter Server • Scheduled Tasks can schedule many key operations on virtual machines and hosts • Use the Events panel and the System Logs tab in the Administration panel to help in the troubleshooting process . 234 VMware Infrastructure 3: Install and Configure Module Summary • The VirtualCenter Server, License Server and Web Access Server are located on the same system • The VI Client can be used to access the VirtualCenter Server as well as the ESX Server • The datacenter is the primary organizational structure in the VirtualCenter inventory • VirtualCenter Server has many controls and buttons in the VI Client that can make administration easier . 6 Virtual Center Installation Module 6 Virtual Center Installation: Using VirtualCenter to Manage Hosts and VMs 235 Questions? Questions? 236 VMware Infrastructure 3: Install and Configure MODULE 7 Virtual Machine Creation and Management 7 7 Importance • The most efficient way to use ESX Server is to create a base image virtual machine. Once you have a base image virtual machine, you can create a template and deploy additional virtual machines from the template. This saves time and prevents mistakes. Virtual Machine Creation and Management Objectives for the Learner • Create a virtual machine • Modify a virtual machine • Create a template • Deploy a virtual machine from a template • Use VMware Converter Enterprise to deploy virtual machines • Understand how Guided Consolidation works Module Lessons • Create a VM VMware Infrastructure 3: Install and Configure 237 • Create Multiple VMs • VMware Converter Enterprise • Manage VMs • Guided Consolidation 238 VMware Infrastructure 3: Install and Configure Lesson 1 Create a VM Lesson Topics • VM virtual hardware : 7 • Create a VM • Installing a guest OS into a VM • VMware Tools Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 239 What is a Virtual Machine (VM)? • Set of virtual hardware on which a supported guest OS and its applications run • A set of discrete files • A VM’s configuration file describes the VM’s configuration, including its virtual hardware • Avoid using special characters and spaces in VM’s display name MyVM.vmx guestOS = “winnetstandard” displayName = “MyVM” (etc.) Virtual Machine A virtual machine is configured with a set of virtual hardware on which a supported guest OS and its applications run. The virtual machine is a set of discrete files. The virtual machine's configuration file describes the VM's configuration, which includes the virtual hardware such as CPU, memory, disk, network interface, CD-ROM drive, floppy drive, etc. Emphasize to students not to use special characters, including spaces, in the VM display name. The display name is used to name the files that make up the VM. Filenames with special characters can cause errors when performing tasks. For example, if you have special characters in the VM's configuration file name, an error will occur when attempting to display the VM's console in Web Access. In general, it is a good practice to not use special characters and spaces in the VM's display name. The virtual machine's display name is used to name the files that make up the virtual machine itself. Therefore, a good practice is to avoid using special characters, including spaces, in the virtual machine's display name. The VI Client interface has no problems with special characters and spaces in a virtual machine’s display name. It is only service console command line administration that might cause issues. 240 VMware Infrastructure 3: Install and Configure What Files Make Up a Virtual Machine? File name VM_name.vmx VM_name.vmdk VM_name-flat.vmdk VM_name.nvram vmware.log vmware-#.log (where # is number starting with 1) VM_name.vswp VM_name.vmsd Description Virtual machine configuration file File describing virtual disk characteristics Preallocated virtual disk file that contains the data Virtual machine BIOS Virtual machine log file Files containing old virtual machine log entries Virtual machine swap file File that describes virtual machine’s snapshots Additional files may exist if snapshots are taken or raw disk mappings are added (to be discussed later) The table above lists the files that make up a virtual machine. Except for the log files, the name of each file starts with the virtual machine's name (VM_name). A virtual machine consists of the following files: • A configuration file (.vmx) 7 • One or more virtual disk files (first virtual disk has files VM_name.vmdk and VM_name-flat.vmdk; subsequent virtual disks are named VM_name_#.vmdk and VM_name_#-flat.vmdk, where # is the next number in the sequence, starting with 1) • A file containing the virtual machine's BIOS (.nvram) • A log file (.log) • A set of files used to archive old log entries (-#.log); 6 of these files are maintained at any time) • A swap file (.vswp) • A snapshot description file (.vmsd); this file is empty if the virtual machine has no snapshots. A virtual machine may have additional files if one or more snapshots have been taken or if raw disk mappings have been added. This is discussed later on in the module. If the VM has more than one disk file, the file pair for the second disk file and on is named VM_name_#.vmdk and VM_name_#-flat.vmdk, where # is the next number in sequence, starting with 1. For example, if the VM named "Test01" has two virtual disks, then this VM will have the files Test01.vmdk, Test01-flat.vmdk, Test01_1.vmdk and Test01_1-flat.vmdk. Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 241 Regarding the archive log files, 6 of these files are maintained at any time. For example, -1.log to -6.log may exist at first. The next time an archive log file is created (e.g. when the VM is powered off and powered back on), -2.log to -7.log are maintained (-1.log is deleted), then -3.log to -8.log, etc. 242 VMware Infrastructure 3: Install and Configure Display VM's Files Using the VI Client • Right-click Datastore to browse its files A virtual machine’s files are located in either a VMFS datastore or an NFS datastore. You can display a virtual machine’s files using the VI Client, if you know the datastore on which the virtual machine is located. To find out what datastore(s) your virtual machine is using, select your virtual machine in the inventory and view its Summary tab. The list of datastores used by the virtual machine is shown in the Resources section. To display the virtual machine’s files on a datastore, select your ESX Server from the inventory, then click its Summary tab. The list of datastores accessible by the ESX Server is shown in the Resources section. Right-click a datastore, then select Browse Datastore from the drop-down menu. The contents of the datastore are displayed. Double-click into any virtual machine’s folder to display its files. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 243 VM Virtual Hardware Up to 2 ports Up to 2 ports Up to 4 CD-ROMs 1-2 drives Up to 64GB RAM 1-4 adapters VM Chipset 1 CPU (2 or 4 CPUs with VMware SMP) 1-4 adapters; 1-15 devices each Make sure it is clear to the student that this is "virtual" hardware. The administrator of the ESX Server gets to manage the mapping between virtual and physical hardware. Each guest OS sees ordinary hardware devices; it does not know that these devices are actually virtual. Furthermore, all virtual machines have uniform hardware (except for a small number of variations the system administrator can apply). This makes virtual machines uniform and portable across platforms. ESX Server VMs lack USB and lack sound adapters. Each virtual machine has a total of 6 virtual PCI slots; one is used for the virtual video adapter. Therefore the total number of virtual adapters, SCSI plus Ethernet, cannot be greater than 5. The virtual chipset is an Intel 440BX-based motherboard with an NS338 SIO chip. This chipset ensures compatibility for a wide range of supported guest operating systems (including legacy OSs such as Windows NT). A virtual machine can have up to 2 IDE controllers, which means up to 4 CD-ROM drives are supported per virtual machine. 244 VMware Infrastructure 3: Install and Configure CPU and Memory • 1, 2 or 4 virtual CPUs (VCPUs) • Virtual SMP license required for 2- and 4-VCPU VMs • Specify maximum memory size (up to 64GB) • Amount the guest OS will be told it has Virtual Machine Although the VI Client interface may provide a default memory size for your VM at the time of creation, understand the memory needs of your application and guest OS and size accordingly. The maximum memory size allowed for any VM is 64 GB. Memory size is the maximum amount of physical memory that the virtual machine can use. If you have purchased the Virtual SMP product, you may take advantage of that purchase by selecting one-, two- or four- processors. Many guest OS/ application combinations are not enhanced by the additional CPU. Two- or four- VCPU VMs should be created only in the comparatively infrequent instances where they are of benefit, not as a standard configuration. Not every computer can host virtual machines with multiple virtual CPUs. In a later module, we will discuss the relationship between a virtual machine's number of virtual CPUs and the physical processors on the computer that hosts it. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 245 Virtual Disk Datastore Virtual Machine Parameter Virtual disk size Datastore Virtual disk node Virtual storage adapter Virtual disk files Advanced setting: Mode Sample value 4 GB MyVMFS 0:0 LSILogic (or BusLogic) VM_name.vmdk and VM_name-flat.vmdk Independent – Persistent or non-persistent A virtual machine has at least one virtual disk. Adding the first virtual disk implicitly adds a virtual SCSI adapter for it to be connected to. ESX Server offers a choice of either a virtual LSILogic adapter or a virtual BusLogic adapter. The virtual machine creation wizard in the VI Client automatically selects the type of virtual SCSI adapter based on the choice of guest OS. Select a VMFS to hold the new, blank virtual disk, and specify the disk's size. Choose a descriptive filename for the virtual disk. You may also site the disk at a specific virtual SCSI target ID and LUN if you wish. Finally, choose the appropriate disk mode. You can change the disk mode anytime the virtual machine is powered off. ESX Server virtual disks are monolithic and pre-extended. In other words, if you make a 6 GB virtual disk under ESX Server, the result will be a single file of size 6 GB. 246 VMware Infrastructure 3: Install and Configure Create a VM-Other Devices • Network adapter • Connect to virtual switch • CD-ROM drive • Connect to CD-ROM or ISO image • Floppy drive • Connect to floppy or floppy image • Generic SCSI devices (such as tape libraries) • May be connected to additional SCSI adapters Virtual Machine Virtual CPU, virtual memory and virtual disk are your required virtual hardware. Additional virtual hardware that you can add to your virtual machine are virtual NIC(s), a virtual CD-ROM drive, a virtual floppy drive and generic virtual SCSI devices. The virtual CD-ROM drive or floppy drive can point to either the CD-ROM drive or floppy drive located on the ESX Server, a CD ISO image (.iso) or floppy (.flp) images, or even the CDROM or floppy drive on your local system. You can map the VM's CD-ROM drive to either a physical drive or an ISO file for your CD-ROM drive. An ISO file is a CD-ROM that has been "ripped": its file system copied byte-for-byte to the disk surface. These virtual CDs can be accessed remotely and are usually faster than physical CDs. Just as you can make ISO files to serve as virtual CDs, you can make .flp files to serve as virtual floppies. In the service console, rip the floppy as follows: dd if=/dev/fd0 of=pathname/myfloppy.flp bs=1k count=1440 7 Virtual Machine Creation and Management The popular Windows freeware utility rawread.exe will also serve this purpose. Module 7 Virtual Machine Creation and Management: Create a VM 247 Virtual Machine (VM) Console • Send power changes to VM • Access VM’s guest OS • Send Ctrl+Alt+Del to guest OS • Press Ctrl+Alt+Ins in VM console VM Console icon • Press Ctrl+Alt to release cursor from VM console The virtual machine's console, available in the VI Client, provides the mouse, keyboard and screen functionality. To install an operating system, you must use the virtual machine's console. The VM console allows access to the BIOS of the virtual machine, and offers the ability to power on and off and to reset the virtual machine. The VM console is normally not used to connect to the VM for daily tasks. Tools such as RDP, Citrix, or VNC, for example, are normally used to connect to the VM. The VM console is used for tasks such as power cycling, configuring hardware, and troubleshooting network issues. The VM console allows you to send the Ctrl+Alt+Del key sequence specifically to the virtual machine. This is accomplished by pressing Ctrl+Alt+Ins in the VM console or by selecting VM in the VM console menu bar and clicking Send Ctrl+Alt+Del from the drop-down menu. Likewise, to release the cursor from the VM console so that you can use it in other windows, press Ctrl+Alt. An alternative way to view the VM's console is to select the virtual machine in the inventory, then click its Console tab. However, the "free-standing" VM console provides you with three new menus - File, View and VM - that do not exist on the Console tab. View the VM's console by clicking the VM console icon, located in the VI Client's toolbar, or right-click the virtual machine in the inventory, then select Open Console from the menu. 248 VMware Infrastructure 3: Install and Configure Install Guest OS into VM VM Console Install from ISO image (mounted on virtual CD-ROM drive) to virtual disk Local We interact with the VM through the VM console, accessible in the VI Client, to do the guest operating system's standard install routine. Note that ISO Images can be stored on either a VMFS datastore or an NFS datastore. Storing ISO images on a VMFS or NFS datastore allows you to share the ISO images across multiple ESX Servers, as long as the datastore is visible to the ESX Server. For details on the supported guest OSes, consult the Guest Operating System Installation Guide, available on the VMware website at http://www.vmware.com/pdf/GuestOS_guide.pdf. 7 Virtual Machine Creation and Management Key points to make on this slide: • We are installing the guest OS, represented by the middle, blue box in the virtual machine icon in the slide. • Mount the guest OS install CD or an ISO image of the physical CD. • Boot from the CD. • Run through the guest OS install program to install the guest OS into the virtual disk. Module 7 Virtual Machine Creation and Management: Create a VM 249 What are the VMware Tools? VMware Tools installs into guest OS like an application Features include: • Device drivers • Manual connection and disconnection of some devices while powered on • Improved mouse • Memory management • Support for quiescing a file system • Time synchronization • Ability to gracefully shut down virtual machine Once you have installed your operating system you should install VMware Tools. VMware Tools is a software package that you install into the guest OS after you have finished installing it. It gives you device drivers specific to VMware virtual devices where those are necessary, and it also installs several communication conduits between the VM and the VMkernel for specific applications. VMware Tools provides the ability to gracefully shut down a VM from the VM's right-click menu. To install VMware Tools in a virtual machine, right-click on the virtual machine name in the inventory and select “Install/Upgrade VMware Tools”. The virtual machine must be powered on and you must be logged in with an administrative or root-level account. Additional Information on VMware Tools: You can tell that a VM has VMware Tools installed if the VMware Tools icon appears in the desk tray (Windows only). If you are on a Linux system, an additional daemon (process) for VMware Tools appears on the system. Before you install VMware Tools, the only driver that any guest OS will have for the video card is its generic SVGA driver. So that's what it'll use. The VMware Tools install puts the custom driver for the VMware virtual video card into the right place, and then reconfigures the guest OS to use it. Linux users must arrange to have vmware-toolbox run while they are logged in, probably by adding it to their GNOME or KDE environment. For instance, in Red Hat 9.0 running GNOME, click the Red Hat logo, choose Preferences, then Window. In the resulting controlpanel list, choose More Preferences, then Settings. In the resulting start-up box, add a startup program entry for /usr/bin/X11/vmware-toolbox . VMware SVGA II VMware Pointing Device VMware SCSI Driver vmmemctl VMware Tools 250 VMware Infrastructure 3: Install and Configure To install VMware Tools on one or more virtual machines: • Select the Virtual Machines tab for a datacenter, cluster, resource pool or host. • Select on or more virtual machines onto which you want to install VMware Tools. • Right-click the selection, then select Install/Upgrade VMware Tools from the drop-down menu. • Click OK to begin the installation process. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 251 VMware Tools' Configurable Features Options Scripts Right-click to open Devices The most visible benefit of VMware Tools are that you get better video performance and that you can move your mouse pointer freely into and out of the VM console window. On VMware's server platforms, VMware Tools also installs other important system services such as virtual machine heartbeat monitoring and time synchronization. If you right-click (or double-click) the VMware icon (located in the lowerright hand corner of the screen on Windows guests) you may open the VMware Tools Properties window. This allows you to control Options, Scripts, and Devices: • Options: There are three options: • Enable time synchronization between the virtual machine and the ESX Server. Enabling this is best-practice, but will require you to disable any internal time-synchronization within the guest operating system. • Show VMware Tools in the toolbar. Without this, the VMware Tools icon is not shown. • Notify if a VMware Tools upgrade is available. • Scripts: Scripts allow you to specify scripts that may be run during power-state changes for the virtual machine. You may use either a default script (included with VMware Tools) or specify a custom script. • Devices: Devices may be connected and disconnected while the VM is powered on. These include CD-ROM and floppy media, and network connections. 252 VMware Infrastructure 3: Install and Configure VMware Tools Properties also has tabs for Shared Folders and Shrink. But those features are not available on VMs being hosted on ESX Servers. NFS, the Unix/Linux equivalent of Windows shares, requires that server and client keep their clocks in synchronization. Even if you are using Windows shares, there are many applications that suffer if systems' clocks are not synchronized. For example, in a software development environment, we rely on files' timestamps to determine whether they are out of date; if files are stored on a server with a clock set in the future, files will appear current when they are not. For more information on time synchronization, there is a white paper on this subject named "Timekeeping in VMware Virtual Machines", located at http://www.vmware.com/pdf/ vmware_timekeeping.pdf. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 253 Lab for Lesson 1 • Create a Virtual Machine Using VirtualCenter • In this lab, you will perform the following tasks: •Create a virtual machine •Install a guest OS into a virtual machine •Install VMware Tools into the guest OS •Verify network connectivity from your virtual machine 254 VMware Infrastructure 3: Install and Configure Lesson Summary • A VM can be configured with up to 4 virtual CPUs and 64 GB of memory • It is a best practice to install VMware Tools in every virtual machine • Use Ctrl+Alt+Ins in the VM’s console to send Ctrl+Alt+Del to the guest OS 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create a VM 255 Lesson 2 Create Multiple VMs Lesson Topics • Templates • Creating and deploying VMs from a template • Cloning a VM • Customizing a guest OS • Deploying across datacenters • Importing and exporting Virtual Appliances : 256 VMware Infrastructure 3: Install and Configure What is a Template? • A VirtualCenter feature used to create commonlydeployed VMs • A VM marked as never to be powered on • Disk files stored in either normal or compact disk format • All files can be stored in a VMFS or NFS datastore A template is a master image of a virtual machine that can be used to create and provision new virtual machines. This image typically includes a specified operating system, a set of applications, and configuration that provides virtual counterparts to hardware components. A template can be stored in either normal or compact disk format. With normal disk format, the virtual machine's disk files remain untouched. Use this option if you want to convert the template back into a running machine. With compact disk format, the virtual disk files are compressed to remove redundant information and save space. This is only supported on VMFS-3 datastores, and the server may ignore this for disks on other types of datastores. Templates can be stored in a VMFS datastore or an NFS datastore. Templates are a VirtualCenter feature. Standalone ESX Servers do not provide the templating feature. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 257 Create a Template • Two methods: • Clone to Template • Convert to Template • Choose Clone to Template if the original VM is still needed There are two ways to create a template: Clone to Template and Convert to Template. When you clone a VM to template, the original VM is retained. When you convert a VM to template, the original VM goes away. 258 VMware Infrastructure 3: Install and Configure Update a Template • Use the “Convert to Virtual Machine” task • Place VM on isolated network to prevent user access • Make changes to VM • Convert VM back to template If you need to update your template to include new patches or software, it is not necessary to create a brand new template. Instead, first convert the template back to a virtual machine. This allows you to power on the virtual machine. Log into the VM's guest OS and apply the patch or install additional software, whatever is necessary. When that is done, convert the VM back to a template. To convert a template back to a virtual machine, in the VI Client, display the Virtual Machines and Templates Inventory view. Right-click the template, then select Convert to Virtual Machine... from the menu. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 259 View Templates • Use the “Virtual Machines & Templates” view • From the “Hosts & Clusters” view, use the Virtual Machines tab To view all templates, use the VI Client. Go to the Virtual Machines & Templates view by clicking the Inventory panel’s drop-down list and selecting Virtual Machines and Templates. You can also view templates from the Inventory panel’s Hosts & Clusters view: select the Hosts & Clusters folder and click its Virtual Machines tab. Templates are distinguished from virtual machines by their icon. 260 VMware Infrastructure 3: Install and Configure Deploy VM from Template • To deploy a virtual machine, provide information such as virtual machine name, inventory location, host, datastore and guest OS customization data To deploy a VM from a template, connect to VirtualCenter using the VI Client. Display the Virtual Machines and Templates Inventory view. Rightclick the template, then select Deploy Virtual Machine from this Template. The Deploy Template wizard asks you for VM deployment information. You also have the option of having VirtualCenter customize the guest OS for you. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 261 Clone a VM • An alternate method of deploying a VM • Exact copy of VM • Customization of a clone’s guest OS is recommended to prevent software and network conflicts • It is also possible to Clone a Template Cloning a VM is an alternative to deploying a VM from a template. Like deploying from template, when you clone, you have the option of customizing the guest OS in the clone. To clone a VM, in the VI Client, right-click your virtual machine in the inventory, then select Clone.... 262 VMware Infrastructure 3: Install and Configure Guest OS Customization • VirtualCenter can apply unique system information to a VM when it is cloned or deployed from template • For guest OS customization to work, it must be enabled in VirtualCenter • To enable for Windows VMs, install sysprep files on VirtualCenter Server • Already enabled for Linux VMs (Open Source components are installed on the VirtualCenter Server) To enable guest OS customization, VirtualCenter must first be configured for this task. To customize Windows VMs, install Microsoft sysprep files on the VirtualCenter Server. For example, for Windows 2003: 7 • Retrieve the installer for Microsoft Windows 2003 sysprep from the Microsoft web site. • Copy the files from the .cab file, WindowsServer2003-KB892778-SP1DeployTools-x86-ENU.cab, to C:\Documents and Settings\ALLUSERSPROFILE\Application Data\VMware\VMware VirtualCenter\sysprep\svr2003 VirtualCenter supports guest OS customization for Windows 2000, Windows XP and Windows 2003. To customize Linux VMs, the Open Source components are used during guest OS customization. The Open Source components are installed when you install the VirtualCenter Server. The following values can be set when customizing a Linux guest OS: • Computer name • Domain name • IP settings (DHCP-assigned or static IP) • DNS server(s) For more details on how to prepare for guest customization, consult the Basic System Administration Guide, available on the VMware website. Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 263 Deploying Across Datacenters (1 of 3) • VM deployment is allowed across datacenters • Clone a VM from one datacenter to another • Deploy from a template located in one datacenter to a VM in a different datacenter • For example • Clone Prod01 from Datacenter A to Datacenter B VirtualCenter allows you to provision virtual machines across datacenters. As a result, VMware Infrastructure administrators can now clone a virtual machine from one datacenter to another datacenter. Administrators can also create a template in one datacenter, then deploy a VM from that template, placing the VM in a different datacenter. 264 VMware Infrastructure 3: Install and Configure Deploying Across Datacenters (2 of 3) • Example (continued) • Right-click Prod01 • Select Clone from the dropdown menu • Work through the Clone Virtual Machine Wizard The Clone Virtual Machine Wizard is used to clone any virtual machine, whether within the same datacenter or across datacenters. The Inventory Location area in the wizard shows the datacenters available. Choose the appropriate datacenter. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 265 Deploying Across Datacenters (3 of 3) You may receive one or more warning messages, however, you may still proceed with the clone Since the operation you are performing is being done across datacenters, you may receive a warning message. In the example above, the warning message is about the network named “Production”, which is being used by the source VM. This network may not actually be the same network on the source and destination. This is because network names are only unique within a datacenter. 266 VMware Infrastructure 3: Install and Configure Virtual Appliances • Pre-configured virtual machines • Usually designed for a single purpose • Examples: •Safe-browser •Firewalls http://www.vmware.com/appliances/ • Import from websites such as the Virtual Appliance Marketplace • Export your own VMs as Virtual Appliances Appliances in your home include devices such as a washing machines, refrigerators, table lamps, and televisions. These are all devices that have basically one function and are designed to be used by almost anyone with little or no training. The same things are true about virtual appliances. Virtual Appliances should be simple to use and designed primarily for a single purpose. Virtual Appliances are pre-configured virtual machines that typically include a preinstalled guest operating system and other software. These appliances are often built with public-domain or open-source software if they are designed to be shared outside of a corporation. Virtual Appliances can be imported from web-sites such as http://www.vmware.com/ appliances/. You can also export your own VMs as Virtual Appliances. Appliances can also be similar to templates. For example, it is possible to create a standardized VM with a pre-configured operating system and VMware Tools already installed. This VM can then be exported as an appliance from the ESX Server it was created on and easily moved to a central directory or web-site for easy importation by other ESX Servers. This allows a corporation to set up a central repository of standard VM starting points that can be accessed by ESX Servers Importing virtual appliances allows you to add pre-configured virtual machines to your VirtualCenter or ESX Server inventory. Importing a virtual appliance is similar to deploying a virtual machine from a template. However, you can import a virtual appliance from any local file system accessible from the VI Client machine, or from a remote web server. The 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 267 local file systems can include local disks (such as C:), removable media (such as CDs or USB keychain drives), and shared network drives. Exporting virtual machines allows you to create virtual appliances that can be imported by other users. You can use the export function to distribute pre-installed software as a virtual appliance, or as a means of distributing template virtual machines to users, including users who cannot directly access and use the templates in your VirtualCenter inventory. 268 VMware Infrastructure 3: Install and Configure Export VM with OVF Format • Select VM • Use File menu • Export Storage on VI Client host folders OVF (Open Virtual machine Format) is a file format that allows for the exchange of virtual appliances across products and platforms. The OVF format offers the following advantages: • OVF files are compressed, allowing for faster downloads. 7 • The VI Client validates a OVF file before importing it, and ensures that it is compatible with the intended destination server. If the appliance is incompatible with the selected host, it cannot be imported and an error message is displayed. To Export a VM with OVF simply use the following procedure: 1 2 3 4 5 Virtual Machine Creation and Management Select the VM within the VI Client VM must be powered off. VM must not have connections to local devices like CD-ROMs. Use the File pull-down menu. Select Virtual Appliances/ Export. The locations offered to store the appliance on will be any storage available to the PC that the VI Client is running on. This includes mapped drives. For more informaton about the OVF format, consult the technical paper, VMware OVF Tool, available on the VMware web site. Module 7 Virtual Machine Creation and Management: Create Multiple VMs 269 Import Virtual Appliance •Select host or cluster •Import from Virtual Appliance Marketplace, •Appliance validated for ESX prior to import file, or URL Importing a Virtual Appliance in OVF format is just as easy: 1 2 3 Select the host or cluster you plan to run the appliance on within the VI Client. Use the File pull-down menu. Select Virtual Machines / Import. The locations offered to import the appliance from include: • The Virtual Appliance Marketplace on VMware.com • An OVF file in some storage area accessible to the PC that VI Client is running on. This includes mapped drives. • A URL that is hosting OVF files 270 VMware Infrastructure 3: Install and Configure Lab for Lesson 2 • Template Provisioning • In this lab, you will perform the following tasks: •Configure guest OS customization on the VirtualCenter Server •Convert a virtual machine to a template •Convert a template back to a virtual machine •Clone a virtual machine to a template •Deploy a virtual machine from a template 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Create Multiple VMs 271 Lesson Summary • A template’s virtual disk files can be stored in either normal or compact disk format • A template can be converted back to a VM; this is useful if you need to update your template with new software • When you clone a VM or deploy a VM from a template, VirtualCenter can automatically customize the guest OS for you • A virtual appliance is a pre-configured virtual machine, designed for a specific purpose, and available from the VMware website 272 VMware Infrastructure 3: Install and Configure Lesson 3 VMware Converter Enterprise : Lesson Topics • VMware Converter Enterprise: 7 • Capabilities • Components • Concepts • Hot cloning • Cold cloning • Cloning modes • Changes to virtual hardware Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 273 VMware Converter Enterprise Capabilities • VMware Converter is a productivity tool that allows you to • Import physical machines to VMs • Import non-ESX VMware VMs • Import Microsoft Virtual Server 2005 VMs • Convert 3rd-party backup or disk images to VMs • Restore VCB images to VMs • Export VirtualCenter VMs to other non-ESX VMware VM formats • Reconfigure VirtualCenter VMs so they are bootable • Customize VirtualCenter VMs VMware Converter Enterprise is a tool integrated with VirtualCenter that allows administrators to convert almost any type of physical machine or virtual machine that is running a Windows operating system into a VM that runs on ESX Servers. Physical machine to virtual machine (P2V) conversions help during server consolidation, testing and troubleshooting, and disaster recovery. Virtual machine to virtual machine conversions allow VM mobility across VMware platforms as well as the ability to import Microsoft Virtual PC/Server VMs. VMware Converter Enterprise lets you move VMware virtual machines between VMware Workstation, VMware Fusion, VMware Player, VMware ACE, ESX Server, VMware Server and .Microsoft Virtual Server and Virtual PC For VMware virtual machines whose disks have been populated by restoring from a backup of a physical host or by some other direct means of copying, VMware Converter Enterprise prepares the image to run on VMware virtual hardware. Converter Enterprise supports restoring VMware Consolidated Backup images of any guest operating system type. For Windows, Converter Enterprise enables users to select and resize volumes, and customize the identity for the restored virtual machine. For all other guest operating systems, Converter Enterprise supports only restoring the disks as-is. 274 VMware Infrastructure 3: Install and Configure VMware Converter Components VI-Clients with Converter Enterprise Client Plug-in PCs and Servers to be converted (either physical or virtual), with Converter Enterprise Agent present Converter Agent is only present during conversion VC Server ESX Hosts Converter Enterprise Server may be installed directly on VC Server Converter Enterprise Server Converter Enterprise CLI VMware Converter Enterprise is a client/server architecture, which includes three components: • Server: Enables import and export of VMs through VI Client or CLI • CLI: Command line interface that connects to a Converter Enterprise Server and submits Converter tasks • Agent: Prepares a physical machine for import VMware Converter Enterprise Server works with VMware Converter Enterprise Client, which consists of: • Plug-in: Provides access to Converter Enterprise’s import, export and reconfigure wizards from within a VI Client In addition, for cold cloning of physical machines, VMware Converter Enterprise includes a Boot CD with standalone VMware Converter Enterprise. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 275 Installing VMware Converter Enterprise • To install the Converter Server: • Install VirtualCenter Server and Converter Server will be installed as well • Run the VMware Converter installer on a standalone server •Installer bundled with VirtualCenter Server software • To install the Converter Client: • Install the VI Client plug-in: •Log into VirtualCenter using the VI Client •From the VI Client menu bar, click Plugins -> Manage Plugins •Select under Available Plugins and install The Vista and Longhorn support only applies if installing Converter on a standalone server. The Converter Server is a Windows-based application capable of running on Windows 2000, XP/2003 (32-bit and 64-bit), Vista (32-bit and 64-bit), Windows Longhorn (32-bit and 64-bit). Its install options include: • Installing locally on the VirtualCenter Server • Installing on a standalone physical server • Installing in a VM VMware will have the ability to release Converter asynchronously from VCESX releases. The VI Client plug-in is based on the standalone Converter’s wizard Some fields, such as VirtualCenter credentials, will not be required. Other fields will be pre-populated based on how it is launched With the last two option, Converter Server requires a connection to a VirtualCenter Server running VirtualCenter Server 2.5 or later. The Converter Client is a VI Client plug-in which adds new menu items to the interface, both in the VI Client’s menu bar and right-click objects. Install the Converter Client from the VMware Infrastructure Client Plugins menu. Install the Converter Enterprise CLI on the same machine as Converter Enterprise Server, or on a different machine with access to a Converter Enterprise Server. Converter Enterprise Agent prepares a physical machine for import from a remote machine running Converter Enterprise Server. Converter Enterprise Server installs Converter Enterprise Agent on physical machines as needed, in order to import them as virtual machines. Users have the option to automatically remove Converter Enterprise Agent from the source physical machine after the import is complete. 276 VMware Infrastructure 3: Install and Configure VMware Converter Concepts • Cloning - To create an exact copy of a disk • System Reconfiguration - The replacement of physical hardware drivers with virtual hardware drivers • Hot Cloning - Cloning a system while it is running • Cold Cloning - Cloning a system while it is not running • Remote Cloning - Cloning a system using an agent without having to physically touch it • Local Cloning - Conversion performed using standalone VMware Converter Enterprise running on the source machine Converter Enterprise uses cloning and system reconfiguration to create a virtual machine that is compatible with ESX Server. Cloning is the process of creating a cloned disk, where the cloned disk is a virtual disk that is an exact copy of the source physical disk. This involves copying the data on the source machine’s hard disk and transferring that data to a target virtual disk (the new cloned disk). System reconfiguration is the process of adjusting the migrated operating system to enable it to function on virtual hardware. This adjustment is performed on the target virtual disk after cloning and enables the target virtual disk to function as a bootable system disk in a virtual machine. The process is nondestructive, so you can continue to use the original source machine after the import completes. If you plan to run an imported virtual machine on the same network as the source physical machine, modify the network name and IP address on one of the machines, so the physical and virtual machines can coexist properly. Hot cloning, also called live cloning or online cloning, entails cloning the source machine while it is running its operating system. Cold cloning, also called offline cloning, entails cloning the source machine when it is not running its operating system. With cold cloning, the user reboots the source machine from a CD that has its own operating system and includes the standalone VMware Converter Enterprise application. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 277 For remote cloning, the source machine can be accessed via an agent without having to physically touch it, as long as it is running and networkaccessible. Remote cloning installs, uses, then deinstalls an agent. With local cloning, the migration is performed using standalone VMware Converter Enterprise running on the source machine. 278 VMware Infrastructure 3: Install and Configure Hot Cloning - Four Stages VI Client with Converter Client Plug-in VirtualCenter Server with Converter Enterprise Server Running physical machine ESX Server Source volumes source destination Here are the general steps that occur during a remote hot cloning operation. All steps are automated, in other words, they are performed by Converter Enterprise without user involvement after the user has created and initiated the task. Stage 1: Preparing source machine for conversion • Converter Enterprise Server installs Enterprise Agent on source machine • Converter Enterprise Agent takes a snapshot of the source volumes Stage 2: Preparing the virtual machine on the destination machine • Converter Enterprise Server creates a new virtual machine on the destination machine, in other words, a destination ESX Server • Converter Enterprise Agent copies volumes from the source machine to the destination ESX Server Stage 3: Completing the conversion process • Converter Enterprise Agent installs required drivers to allow OS to boot in virtual machine • Converter Enterprise Agent customizes the virtual machine, for example, changes IP information Stage 4: Cleaning up • Agent removes all traces from the source machine. In other words, the snapshot created in stage 1 is deleted and the Converter Enterprise Agent is uninstalled from the source machine. You have the option of uninstalling the agent automatically or manually. Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise Volumes can be resized during hot cloning. 7 Virtual Machine Creation and Management 279 Cold Cloning - Four Stages physical machine ESX Server managed by VirtualCenter VMware Converter Enterprise Boot CD Source volumes destination source Standalone VMware Converter Enterprise application is located on the Converter Enterprise Boot CD, which a user reboots the source machine from in order to perform cold cloning. Volumes can be resized during cold cloning. Here are the general steps that occur during a cold cloning operation. After the user boots from the Converter Enterprise Boot CD and uses the wizard to set up and run the task, standalone Converter Enterprise performs the remaining steps without user involvement. Stage 1: Preparing the source machine image • User boots the source machine from the Converter Enterprise Boot CD and uses Standalone VMware Converter Enterprise to define and start the migration. • Standalone Converter Enterprise copies the source volumes into a RAM disk. Stage 2: Preparing the virtual machine on the destination machine • Standalone Converter Enterprise creates a new virtual machine on the destination machine. • Standalone Converter Enterprise copies volumes from the source machine to the destination machine. 280 VMware Infrastructure 3: Install and Configure Stage 3: Completing the conversion process • Standalone Converter Enterprise installs the required drivers to allow the operating system to boot in a virtual machine. • Standalone Converter Enterprise customizes the virtual machine, for example, it changes the IP configuration. Stage 4: Cleaning up • User removes Boot CD and reboots the source physical machine into its own operating system. The virtual machine is ready to run on the destination machine. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 281 Importing a Server The VMware Converter Enterprise Import Wizard allows you to import physical machines, virtual machines or backup/disk images. To launch the import wizard, in the VI Client, right-click your ESX Server in the inventory, then select Import Machine in the drop-down menu. The Import Machine selection in the drop-down menu will be grayed out if you do not install and enable the VMware Converter plug-in into the VI Client. The import wizard is also launched when you use standalone VMware Converter Enterprise, which is available when you perform a cold clone using the Convert boot CD. 282 VMware Infrastructure 3: Install and Configure Cloning Modes • Hot cloning uses volume-based disk cloning • Take all or selected volumes on disk(s) • Maintain or resize volumes • Cold cloning or VM conversion uses • Volume-based disk cloning or • Disk-based cloning •Copy disk(s) as is and maintain size VMware Converter Enterprise supports two cloning modes: volume-based cloning and disk-based cloning. Converter Enterprise supports volume-based cloning for hot and cold cloning and for importing existing virtual machines. 7 With volume-based cloning, all volumes in the destination virtual machine are basic volumes (primary partitions or logical drives that can be accessed by all Windows-based operating systems), regardless of the type in the corresponding source volume. Cloning is done on a block-level basis if you maintain the volume size. However, if you resize the volumes to be smaller than its original size, cloning is done on a file-level basis which can result in slower performance of the cloning operation. With disk-based cloning, if the size of the volume is maintained (not resized) the entire disk is copied as is. Disk-based cloning transfers all sectors from all disks, preserving all volume metadata. The destination virtual machine receives exactly the same volumes, of the same type, as those of the source virtual machine. Disk-based cloning supports all types of basic and dynamic disks. Disk-based cloning is only available with cold cloning and VM imports, it is not available with hot cloning. Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 283 Changes to Virtual Hardware • Most applications function correctly • Watch for applications that depend on: • Specific hardware characteristics • Different serial numbers • Software licensed to MAC addresses • Applications that depend on special graphics cards Most applications should function correctly in the VMware virtual machine because their configuration and data files have the same location as they did on the source virtual machine. However, applications might not work if they depend on specific characteristics of the underlying hardware such as the serial number or the device manufacturer. When troubleshooting after virtual machine migration, notice the following potential hardware changes: • CPU model and serial numbers (if activated) can be different after the migration. They correspond to the physical computer hosting the VMware virtual machine. • Ethernet adapter can be different (AMD PCNet or VMXnet) with a different MAC address. Each interface’s IP address must be individually reconfigured. • Graphics card can be different (VMware SVGA card). • Numbers of disks and partitions are the same, but each disk device can have a different model and different manufacturer strings. • Primary disk controllers can be different from the source machine’s controllers. • Applications might not work if they depend on devices that are not available from within a virtual machine. Settings that remain identical include operating system configuration, computer name, SID, user accounts, profiles, preferences, applications and data files and the volume serial number of each disk partition. 284 VMware Infrastructure 3: Install and Configure Common Converter Problems • Converter agent cannot install or run properly • Insufficient privileges to install and run agent as a service • Dependent services are disabled • Windows NT4/2000 failed to reboot • Converter Server cannot connect to remote system to import • Correct ports are not open • Physical switch configuration is not configured properly • Converter agent cannot detect OS on remote system • Unsupported OS • Windows 2003 software mirroring is enabled • boot.ini is set to read-only If VMware Converter Enterprise fails during the import process, here are some possible causes: • Converter Enterprise Agent cannot install or run properly on the remote system: 7 • • Verify that your user account has administrative privileges in order to install and run the agent as a service. Converter Enterprise Agent runs on the remote system as a Windows service named VMware Converter Enterprise service. • Verify that none of the VMware Converter Enterprise service’s dependencies are disabled. View what service’s dependencies by displaying the Dependencies tab in the service’s Properties window. • If you installed Converter Enterprise Agent on a Windows 2000 or Windows NT machine, Windows prompts you to restart your system. Verify that the system rebooted successfully after agent installation. • Converter Enterprise Server cannot connect to the remote system to import: • Verify that the correct ports are open in your firewall, which are ports 445, 139, 902 and 443. Ports 445 and 139 are used by the Converter Enterprise Server to communicate to the remote system during the initial discovery. Ports 902 and 443 are used by the Converter Enterprise Agent on the remote system to communicate back to the VirtualCenter Server (902) or the ESX Server (443). Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 285 Virtual Machine Creation and Management • Verify that the physical network switches on the source machine’s network are configured properly, for example, the speed and duplex settings are set correctly. • Converter Enterprise Agent cannot detect the OS on the remote system: • VMware Converter Enterprise can import any physical machine running one of the following operating systems: Windows NT 4 Workstation/Server SP4+, Windows 2000 Professional/Server/ Advanced, Windows XP Home (cold cloning only), Windows XP Professional (32-bit and 64-bit), Windows 2003 Standard/Web/ Enterprise (32-bit and 64-bit) and Windows Vista (32-bit and 64bit). • If you attempt to hot-clone a Windows physical machine, and this machine uses Windows software mirroring, the import fails with the error message, “Unable to determine guest operating system”. If this is the case, break the software mirror before attempting the hotclone. • From practical experience, it has been found that sometimes changing the boot.ini file’s permissions from read-only to read/ write fixed conversion problems. For more details on the VMware Converter Enterprise product, consult the VMware Converter Enterprise Administration Guide, available on the VMware Web site. 286 VMware Infrastructure 3: Install and Configure Lab for Lesson 3 • Use VMware Converter to Create Virtual Machines • In this lab, you will perform the following tasks: •Hot clone a system •Cold clone a system 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: VMware Converter Enterprise 287 Lesson Summary • VMware Enterprise Converter can convert most physical and virtual machines that are running Windows-based operating systems into ESX-hosted virtual machines • Cloning can be done in a “hot” mode, while the original machine continues to run • “Remote” cloning converts the system on-line over the network 288 VMware Infrastructure 3: Install and Configure Lesson 4 Manage VMs Lesson Topics • Move VM to a different ESX Server : 7 • Snapshot a VM • Modify a VM’s configuration Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 289 Move VM Between ESX servers: Cold Migration • A cold migration moves a VM that is powered off • May or may not involve movement of virtual disk • Perform a cold migration when • Moving VM to an ESX Server with a local (non-shared) datastore • Moving VMs between ESX Servers using different CPU families VM files are not moved VM files are moved A cold migration is used to move a virtual machine from one ESX Server to another while the VM is powered off. With a cold migration, all the VM's files may or may not move. Keep in mind that the VM's files are located in a subdirectory on either a VMFS datastore or an NFS datastore. When the destination ESX Server is not able to see the VM's files (because, for example, the VM's files are located in a local datastore on the source ESX Server), then the files must be moved to a datastore visible to the destination ESX Server in order for the VM to be migrated. The migration wizard offers the choice to move the VM from one host to another, or move just the VM files from one datastore to another, or both. You can also use cold migration to move a VM's files from its current datastore to a different datastore, without moving it to a different ESX Server. 290 VMware Infrastructure 3: Install and Configure Snapshot a VM • Snapshots • Useful when you need to revert repeatedly to the same state, without creating new VMs • Useful in test/dev, training scenarios • Snapshot manager manages your snapshots •Right-click virtual machine in inventory •Select Snapshot -> Snapshot Manager from menu Snapshot Manager Snapshots let you preserve the state of a virtual machine so you can return to the same state repeatedly. A snapshot captures the entire state of a virtual machine at the time you take the snapshot. This includes the settings state, the disk state and the memory state. The settings state contains the virtual machine settings. The disk state contains the state of all the virtual machine's virtual disks. The memory state represents the contents of the virtual machine's memory. Memory state is captured only if you are snapshotting a virtual machine that is powered on. When taking a snapshot, the user has the option of snapshotting the virtual machine’s memory or not. By default, the option to capture the virtual machine’s memory state is selected. A virtual machine can have one or more snapshots. Each snapshot consists of the following files: • Snapshot differences file: VM_name-00000#-delta.vmdk, where # is the next number in the sequence, starting with 1 • Snapshot description file: VM_name-00000#.vmdk • Memory state file: VM_name-Snapshot#.vmsn; size of this file is the size of the VM's maximum memory (only if memory is captured, else the file is much smaller.) To display the Snapshot Manager, right-click the virtual machine in the inventory, then select Snapshot -> Snapshot Manager... from the menu. The Snapshot Manager window allows you to perform three tasks: 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 291 • Delete: This task commits the snapshot data to the parent snapshot, then removes the selected snapshot. • Delete All: This task commits all the immediate snapshots before the current state icon (“You are here”) to the base disk and removes all existing snapshots for that virtual machine. • Go to: Ths task allows you to restore a particular snapshot. The snapshot that you restore becomes the current snapshot. In the example above, the current snapshot is Snapshot 2. If you restored Snapshot 1, Snapshot 1 would become the current snapshot and he You are here icon would be positioned under Snapshot 1. 292 VMware Infrastructure 3: Install and Configure Modify Virtual Machine Settings • Many Virtual Machine settings can be customized and / or modified • Many of these can only be changed while the VM is powered off • These settings are controlled by three tabs: • Hardware Tab • Options Tab • Resources Tab It is possible to make customizations and modifications to an existing virtual machine, for example, adding an additional virtual NIC or additional virtual disk. All of these items can be changed when the VM is powered off. A virtual disk can be added to the VM while it is powered on. This is known as a "hot pluggable" device. Use the following procedure to launch the Virtual Machine Properties window: 1 2 7 Virtual Machine Creation and Management Power off the VM. When the power-off state change is complete, right-click on the VM and select Edit Settings. You may also just click the “Edit Settings” button in the right window. The properties window will open. It has three tabs on it: Hardware, Options, and Resources. Use the Hardware tab to modify the hardware on the virtual machine. This allows you to do things like add more hard disks and network adapters. It also allows you to connect hardware like CD-ROMs and Floppy Drives to specific hardware either on the host or on the VI Client PC, or to a .iso or .flp image file. You may also make changes to the virtual network adapter by controlling whether or not they are connected when the VM is first powered on and which virtual machine port group they are connected to. The Options tab will be covered later on in this lesson. Module 7 Virtual Machine Creation and Management: Manage VMs 293 The Resources tab will be covered in detail during the Resource Management module of the course. The next several pages will focus on the Hardware tab. 294 VMware Infrastructure 3: Install and Configure Example 1: Add Raw LUN Access to VM • Why use a raw LUN with a VM? • To allow VM clustering—across boxes, or physical-to-virtual • To enable use of SAN management software inside guest OS • Can be added while VM is powered on • A VM can access a raw SAN LUN using an RDM (Raw Device Mapping) • An RDM allows a special file in a VMFS volume to act as a proxy for a raw device An example of adding virtual hardware to a VM is adding another disk to the VM. There are two types of virtual disks that can be added to a VM: a virtual disk file or a raw disk mapping. A virtual disk file is a file in a datastore (VMFS or NFS.) A raw disk mapping gives your virtual machine direct access to any LUN - SAN, iSCSI or local. The raw disk mapping (RDM) is a special file that lives in a datastore (VMFS and NFS) and points to the actual SAN LUN. The VM is able to access its SCSI LUN through this RDM. Both virtual disks and raw disk mappings are hot-pluggable devices. They can be added while the VM is powered on. A raw disk mapping runs in one of two modes: physical compatibility mode and virtual compatibility mode. Physical compatiblity mode allows the guest operating system to directly access the hardware and is useful if you are using SAN-aware applications in the virtual machine. Virtual compatibility mode allows the LUN to behave as if it were a virtual disk, so you can use features like snapshotting, cloning and creating templates. With physical compatibility mode, a LUN configured for physical compatibility cannot be cloned, made into a template, or migrated if the migration involves copying the disk. Raw disk mappings are made up of the following files: If using virtual compatibility mode, the files are VM_name_#.vmdk and VM_name_#rdm.vmdk (where VM_name is the name of the virtual machine and "#" is the next number in the sequence). If using physical compatibility mode, the files are VM_name_#.vmdk and _#-rdmp.vmdk. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 295 Example 2: Add a Virtual NIC to VM • Why add an additional NIC? • To allow a VM to access multiple networks • To create a firewall environment • Must be added while VM is powered off Another example of adding virtual hardware to a VM is adding another virtual NIC. In this case, the VM must be powered off in order to add the virtual NIC, since virtual NICs are not hot-pluggable devices. In the example above, a second NIC is added to the virtual machine in order to create a firewalled environment. 296 VMware Infrastructure 3: Install and Configure Example 3: Resize the Disk Increased from 7 GB to 9 GB Format new space within the guest operating system It is possible to expand the size a virtual disk. This task can only be performed while the VM is powered off. To resize a VM’s disk, right-click your VM in the inventory, then select Edit Settings from the drop-down menu. Select the desired hard disk and enter a new size. The virtual hard disk will expand to the new size. This is similar to increasing the size of a LUN. If the VM were a physical machine it would suddenly think the hard disk was bigger, leaving unallocated space on the disk. This is the same in the virtual world and you will need to add a partition from within the guest OS. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 297 Virtual Machine Properties Options • General • VMware Tools • Power Management • Advanced The Options tab in the Virtual Machine Properties window allows you to change a VM’s options. It has several powerful features that are broken down into four categories: • General • VMware Tools • Power Management • Advanced The next several pages will cover some of the important things you can do to modify a VM from the Options tab. 298 VMware Infrastructure 3: Install and Configure Options - General Options VM display name .VMX file location VM directory Guest operating system type The General Options can be used to modify things like the display name used for the VM and the type of guest operating system installed. The location and name of the configuration file (.vmx file) is displayed and the location of the virtual machine’s directory is also shown. You can select the text for the configuration file and working location if you need to cut and paste them into a document. But only the display name and the guest operating system type may be modified. NOTE 7 Virtual Machine Creation and Management If you change the display name, that is not going to change the names of all of the VM files or the directory the VM is stored in. When a VM is first created, the file names and the directory name associated with the VM are based on its display name. But changing the display name later does not modify these file and directory names. Module 7 Virtual Machine Creation and Management: Manage VMs 299 Options - VMware Tools Customize power button actions When to run VMware Tools scripts Update checks and time synch The VMware Tools options window controls how the VMware Tools inside the virtual machine respond to certain external events. You can use these to customize the power buttons on the VM. For example, the red square power-off button for a VM can be set to always perform a guest shutdown. This is far safer for the VM. It is like the difference between using the Start / Shutdown command within Windows as opposed to just unplugging the PC. The VMware Tools program can be set to run certain scripts when specific events (like a power-off) occur. That has to be set from within the guest OS though by opening the VMware Tools window. Once those scripts are selected and enabled this screen controls when the VM checks to see if scripts should actually be run. This gives you the advantage of enabling or disabling script operations from outside the VM while it is powered-off. The Advanced box has two important functions. One is to check possibly update VMware Tools automatically if a newer version becomes available. The other is to enable time synchronization with the host. As a best practice time synchronization with the host should always be enabled. However, if the VM is forcing its clock to synch to the ESX Server you must ensure two other things have been configured: The ESX Server should have its time synch’ed to some external source, preferably via NTP. The Guest OS should NOT be trying to synchronize time on its own. Most Windows systems automatically synchronize to a Windows Active Directory Domain Controller. Many UNIX and Linux systems are 300 VMware Infrastructure 3: Install and Configure configured to synchronize to external NTP servers. Best practice is to let VMware Tools synchronize time to the host – and disable these other time synchronization systems within the guest OS. If you configure the VM to synchronize time to the ESX Server and also allow the guest to try to synchronize time to something else time on the virtual machine will become unstable and erratic. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 301 Options - Power Management Suspend or standby the guest OS gracefully. Wake on LAN The Power Management options allow you to choose how the virtual machine should respond when it is placed in the Standby power state. The VM can either be suspended or the guest OS can be placed into standby mode, leaving the virtual machine powered on. If you opt for placing the guest OS into Standby mode you can enable Wake on LAN. This is not available on all guest operating systems. 302 VMware Infrastructure 3: Install and Configure Options - Advanced Advanced options usually do not need to be set The group of options known as “Advanced” cover things that usually do not need to be set for a virtual machine. Some of these can improve performance, allow VMotion between CPUs with minor differences, and adjust logging and debugging settings. We will specifically cover two of these advanced options here – boot options and swapfile location. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 303 Advanced - Boot Options Delay power-on Boot into BIOS The Advanced Boot options allow you to do two things. One is to delay a power-on. This may be useful to help stagger VM startup when several VMs are being powered on. It is also possible to actually specify a poweron order within the VI Client by selecting an ESX Server and then going to the Configuration tab and selecting Virtual Machine Startup/Shutdown. The “Boot into BIOS” option is extremely useful for making changes to the BIOS settings such as forcing a VM to boot off of a CD-ROM. The next time the VM powers-on, it goes straight into BIOS. This is much easier than powering the VM on, opening a console, and quickly trying to hit the F2 key to go into BIOS. 304 VMware Infrastructure 3: Install and Configure Swapfile Location Each host or cluster can have a custom “swapfile datastore” location defined Each virtual machine has its own swapfile. These are normally stored in the same location that the other virtual machine files are located in. However, if the VM’s files are stored on a network storage location that has poor performance (such as a slow NFS server) you may see a performance boost by storing the VM’s swap file on faster storage. To facilitate this “swapfile datastores” can be defined for each ESX Server and/or cluster. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 305 Labs for Lesson 4 1. Create a Firewalled Virtual Machine Environment • In this lab, you will perform the following tasks: • • Create a Linux virtual machine that functions as a NAT Router Configure an existing virtual machine as a NAT Client that uses the NAT Router to access the external network 2. (OPTIONAL) Allow Virtual Machine Access to a Raw LUN • In this lab, you will perform the following tasks: • • Add a raw LUN to a virtual machine’s configuration Verify that the virtual machine can access its new LUN 306 VMware Infrastructure 3: Install and Configure Lesson Summary • VirtualCenter allows automatic guest OS customization when cloning a VM or deploying a VM from a template • It is possible to resize a VM’s hard disk, provided the VM is powered off • It is possible to add a hard disk to your VM while the VM is up and running 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Manage VMs 307 Lesson 5 Guided Consolidation Lesson Topics • Guided Consolidation • Capabilities • Architecture • Discovery • Analysis • Consolidation : 308 VMware Infrastructure 3: Install and Configure Guided Consolidation • Automatically discovers physical servers • Analyzes utilization and usage patterns • Converts physical servers to VMs placed intelligently based on user response • Lowers training requirements for new virtualization user • Steers users through the entire consolidation process Convert Discover Analyze For first time virtualization users, a new feature in VirtualCenter 2.5 guides users through the process of server consolidation. Recommended for smaller, simpler environments, this feature steers users through discovering physical servers, collecting performance data from these servers and converting these servers to virtual machines placed intelligently on the most appropriate hosts. Guided Consolidation allows new users to quickly realize the benefits from server consolidation and reduces the training requirements for first time “virtualizers”. Guided Consolidation automatically consolidates existing servers by discovering existing servers in the environment, either physical servers or virtual machines. Guided Consolidation can discover and analyze only Windows server-family OSes. Servers are analyzed to determine whether or not they are suitable for consolidation. Servers’ usage patterns are discovered and analyzed. No agent software is involved to perform these tasks. Recommendations are made based on the utilization metrics that have been collected. Guided Consolidation recommends a consolidation plan, matching discovered systems to candidate ESX Servers. Finally, physical machines are converted into virtual machines while these servers continue to run. Guided Consolidation is a tool intended for small-to-medium businesses, with approximately 100 physical servers or less. Its architecture assumes a small environment. For example, it assumes that you have only one Active Directory server and a limited number of domains in the environment. It is not appropriate for large-scale enterprises, since the user interface does not provide a good way to deal with hundreds of physical servers. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 309 If you do not want Guided Consolidation installed, it can be omitted when installing VirtualCenter. Additional Information about current constraints in Guided Consolidation: Guided Consolidation currently requires default username and password (needed for discovery) to only contain ASCII strings. As such the discovery feature may not work on non-English OSes. This is essentially a bug in collector which currently does not support Unicode. (we hope to fix this by GA) Users cannot limit discovery to a certain IP range or filter search results based on wildcards or other search criteria Guided consolidation does not discover across multiple AD sources as the collector engine can only discover and report on domains managed by a single AD source Guided Consolidation relies on AD and/or lanman to discover machines on the network and gather data on the same. Systems that have lanman service disabled (typical in larger enterprise environments) will not show up in discovery in the absence of AD. In such a scenario, VC needs to be installed on a system that is part of an AD domain and the user needs to provide credentials that has read access on the AD domain to begin discovery and data gather. Needless to say in the absence of both lanman and AD, guided consolidation will not be able to discover any physical systems 310 VMware Infrastructure 3: Install and Configure Guided Consolidation Architecture • Guided Consolidation depends on two services, Data Collector and VMware Converter VMware Converter vpxd Data Collector VMware Converter Service (can run on VirtualCenter Server or on separate machine) VC Database VirtualCenter Server Data Collector Service (CapacityPlanner is automatically installed on VirtualCenter Server) The Guided Consolidation architecture consists of two services, Data Collector and VMware. These services are installed together with VirtualCenter Server. The Data Collector Service runs under the name of “VMware Capacity Planner Service”. It is responsible for discovering existing systems in the environment, getting their hardware information and probing them periodically to collect their performance information. It uses a “hidden” database, in other words, a database that is not intended to be managed by end users and used for storing results as they are collected. The Data Collector Service uses LAN Manager (lanman) or Active Directory (AD). In the case of LAN Manager, the Data Collector Service needs individual systems to be visible by lanman. Systems are reported as present when queried. If LAN Manager is not present and Active Directory is, the Data Collector Service must be installed on a member of the domain. The user running the Data Collector Service must have read permissions on the Active Directory. The VMware Converter service runs under the name of “VMware Converter Enterprise Service”. It converts physical systems to virtual machines. VirtualCenter provides this service with information about the destination and other parameters, and VMware Converter Enterprise Service handles the conversion operation. This service can be installed on a separate machine. The Data Collector Service is based on the Capacity Planner product. It is not exactly the same, so if you are familiar with Capacity Planner, you will notice that the results will differ. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 311 Physical System Discovery Click the Start Analysis button to begin discovering physical (and virtual) systems To start the discovery process, click the Consolidation panel in the VI Client, then click the Start Analysis button. VirtualCenter credentials will be requested at this point and authentication is required to search the domains for physical computers. The Data Collector Service is required when initiating the consolidation analysis. If this service is not running, VirtualCenter will ask for user credentials and try to start it. The user must have Windows Administrator privilege and read privileges on Active Directory, if Active Directory is being used. 312 VMware Infrastructure 3: Install and Configure Add to Analysis (1 of 2) • Select a Domain or Workgroup for discovery • The list of Domains and Workgroups is concatenated from AD and Lanman results The Add to Analysis dialog box enables you to discover systems on your network and select the ones you want to analyze. This dialog box lists the systems found on the network for the domain selected in the Show domain drop-down menu. The first time this dialog box is launched, the domain where the VirtualCenter server is located is selected by default. After that, the menu defaults to the previously selected domain. The first time a domain is selected, it might take some time for VirtualCenter to discover and list the systems it finds. After that, the list is cached so that subsequent searches take less time. The list can also be sorted. 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 313 Add to Analysis (2 of 2) • Select Hosts from the list to analyze • Discovery of systems is repeated periodically • Newly added systems will be discovered automatically. • Every ½ hour: check for new servers • Every day: check for new domains From Add to Analysis dialog box, you can select hosts from a particular domain or workgroup to analyze. System discovery is repeated periodically, just in case new systems come on-line. Every half hour, new systems will be discovered in each domain, and every day, there will be a check for new domains. 314 VMware Infrastructure 3: Install and Configure Set Authentication • Enter Windows Administrator user and password • Enter here if same for all/most hosts • Next screen lets you specify per-host credentials VirtualCenter requires administrator access to the systems selected for analysis before it can begin to analyze them. You can specify credentials on a system-by-system basis, and you can specify default credentials that VirtualCenter can use when credentials have not been explicitly specified. 7 To set credentials per system, in the Add to Analysis dialog box, select the systems you want to analyze. Click the Add to Analysis button. The Set Authentication dialog box is displayed, shown above. Enter authentication credentials and click OK. Default credentials can also be set through the Consolidation Settings dialog box. To set default credentials using Consolidation Settings, select Administration from the VI Client menu bar, then select Consolidation Settings -> Credentials tab. Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 315 Analyze • Statistics collected on each host • Metrics collected once per hour • 10-12 metrics total: CPU, Memory, Disk, Network • Columns populated as information obtained • Data put into table in VirtualCenter database • Confidence level • Based on the number of performance samples that VC has collected • As VC collects more performance samples, the confidence goes up The selected systems are analyzed and results are displayed in the Analysis tab. In the example above, MKTG1 and MKTG2 were selected for analysis from the previous step. You can right-click a host to set per-host credentials if necessary. The Data Collector starts collecting data once per hour on each host. 10-12 metrics are collected on CPU, memory, disk and network usage, and the columns in the display are populated, such as CPU Usage and Memory Usage, as information is obtained. All data is stored into tables in the VirtualCenter database. The Data Collector is agentless and does not install any software on target machines. Information is collected using remote data retrieval methods, such as WMI and Remote Registry. This is why the service must run with administrator privileges. If target systems are protected by a firewall, then ports need to be opened to allow incoming WMI, Perfmon and Remote Registry requests to pass through (ports 135, 137, 138, 139 and 445). The Confidence Level indicates the degree to which VirtualCenter is able to gather performance data about the system and how good a candidate the system is for consolidation based on the available data. The confidence level is based on the number of performance samples that VirtualCenter has collected. The more performance samples that VirtualCenter collects, the higher its confidence level. 316 VMware Infrastructure 3: Install and Configure One important metric displayed in the Analysis tab is the Confidence metric. During the analysis phase, performance data about each selected system is collected. This data is compared to host resources to determine a recommendation for each candidate. The recommendation indicates how well suited, based on the collected data, a candidate is to a particular virtual machine host system. Confidence refers to the reliability of the recommendation and it is a function of the duration of the analysis. Recommendations based on longer periods of analysis – and therefore more performance data – receive a higher level of confidence. NOTE After 24 hours of analysis, VirtualCenter indicates a high level of confidence in its recommendations. However, this can be misleading if a system’s workload varies significantly over weeks or months. To ensure a high level of confidence in a recommendation, allow the duration of the analysis phase to encompass an amount of time that includes representative peaks and troughs in the systems’ workload. Analysis 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 317 Plan Consolidation • Choose “Plan Consolidation” • Select systems to import, then click Consolidate After the Analysis phase, you are ready to plan consolidation. In the Analysis tab select the systems you want to consolidate, then click the Plan Consolidation button (not shown above). A list of analyzed systems is presented. For each system, a drop-down menu exists identifying the candidate destination ESX Servers. A destination rating (or star rating) is also displayed. The star rating is used to determine suitability of the destination server for consolidation. Each candidate destination host gets a separate star rating and is based on the destination server’s compatibility with the ESX Server. Compatibility is considered for things such as sufficient number of CPUs and the ability to run the guest OS. The rating is based on the average CPU usage, memory usage and disk space usage of the destination host. The networking check only verifies the number of NICs, not network usage. The lower the resource usage, the higher the star rating. The higher the star rating, the better suited that destination host is for consolidation. When ready, select the systems to import. For each one, select the destination ESX Server. Click the Consolidate button when ready. The import process is performed by the VMware Converter Enterprise Service. 318 VMware Infrastructure 3: Install and Configure Lesson Summary • Guided Consolidation is a tool intended for small to medium businesses • Guided Consolidation consists of two services: Data Collector service and the Converter service • The Consolidation services automatically discovers physical servers, analyzes utilization and usage patterns, and converts them into virtual machines 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 319 Module Summary • A virtual machine can be created from scratch or deployed from a template • A virtual appliance is a pre-configured virtual machine, designed for a specific purpose, and available from the VMware website • VMware Converter allows hot cloning or cold cloning of physical servers to virtual machines • A VM snapshot is useful when you need to revert repeatedly to the same state, without creating new VMs • Guided Consolidation allows the discovery and consolidation of physical servers to virtual machines 320 VMware Infrastructure 3: Install and Configure Questions? Questions? 7 Virtual Machine Creation and Management Module 7 Virtual Machine Creation and Management: Guided Consolidation 321 322 VMware Infrastructure 3: Install and Configure MODULE 8 Virtual Infrastructure Access Control 8 Importance • When there are multiple users accessing the virtual infrastructure, it is a good idea to give each user only the necessary permissions, nothing more. VirtualCenter access controls allow flexible assignment of permissions. 8 Virtual Infrastructure Access Control Objectives for the Learner • Configure VirtualCenter permissions • Configure ESX Server permissions • Manage access to VMs using Web Access Module Lessons • VMware Infrastructure User Access • Accessing VMs Using Web Access VMware Infrastructure 3: Install and Configure 323 Lesson 1 VMware Infrastructure User Access Lesson Topics • Security model • VirtualCenter permissions • ESX Server permissions : 324 VMware Infrastructure 3: Install and Configure Security Model Overview User/Group Role Privileges Permission Inventory Objects The main components of the Virtual Infrastructure security model are the following: • User/Group - User/group account with access to the Virtual Infrastructure • Role - A set of one or more privileges • Privilege - Specifies a task that a user/group is authorized to perform • Permission - The pairing of a user/group and role (which consists of a set of privileges) Users or groups are granted permission to the inventory based on the roles that they are assigned. Roles are made up of one or more privileges, each privilege allowing access to perform a specific task. There are approximately 100 defined privileges. Some tasks require only a single privilege while other tasks require multiple privileges. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 325 Defining Users and Groups VirtualCenter users and groups are those defined in the VirtualCenter Server’s Windows domain ESX Server users and groups are those defined in its service console No attempt is made to reconcile these users and groups Using the VI Client, you have access to either the VirtualCenter server or the ESX Server using the same user interface. The same security model applies to both VirtualCenter users and ESX users, however, the permissions are different and there is no synchronization of permissions between VirtualCenter and ESX Server. 326 VMware Infrastructure 3: Install and Configure Privileges • Privileges are the building blocks of roles • They allow users to perform tasks • They are grouped in categories A role is a set of one or more privileges. A privilege allows access to a specific task and is grouped with other privileges related to it. For example, the role named "Virtual Machine User" consists of several privileges in categories such as Global, Virtual Machine, Scheduled Task. A role is assigned to a user or group and determines that user or group's level of access. To get to the screen shown above, use the VI Client to connect to the VirtualCenter Server. Select an object in the VirtualCenter inventory, for example, the Hosts & Clusters folder. Right-click the object, then select Add Permission... from the menu. In the Assigned Role section, select Virtual Machine User from the drop-down list. Expand the Virtual Machine category, then the Interaction subcategory. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 327 Roles The VI Client displays users/groups associated with a given role • Roles are collections of privileges • Can be optionally propagated to child objects in the inventory A role consists of one or more privileges managed through the VI Client. The VI Client will display all users and/or groups associated with a given role. To display this information, go to the Admin panel, then view the Roles tab. Select any role to view the users and/or groups, if any, associated with that role. Roles are not hierarchically organized, in other words, a role is neither superior to or subordinate to another role. All roles are independent of each other. Role propagation is the act of passing along permissions. Roles can be optionally propagated to its child objects in the inventory. 328 VMware Infrastructure 3: Install and Configure Pre-defined and Custom Roles Roles Default ESX Server user and group roles No Access Read-Only Administrator Virtual Machine Administrator Datacenter Administrator Virtual Machine Power User Virtual Machine User Resource Pool Administrator VMware Consolidated Backup User Night-shift Operator Backup Administrator Default VirtualCenter user and group roles Create your own roles for either ESX Server or VirtualCenter users and groups ESX Server provides the following default roles: No Access, Read-Only, and Administrator. VirtualCenter provides the following default roles: No Access, Read-Only, Administrator, Virtual Machine Administrator, Datacenter Administrator, Virtual Machine Power User, Virtual Machine User, Resource Pool Administrator and VMware Consolidated Backup User. You cannot modify the default roles No Access, Read-Only and Administrator. You will be able to modify the other default roles, however, it is recommended to create a custom role instead. Custom roles can be created for either ESX Server or VirtualCenter, such as Night-shift Operator and Backup Administrator. Custom roles cannot be shared between ESX Server and VirtualCenter. It is a good practice to manage your ESX Servers and virtual machines through VirtualCenter. Therefore, create any necessary custom roles in VirtualCenter instead of directly on the ESX Server. NOTE Instructor, for background information and possibly adding value to your class, consider reading the Best Practices paper, Managing VMware VirtualCenter Roles and Permissions, availabe on the VMware Web site at http:// www.vmware.com/pdf/ vi3_vc_roles.pdf 8 Virtual Infrastructure Access Control The Best Practices paper, Managing VMware VirtualCenter Roles and Permissions provides some very good recommendations for custom VirtualCenter roles that you might consider for your environment. This paper is availabe on the VMware Web site at http://www.vmware.com/pdf/ vi3_vc_roles.pdf. Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 329 Permissions • Permissions are granted by pairing a user (or group) with a role and assigning them to an inventory object Greg – Datacenter Administrator Susan – Resource Pool Administrator Greg– No Access Carla – Virtual Machine Power User What happens if a user is granted different roles in different areas of the inventory tree? For example, does a permission for Carla set at "Hosts & Clusters" override a permission for Carla set at "Carla04VM", or vice versa? --> Permissions can be overridden at a lower level by adding a new permission to the same user. Also, roles will only flow down if propagation is turned on. The permissions that a user is given is a combination of the user (or group) account, the role assigned to the user (or group) and the position in the inventory to which the user/role combination applies. Roles can also be propagated downwards through the inventory, if you choose. Permissions can be overridden at a lower level by adding a new permission to the same user. In most cases, propagation should be enabled when building a role. When it comes to assigning the permission, propagation helps insure consistency if and when new objects are inserted as child objects in the inventory. For example, if permissions are assigned on a folder which contains VMs, you typically want the same permissions on all VMs that are contained in that folder. If propagation is not desired, consider limiting the extent of propagation with the No Access (built-in) role directly on the object that should be left out of the propagation. In the example above, Greg has been assigned the Datacenter Administrator role at the Training datacenter level and all the objects below it, assuming the role has been propagated to the child objects. However, Greg is not assigned the Datacenter Administrator role on the Test and Dev resource pool and all the objects under it. For these objects, he has no access. 330 VMware Infrastructure 3: Install and Configure How Permissions Are Applied: Scenario 1 • If a user is a member of multiple groups with permissions on different objects • For each object on which the group has permissions, the same permissions apply as if granted to the user directly Group1 – VM Administrator Group2 – Read-Only Members of Group1: Greg Susan Members of Group2: Greg Carla If a user is a member of multiple groups, and has permissions on different objects in the inventory, then for each object on which the group has permissions, the same permissions apply as if they were granted to the user directly. In the example above, there are two groups, Group1 and Group2. Group1 is assigned the VM Administrator role at the Training datacenter and Group2 is assigned the Read-Only role on the virtual machine object, Prod03-1. Both roles propagate to their child objects. Let's say that user Greg is a member of both Group1 and Group2. If this is the case, then Greg gets Virtual Machine Administrator privileges on the entire Training Datacenter, except for the virtual machine named Prod03-1. For this particular object, Greg gets Read-Only access. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 331 How Permissions Are Applied: Scenario 2 • If a user is a member of multiple groups with permissions on the same object • The user is assigned the union of privileges assigned to the groups for that object Group1 – VM_Power_On (custom role) Group2 – Take_Snapshots (custom role) Members of Group1: Greg Susan Members of Group2: Greg Carla If a user is a member of multiple groups, and these groups have permissions on the same object in the inventory, then the user is assigned the union of privileges assigned to the groups for that object. In the example above, there are two groups, Group1 and Group2. Group1 is assigned the role, VM_Power_On, a custom role that contains only one privilege, the ability to power on a VM. Group2 is assigned the role, Take_Snapshots, another custom role that contains the privileges to create and remove snapshots. Both roles propagate to the child objects. Let's say that Greg belongs to both Group1 and Group2. If this is the case, then Greg gets both VM_Power_On and Take_Snapshots privileges for objects within the Training datacenter. 332 VMware Infrastructure 3: Install and Configure How Permissions Are Applied: Scenario 3 • Permissions defined explicitly for the user on an object take precedence over all group permissions on that same object Group1 – VM_Power_On (custom role) Group2 – Take_Snapshots (custom role) Greg – Read-Only Members of Group1: Greg Susan Members of Group2: Greg Carla This example falls under the category of “the particular taking precedence over the general”. Permissions defined explicitly for the user on an object take precedence over a user’s group permissions on that same object. In the example above, three permissions are assigned to the Training datacenter: Group1 is assigned the VM_Power_On role, Group2 is assigned the Take_Snapshots role, and user Greg is assigned the Read-Only role. Let's say Greg is a member of both Group1 and Group2. Let’s also assume that propagation to child objects is enabled on all roles. In this case, even though Greg is a member of both Group1 and Group2, Greg gets Read-Only privilege to the Training datacenter and all objects under it. This is because explicit user permissions on an object take precedence over all group permissions on that same object. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 333 How Permissions Are Applied: Scenario 4 • Permissions applied directly to an object override inherited permissions Greg – VM User Greg – Administrator This is another example that falls under the category of “the particular taking precedence over the general”. Permissions applied directly to an object in the inventory take precedence over permissions inherited from roles that are propagated down to child objects. In the example above, user Greg is given the VM User role at the Training datacenter. This role is propagated to all child objects except one, Prod031. For the virtual machine object, Prod03-1, Greg has Administrator privileges instead. 334 VMware Infrastructure 3: Install and Configure VirtualCenter Security Model Active Directory or Local Windows User/Group VirtualCenter user role permission privileges In the VirtualCenter security model, the VirtualCenter user is a Windows user account, either local or domain. The user is assigned a role. The user/ role combination is applied to an object in the VirtualCenter inventory. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 335 Default Permissions for VirtualCenter • Local Windows Administrators group is assigned the Administrator role at the topmost level in the inventory By default, the local Windows group, Administrators, is assigned the Administrator role at the topmost level of the Hosts & Clusters view and the Virtual Machines & Templates view. 336 VMware Infrastructure 3: Install and Configure ESX Server Security Model Service console ESX Server user role permission privileges In the ESX Server security model, the ESX user is a service console (Linux) user account. The ESX user is assigned either a default role or a custom, ESX Server role. The user/role combination is applied to a level in the ESX inventory (host, VM or resource pool level.) User accounts, roles and permissions can be configured using the VI Client connected directly to the ESX Server. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 337 Default Permissions for ESX Server • ESX Server users, root and vpxuser, are assigned the Administrator role at the ESX Server level By default, the service console users, vpxuser and root, are assigned the Administrator role at the ESX Server level in the inventory. root is the administrator account on any Linux/UNIX system. is created when an ESX Server is added to the VirtualCenter inventory. vpxuser is the user account used by the VirtualCenter Server to authenticate itself when sending pre-approved task requests to the ESX Server. Only pre-approved task requests are sent to the ESX Server. Tasks are pre-approved by VirtualCenter based on user and group permissions. vmware-hostd, running as root, performs the tasks requested by VirtualCenter vpxuser CAUTION Do not change vpxuser and do not change its permissions. If you do so, you might experience problems working with the ESX Server through VirtualCenter. 338 VMware Infrastructure 3: Install and Configure Prevent root Access to VI Client • Enable Lockdown Mode • Prevents ESX user root from logging directly into the ESX Server using the VI Client • Normal ESX user accounts can still use VI Client To prohibit ESX Server administration by direct VI Client login as root, enable Lockdown Mode on that ESX Server. Lockdown mode can be used to ensure that the ESX Server is managed only through VirtualCenter. To enable Lockdown Mode, select your ESX Server from the inventory, then click its Configuration tab. In the Software section, click the Security Profile link, then click Edit... next to the Lockdown Mode section. A check box allows you to either enable or disable lockdown mode. Although user root will be prevented from logging directly into the ESX Server using the VI Client, a normal, non-administrator ESX Server account will still be able to log in. User root will still have the ability to log into the ESX Server using a secure shell. If VirtualCenter becomes unavailable (e.g. the service stopped) and lockdown is enabled on an ESX Server, root is not able to log into the ESX Server using the VI Client. However, a normal ESX Server user account will still be able to log in. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 339 Labs for Lesson 1 1. Accessing Virtual Machines in VirtualCenter • • • In this lab, you will perform the following tasks: Configure a VirtualCenter permission using an existing role Configure a VirtualCenter permission using a custom role 2. Multi-Role Users • • In this lab, you will perform the following task: Determine the effects of individual user permissions vs. group permissions at different levels of the inventory 3. (Optional) Accessing Virtual Machines in ESX Server • • In this lab, you will perform the following task: Configure an ESX Server permission using an existing role 340 VMware Infrastructure 3: Install and Configure Lesson Summary • A VirtualCenter user is a Windows user, either local or domain-based • An ESX Server user is a Linux user, defined in the service console • Permissions, composed of user/group role assignments, are assigned to objects in the inventory and control what users can do 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: VMware Infrastructure User Access 341 Lesson 2 Accessing VMs Using Web Access Lesson Topics • Logging into Web Access • Web Access functionality : 342 VMware Infrastructure 3: Install and Configure What is Web Access? • A browser-based application that focuses on managing VMs on ESX Server and VirtualCenter deployments • Benefits: • Administrators can provide end users browser-based access to VMs without the need to install the VI client on their desktop • Client Devices allow VMs to access media on the user’s local floppy and CD/DVD drives •Reduces the need to access these drives on the ESX Server host Web Access Web Access (Apache Tomcat Service) installed here Web Access VMware Virtual Infrastructure Web Access (Web Access for short) is a way to allow end users to access VMs without needing to install the VI Client onto their desktop. Web Access is a Web application running under the Apache Tomcat Web server, which is started on either the VirtualCenter Server or the ESX Server. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: Accessing VMs Using Web Access 343 Log into Web Access (1 of 2) Users access Web Access via a web browser. Use a URL based on either the VirtualCenter’s or ESX Server’s host name or IP address. In VirtualCenter environments, one should normally login and use Web Access through the VirtualCenter Server and not the ESX Server. The reason is that in VirtualCenter environments, VMs might move from ESX Server to ESX Server due to VMotion and VMware DRS cluster software. To VI Web Access client is designed for these browsers: • Windows: • Internet Explorer 6.0 or higher • Netscape Navigator 7.0 • Mozilla 1.x • Firefox 1.0.7and higher • Linux: • Netscape Navigator 7.0 or later • Mozilla 1.x • Firefox 1.0.7 and higher 344 VMware Infrastructure 3: Install and Configure Log into Web Access (2 of 2) • If logging into VirtualCenter, enter a VirtualCenter user account and password Before using Web Access, users must use either a valid user name and password to access the VirtualCenter Server or a valid user name and password to access the ESX Server. This user name and password information will be used to log into Web Access. When a user logs into Web Access on the ESX Server, the user will be able to manage only the virtual machines found on that ESX Server, as long as the user has the appropriate permissions. When a user logs into Web Access on the VirtualCenter Server, the user will be able to manage virtual machines found on all ESX Servers in the VirtualCenter inventory, provided that user has the appropriate permissions. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: Accessing VMs Using Web Access 345 Web Access Tasks View a VM’s console View VMs and their details Perform select VM tasks Whereas the VI Client can be used to manage ESX Servers and VMs, Web Access is used to manage VMs only. From Web Access, you can display a list of VMs, view a VM's console, view a VM's status, perform power operations and edit a VM's configuration. The list of VMs displayed depends on what you are logging into. If you log into Web Access on an ESX Server, you will see a list of all VMs located on this server. If you log into Web Access on the VirtualCenter Server, you will see a list of all VMs located on all ESX Servers managed by VirtualCenter. You cannot create new VMs using Web Access. Creating new VMs must be done using the VI Client instead. In order to access a virtual machine's console from the Web Access interface, the VMware Virtual Infrastructure Plug-In needs to be added to your browser. See the Virtual Infrastructure Web Access Administrator's Guide for details on installing the plug-in. 346 VMware Infrastructure 3: Install and Configure Generate Remote Console URL • Way to provide access to a VM through a URL • Useful for including in an e-mail message Using Web Access, you can create a remote console URL of a virtual machine using ordinary Web browser URLs. When creating a remote console URL, you can customize the Web Access user interface controls, or use the remote console URL for personal use. If desired, you can disable nonessential controls permanently. This allows a remote console URL user to concentrate on using the guest operating system. Using remote console URLs, you can: • Add the remote console URL to a list of favorite Web pages • Share the remote console URL with one or more users in an e-mail message Let's say you no longer want a user to access a remote console URL of a particular virtual machine. To disable this remote console URL, create a second remote console URL to the same virtual machine. This new URL is now required to access the virtual machine. Since the generated URL is quite long, it might be useful to mention using http://www.tinyurl.com to shorten the URL and avoid wrap issues when inserting the URL into e-mails. 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: Accessing VMs Using Web Access 347 Activity • Using Web Access • Take a few minutes to explore the Web Access interface: •Log into Web Access on the VirtualCenter Server and perform a few tasks on your virtual machines •Log into Web Access on the ESX Server and perform a few tasks on your virtual machines This is an informal lab activity that allows you to login and explore Web Access. Web Access is installed on both the VirtualCenter Server and the ESX Server. Use a web browser to access each one. If you are unable to access Web Access on your VirtualCenter Server, verify that the VMware Virtual Infrastructure Web Access is started: on your VirtualCenter Server, select Start -> Administrative Tools -> Services. If students are unable to access Web Access on VirtualCenter, have them enter the following URL as a workaround: https://IP_Address_of_VC_Server/ui/ 348 VMware Infrastructure 3: Install and Configure Module Summary • A permission is a pairing of a user and a role • A role is a set of pre-defined privileges • VirtualCenter users are different from ESX Server users • Web Access is used to manage VMs, not ESX Server hosts 8 Virtual Infrastructure Access Control Module 8 Virtual Infrastructure Access Control: Accessing VMs Using Web Access 349 Questions? Questions? 350 VMware Infrastructure 3: Install and Configure MODULE 9 Resource Management 9 Importance • Resource pools allow CPU and memory resources to be hierarchically assigned. VMotion is a valuable tool for availability and resource management. VMware DRS-enabled clusters provide automated resource management for multiple ESX Servers Objectives for the Learner 9 • To use resource pools for single-host resource policy control • To migrate virtual machines with VMotion • To create and configure a DRS cluster • To create resource pools in a DRS cluster for multi-host resource policy control Resource Management Module Lessons • Using Resource Pools • Migrate VMs with VMotion • VMware DRS (Distributed Resource Scheduler) • Resource Pools in a VMware DRS Cluster VMware Infrastructure 3: Install and Configure 351 Lesson 1 Using Resource Pools Lesson Topics • How are VMs’ CPU and memory resources managed? • What is a resource pool? • Managing a pool’s resources • A resource pool example • An expandable reservations example • Admission control : 352 VMware Infrastructure 3: Install and Configure VMs' CPU Resource Settings • Limit • A cap on the consumption of CPU time by this VM, measured in MHz • Reservation • A certain number of CPU cycles reserved for this VM, measured in MHz • The VMkernel chooses which CPU(s), and may migrate • Shares • More shares means that this VM will win competitions for CPU time more often • All the VCPUs in a VM must be simultaneously scheduled • Therefore, a reservation of 1000 MHz might be generous for a 1-VCPU VM, but not for a 4-VCPU VM A virtual machine has three user-defined settings that affect its CPU resource allocation: CPU limit, CPU reservation and CPU shares. CPU limit defines the maximum amount of CPU, measured in MHz, that this virtual machine is allowed. CPU reservation defines the amount of CPU, measured in MHz, reserved for this virtual machine when CPU contention occurs. If the virtual machine does not use the total amount of its CPU reservation, then the unused portion is available for use by other virtual machines, until the virtual machine needs it. Each virtual machine is granted a number of CPU shares. The more shares a VM has, the more often it gets a timeslice of a CPU when there is no CPU idle time. All the virtual CPUs (VCPUs) in a virtual machine must be scheduled at the same time. Therefore, a CPU reservation of 1000 MHz might be generous for a 1-VCPU virtual machine, but not for a 4-VCPU virtual machine (250 MHz per VCPU.) 9 Resource Management Module 9 Resource Management: Using Resource Pools 353 VMs' Memory Resource Settings • Available Memory • Memory size defined when the VM was created • Limit • A cap on the consumption of physical memory by this VM, measured in MB • Reservation • A certain amount of physical memory reserved for this VM, measured in MB • Shares • More shares means that this VM will win competitions for physical memory more often • VMkernel allocates a per-VM swap file to cover each VM’s range between available memory and reservation A virtual machine has four user-defined memory settings that affect its memory resource allocation: available memory, memory limit, memory reservation and memory shares. Available memory is the amount of memory given to the virtual machine at the time it was created. It is the maximum amount of memory the virtual machine supplies to the guest OS. The VM cannot address a larger memory area than this size of available memory, unless it is powered down and more memory is configured for the VM. Memory limit defines the maximum amount of virtual machine memory that can reside in RAM, not to exceed available memory. By default, available memory and memory limit are initially the same value. Memory reservation is the amount of RAM reserved for that virtual machine memory. Unused memory reservations, like CPU reservations, are not wasted. If a VM does not consume all of the RAM that is reserved for it, other VMs can use that RAM. But once the VM uses that RAM, no portion of the VM’s reservation will ever be ballooned or swapped, even if that RAM is completely idle. Transparent page sharing, however, is not prevented from reclaiming reserved memory. Memory shares are separate from CPU shares but are applied in the same way. A virtual machine's memory shares controls how often it wins competition for RAM when RAM is scarce. Virtual machines that lose must wait until RAM becomes available. 354 VMware Infrastructure 3: Install and Configure If the values for available memory and memory reservation differ, the VMkernel allocates a per-VM swap file to cover the difference between available memory and the memory reservation. During periods of RAM shortage, the virtual machine’s available memory could consist of physical RAM and disk space. If necessary, here is information to explore the ramifications of setting reservation and limit for memory. Students tend to get confused with cases 3 and 4. You might write this on the board and make it clear that they are valid (but rare) settings: Case1: Memory is not overcommitted (less common) • VM built with: X amount of memory • Reservation: X amount of memory • Limit: X amount of memory • Size of VMkernel swap file: 0 Case 2: Memory is overcommitted (most common case) • VM built with: X amount of memory • Reservation: less than (<) X amount of memory • Limit: X amount of memory • Size of VMkernel swap file: (X - < X) Case 3: Memory limit is set by Administrator planning for possible future growth of VM memory (rarely used) • VM built with: X amount of memory • Reservation: X amount of memory or < X amount of memory • Limit: Greater than (>) X amount of memory Case 4: Memory limit is set by Administrator to sacrifice performance in a VM temporarily (rarely used) • VM built with: X amount of memory • Reservation: < X amount of memory • Limit: < X amount of memory 9 Resource Management Module 9 Resource Management: Using Resource Pools 355 How VMs Compete for Resources • Proportional-share system for relative resource management • Applied during resource contention • Prevents VMs from monopolizing resources • Guarantees predictable resource shares Number of Shares • Change number of shares • Power on VM • Power off VM The proportional share mechanism applies to CPU and RAM allocation, and only operates when virtual machines are contending for the same resource. Shares guarantee that a virtual machine be given a certain amount of a resource (CPU or RAM.) For example, consider the third line of the example on the slide, where VM D has just been powered on with 1000 shares. Beforehand, there were 5000 total shares, but D's addition increases the total shares to 6000. This means that all other virtual machines' shares decline in value. However, each virtual machine's share value still represents a minimum guarantee. VM A is still guaranteed one-sixth of the resource, because it owns one-sixth of the shares. We can add shares to a VM while it is running, and it will get more access to that resource (assuming there was competition). When we add a new VM, it gets shares too. Its share amount factors into the total number of shares; but the existing VMs are guaranteed not to be starved for the resource. When we delete or power off a VM, there are fewer total shares, so the surviving VMs get more access. When configuring shares for a VM, you specify High, Normal, Low or Custom. For CPU shares: • High: # shares = 2000 * (# of vCPUs) • Normal: # shares = 1000 * (# of vCPUs) • Low: # shares = 500 * (# of vCPUs) • Custom: # shares = user-specified value 356 VMware Infrastructure 3: Install and Configure For memory shares: • High: # shares = 20 * size of VM’s available memory • Normal: # shares = 10 * size of VM’s available memory • Low: # shares = 5 * size of VM’s available memory • Custom: # shares = user-specified value 9 Resource Management Module 9 Resource Management: Using Resource Pools 357 What is a Resource Pool? • A logical abstraction for hierarchically managing CPU and memory resources • Used on a standalone hosts or VMware DRSenabled clusters • Provides resources for VMs and child pools Resource Pools Root Resource Pool Defer the discussion of DRS clusters to Lesson 3. A resource pool allows you as the administrator to divide and allocate resources to VMs and other resource pools. A resource pool allows you to control the aggregate CPU and memory resources of the compute resource, which is either a standalone host or a VMware DRS cluster. Resource pools are also used to delegate privileges to other users and groups. The topmost resource pool is known as the root resource pool. The root resource pool consists of the CPU and memory resources of a particular ESX Server or VMware DRS cluster. 358 VMware Infrastructure 3: Install and Configure Configuring a Pool's Resources • Resource pools have the following attributes: • Shares • Low, Normal, High, Custom • Reservations, in MHz and MB • Limits, in MHz and MB • Unlimited access, by default (up to maximum amount of resource accessible) • Expandable Reservation? • Yes: VMs and sub-pools may draw from this pool’s parent • No: VMs and sub-pools may only draw from this pool, even if its parent has free resources Each resource pool has reservation (minimum), limit (maximum) and share values for both CPU and memory resources. A resource pool has the following attributes: • Shares: Shares guarantee that the resource pool be given a certain amount of CPU and memory resources. • Reservation: This is the minimum amount of resources required by the resource pool. For example, you can set a CPU reservation, which is the minimum amount of CPU that this pool must have. • Limit: This is the maximum amount of resources given to this resource pool. By default, the resource pool is given "unlimited" access to the maximum amount of resource (specified by the limit.) The Limit is adjustable. You can limit a resource pool to a specific amount of resource, which is less than the absolute maximum. • Expandable Reservation: This allows a resource pool that cannot satisfy a reservation request to search through its hierarchy to find unreserved capacity to satisfy the reservation request. Shares, reservations and limits can also be applied at the virtual machine level, and are constrained by the resources of the resource pool to which the virtual machine belongs. Virtual machines do not have expandable reservation. Expandable reservations can only be set at the resource pool level. Expandable reservations will be covered in more detail later on in the module. 9 Resource Management Module 9 Resource Management: Using Resource Pools 359 Viewing Resource Pool Information (1 of 2) • Display the resource pool’s Summary tab Get information about your resource pool by viewing the Summary tab. This tab displays the current values for Shares, Reservation, Expanded Reservation and Limit. 360 VMware Infrastructure 3: Install and Configure Viewing Resource Pool Information (2 of 2) • Display the resource pool’s Resource Allocation tab Get further information about your resource pool by viewing the Resource Allocation tab. This tab displays information about how the CPU and memory resources are being used by the virtual machines and child pools in the resource pool. 9 Resource Management Module 9 Resource Management: Using Resource Pools 361 Scenario • Company X’s IT department has two internal customers • The finance department supplies 2/3 of the budget • The engineering department supplies 1/3 of the budget • Each internal customer has both production and test/dev virtual machines • We must cap the test/dev VMs’ resource consumption To explain how resource pools work, let's take an example. Company X's IT department has two internal customers, finance and engineering. Both departments have production virtual machines as well as virtual machines for testing and application development. The finance department provides the majority of IT's budget and therefore gets the majority of resources provided by IT. Resource pools can be used to control resource consumption between the two departments and ensure that the finance department gets the resources that it is entitled to. 362 VMware Infrastructure 3: Install and Configure Resource Pool Example stand-alone host – Svr001 (root resource pool) CPU: 12000 MHz Memory: 4 GB Engineering (Resource Pool) CPU Shares: 1000 Reservation: 1000 MHz Limit: 4000 MHz Expandable Reservation: Yes Eng-Test (VM) CPU Shares: 1000 Reservation: 0 MHz Limit: 4000 MHz Eng-Prod (VM) CPU Shares: 2000 Reservation: 250 MHz Limit: 4000 MHz Let's take a look at an example where resource attributes are set on a resource pool. Resource pools can be organized hierarchically. The root resource pool is the topmost resource pool and is comprised of the sum of all MHz for all CPUs and the sum of all the installed RAM (in MB) available in the compute environment (standalone host or cluster). In this example, the root resource pool is a standalone host named Svr001. It has 12000 MHz of CPU and 4 GB of RAM, available for use by other resource pools or VMs. Except for the root resource pool, every resource pool has a parent resource pool. A resource pool might contain child resource pools or just VMs that are powered on within it. A child resource pool is used to allocate resources from the parent resource pool for the child’s consumers. Administrative control can also be delegated to various individuals or organizations. A child resource pool cannot exceed the capacity of the parent resource pool. Creating a child pool actively reserves resources from the parent pool, whether or not any VMs in the child pool are powered on. 9 Resource Management Module 9 Resource Management: Using Resource Pools 363 Resource Pools Example: CPU Shares stand-alone host – Svr001 (root resource pool) Engineering (Resource Pool) CPU Shares: 1000 Finance (Resource Pool) CPU Shares: 2000 Eng-Test (VM) CPU Shares: 1000 Eng-Prod (VM) CPU Shares: 2000 Fin-Test (VM) CPU Shares: 1000 Fin-Prod (VM) CPU Shares: 2000 Shares specify the relative priority or importance of either a resource pool or virtual machine. If a resource pool has twice as many shares of a resource as another resource pool, it is entitled to consume twice as much of that resource. The same thing can be applied to virtual machines. In the example above, the Finance resource pool has twice as many CPU shares as the Engineering resource pool and therefore, is entitled to twice as much CPU resources as the Engineering resource pool. The next slide further explains this concept. 364 VMware Infrastructure 3: Install and Configure Resource Pools Example: CPU Contention Svr001 All VMs below are running on same physical CPU (PCPU) Engineering CPU Shares: 1000 ~33% of PCPU Finance CPU Shares: 2000 ~67% of PCPU Eng-Test CPU Shares: 1000 Eng-Prod CPU Shares: 2000 Fin-Test CPU Shares: 1000 Fin-Prod CPU Shares: 2000 11% Engineering ~33% 45% 22% 22% Eng-Test gets ~33% of Engineering’s CPU allocation = Approximately 11% of the PCPU Finance ~67% % of PCPU allocation As an example, let's assume that all four virtual machines have been scheduled by the VMkernel onto the same physical CPU. Thus they are all in direct competition. Engineering gets 33% of that CPU, then splits up its 33% allotment between virtual machines Eng-Test and Eng-Prod. Likewise, Finance gets 67% of that CPU, then splits up its 67% allotment between virtual machines FinTest and Fin-Prod. The virtual machine, Eng-Test, gets ~33% of the CPU allocation of the Engineering resource pool, [1000/(1000+2000)]. This works out to about 11% of the physical CPU (33% of 33% equals ~11%). Each of the virtual machines get a percentage of the physical CPU allocated to its resource pool based on its individual share allocation. Note that the example above uses general approximations to explain how the number of shares affects the amount of CPU allocated to a virtual machine. 9 Resource Management Module 9 Resource Management: Using Resource Pools 365 Expandable Reservation Root Resource Pool Total CPU: 10200 MHz Total Memory: 3000 MB • Borrowing resources occurs recursively from the ancestors of the current resource pool • As long as the Expandable Reservation option is selected. • Offers more flexibility, but less protection Retail Reservation: 3000 MHz Expandable Reservation: Yes eCommerce Apps Reservation: 1200 MHz Expandable? Yes eCommerce Web Reservation: 1000 MHz Expandable? No • Expanded reservations are not released until the VM that caused the expansion is shutdown or its reservation is reduced An expandable reservation could allow a rogue administrator to claim all unreserved capacity in the environment For this slide, just define expandable reservation. There is a complete example on the next two slides. Expandable reservation allows a resource pool that cannot satisfy a reservation request to search through its hierarchy to find unreserved capacity to satisfy the reservation request. In this example, the child resource pool, “eCommerce Apps” has expandable reservation set. The reservation of a child resource pool may not exceed that of its parent. The search for unused resources goes through the ancestry of the root resource pool or to the first resource pool that does not have expandable reservation set. Use expandable reservation carefully. A single child resource pool may use ALL of its parent’s available resources, leaving nothing directly available for other child resource pools. One reason to disable Expandable Reservation is when you are giving a fixed amount of resources to a group. For example, you are an IT administrator and your customers are different organizations in your company who have paid for a fixed amount of CPU and memory resources. 366 VMware Infrastructure 3: Install and Configure Example of Expandable Reservation (1 of 2) Root Resource Pool Total CPU: 10200 MHz Total Memory: 3000 MB • eCommerce resource pools reserve 2200 MHz of 3000 MHz the Retail resource pool has reserved • Power on virtual machines in the eCommerce Web resource pool • With Expandable Reservation disabled on the eCommerce Web resource pool it is not possible to start VM7 with a reservation of 500 MHz • Lower the VM reservation • Enable Expandable Reservation Retail Reservation: 3000 MHz Expandable Reservation: No eCommerce Apps Reservation: 1200 MHz Expandable? Yes eCommerce Web Reservation: 1000 MHz Expandable? No VM1 R=400 VM2 R=300 • Increase eCommerce Web pool’s reservation In this example, there are three resource pools: Retail, eCommerce Apps and eCommerce Web. The resource pool, eCommerce Web, has a CPU reservation of 1000 MHz. It also does not have expandable reservation set, its reservation is Fixed. There are three virtual machines in the eCommerce Web resource pool: VM1, VM2 and VM7. VM1 has a CPU reservation of 400 MHz and is powered on. Likewise, VM2 has a reservation of 300 MHz and is also powered on. As a result, 700 MHz of the reservation of the eCommerce Web resource pool is in use. VM7 R=500 What happens if you try to power on VM7, which has a CPU reservation of 500 MHz? Since expandable reservation is disabled on the eCommerce Web resource pool, it is not possible to start VM7 with a reservation of 500 MHz. Therefore, either lower VM7’s reservation, enable expandable reservation on the eCommerce Web resource pool, or increase the reservation of the eCommerce Web pool. 9 Resource Management Module 9 Resource Management: Using Resource Pools 367 Example of Expandable Reservation (2 of 2) Root Resource Pool Total CPU: 10200 MHz Total Memory: 3000 MB **200 MHz used by Retail** Retail Reservation: 3000 MHz Expandable Reservation: Yes • Enable expandable reservation on the eCommerce Web resource pool • The system considers the resources available in the child resource pool and its direct parent resource pool **Full Reservation Used** eCommerce App Reservation: 1200 MHz Expandable? Yes eCommerce Web Reservation: 1000 MHz Expandable? Yes • The VM’s reservation is charged against the reservation for eCommerce Web • eCommerce Web’s reservation is charged against the reservation for Retail VM3 R=500 VM5 R=500 VM4 R=500 VM6 R=500 VM1 R=400 VM2 R=300 VM7 R=500 Let’s explain what’s happening, starting from the top: • The root resource pool has a total of 10200 MHz availabe for its child resource pools to use. • The Retail resource pool has a total of 3000 MHz available for its child resource pools to use. It has expandable reservation set. • eCommerce App and eCommerce Web are child resource pools of the Retail pool. They both have expandable reservation set. Together, they have reserved a total of 2200 MHz in the Retail pool. Therefore, the Retail pool has 800 MHz left of its reservation for others to use. • The total amount of VM CPU reservation in the eCommerce App resource pool is 2000 MHz. Since eCommerce App only has 1200 MHz reserved, the remaining 800 MHz needed to satisfy the VMs’ reservations is taken from the Retail resource pool, which has 800 MHz to give. At this point, the Retail pool’s full reservation is used. • The total amount of VM CPU reservation in the eCommerce Web resource pool is 1200 MHz. Since eCommerce Web only has 1000 MHz reserved, the remaining 200 MHz needed to satisfy the VMs’ reservations is taken from the parent resource pool, Retail. But since the Retail pool has no more reservation to give, the 200 Mhz is taken instead from Retail’s parent, the root resource pool. 368 VMware Infrastructure 3: Install and Configure Admission Control for CPU and Memory Reservations Power on a VM Create a new sub-pool with its own reservation Increase a pool’s reservation Succeed Yes Can this pool satisfy reservation? No No Fail Expandable reservation? Yes – Go to Parent Pool Any action that changes a VM's or resource pool's reservation must satisfy admission control. If the object (VM or pool) resides in a pool with an expandable reservation, the current pool's parent will be consulted if need be to satisfy the reservation. 9 Resource Management Module 9 Resource Management: Using Resource Pools 369 Lab for Lesson 1 • Create and Use Resource Pools on a Standalone Host • In this lab, you will perform the following tasks: •Create two resource pools and assign resource policies to them •See the resource pools’ impact on resource allocation In the lab for this lesson, we use CPU affinity in the VMs' properties to restrict available resources to a single PCPU. This is something we do not recommend customers do in a production environment, but it's OK to do here in our training environment. 370 VMware Infrastructure 3: Install and Configure Lesson Summary • Resource pools are a way to aggregate resource policies • Resource pools are used on either standalone ESX Server hosts or DRS clusters • VMs will only power on if their resource reservations can be satisfied 9 Resource Management Module 9 Resource Management: Using Resource Pools 371 Lesson 2 Migrate VMs with VMotion Lesson Topics • VMotion migration • VMotion compatibility requirements • Topology maps : 372 VMware Infrastructure 3: Install and Configure Move VM Between ESX Servers: VMotion Migration • A VMotion migration moves a VM that is powered on • Why migrate using VMotion? • Improve overall hardware utilization • Allow continued VM operation while accommodating scheduled hardware downtime VMotion allows working processes in a virtual machine to continue throughout a migration. The entire state of the virtual machine is moved to the new ESX Server even while the data storage remains in the same datastore. The state information includes the current memory content and all the information that defines and identifies the virtual machine. The memory content includes transaction data and whatever bits of the operating system and applications are in memory. The definition and identification information stored in the state includes all the data that maps to the virtual machine hardware elements, such as BIOS, devices, CPU, MAC addresses for the Ethernet cards, and so forth. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 373 How VMotion Works (1 of 6) • Users currently accessing VM A on esx01 • Initiate migration of VM A from esx01 to esx02 while VM A is up and running VMotion Network Production Network Initiate the VMotion migration using the VI Client. In the example above, the source host is esx01 and the target host is esx02. Both source and target host have access to the shared datastore holding the VM's files. The VMotion network is the network labeled "Red". 374 VMware Infrastructure 3: Install and Configure How VMotion Works (2 of 6) • Pre-copy memory from esx01 to esx02 • Log ongoing memory changes into a memory bitmap on esx01 Memory Bitmap VMotion Network Production Network Memory The virtual machine's memory state is copied over the VMotion network (i.e. the "Red" network) from the source to the target host. While the virtual machine's memory is being copied, users continue to access the virtual machine and potentially update pages in memory. A list of modified pages in memory is kept in a memory bitmap on the source host. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 375 How VMotion Works (3 of 6) • Quiesce virtual machine on esx01 • Copy memory bitmap to esx02 VMotion Network Production Network Memory Bitmap After most of the VM's memory is copied from the source to the target host, the VM is quiesced, meaning the VM is taken to a state where no additional activity will occur on the VM. The quiesce time is the only time in the VMotion procedure in which the VM is unavailable to users and is a very minimal amount of time. During this quiesce period, VMotion starts to transfer the VM to the target host. Only the VM device state and the memory bitmap containing the list of pages that have changed are transferred over during this time. Note that if a failure occurs during the VMotion migration, the VM being migrated is failed back to the source host. For that reason, the source VM is kept around until the VM on the target host starts running. 376 VMware Infrastructure 3: Install and Configure How VMotion Works (4 of 6) • Copy VM’s remaining memory (as listed in memory bitmap) from esx01 Memory Bitmap VMotion Network Production Network Copy Pages The remaining memory (as identified in the memory bitmap) is copied from the source to the target host. Note that a virtual machine's entire network identity, including MAC and IP address, is preserved across a VMotion. Remember that you labelled your NICs on your ESX Servers. Different NICs on different ESX Servers can be associated with the same network label. For example, vmnic0 on esx01 can be associated with the "Red" network while vmnic1 on esx02 can be associated with the "Red" network as well. When performing a VMotion on a virtual machine that uses the Red network from esx01 to esx02, the VMkernel will intelligently remap the virtual machine to use vmnic1 on the target host. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 377 How VMotion Works (5 of 6) • Start VM A on esx02 VMotion Network Production Network Immediately after the VM is quiesced on the source host, the VM is initialized and starts running on the target host. Additionally, a RARP (reverse ARP) request notifies the subnet that VM A's MAC address is now on a new switch port. Additional Information on the RARP request: A RARP is sent after a VMotion. The reason for this is that until the new physical switch port that we are now behind sees a frame from the VM's MAC, the switch fabric will continue to direct frames destined for that VM to the old physical switch port it was behind before the VMotion. In most cases, the VM itself would send some type of frame anyway and the fabric would learn its new location. In other cases (the most important of which is probably the one where an eval customer pings a VM while it gets VMotion'ed to measure the perceived downtime), the VM might not send anything until it receives something, which it won't until the old entries in the physical switch's MAC tables time out (usually 30 seconds or more). Therefore, we send the RARP to update the physical switch tables proactively. The RARP is what we chose to send because it a) will cause the physical switches to update their tables and b) doesn't require any IP information (since we don't have easy access to that, i.e. the RARP just asks "who has this MAC?" so there is no IP info associated with the request). 378 VMware Infrastructure 3: Install and Configure How VMotion Works (6 of 6) • Users now access VM A on esx02 • Delete VM A from esx01 VMotion Network Production Network Users are now accessing the VM on the target host instead of the source host. The VM is finally deleted from the source host. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 379 Virtual Machine Requirements for VMotion • Migrating a VM with the following conditions produces an error: • VM has an active connection to an internal virtual switch • VM has an active connection to a CD-ROM or floppy device with a local image mounted • VM has its CPU affinity set to run on one or more specific, physical CPUs • VM is in a cluster relationship (e.g. using MSCS) with another VM • Migrating a VM with the following conditions produces a warning: • VM is configured with an internal virtual switch but is not connected to it • VM is configured to access a local CD-ROM or floppy image but is not connected to it • VM has one or more snapshots • No guest OS heartbeats are being received (due to guest OS not responding or VMware tools not configured properly) The VMotion migration will produce an error upon certain conditions, which are stated above. When an error is encountered, you must fix the error before proceeding. Likewise, VMotion will produce a warning upon certain conditions, which are also stated above. When a warning is encountered, you are allowed to proceed with the migration. The VI Client interface does a very good job of identifying warnings and errors. In the Migrate Virtual Machine Wizard, when you select the host to VMotion to, a validation check is performed, which is basically a check of all VMotion requirements. If validation succeeds, then you can continue. If validation fails, error messages will be displayed, at which point you must exit the wizard and resolve the problem. 380 VMware Infrastructure 3: Install and Configure Host Requirements for VMotion • Source and destination ESX Servers must have • Visibility to all SAN LUNs (either FC or iSCSI) and NAS devices used by VM • A Gigabit Ethernet backplane • Access to the same physical networks • Consistently labeled virtual switch port groups • Compatible CPUs •New CPU features exposed, which introduce new VMotion compatibility constraints and trade-offs There are several important host requirements for a successful VMotion migration: • SAN visibility of virtual disks • Gigabit Ethernet interconnection • Consistent network configuration, both physical and virtual • Source and destination server have CPUs from the same compatibility group The names of the virtual switches (vSwitches) on the source and destinations hosts do not have to match, and the names of the vmnic’s do not have to match. However, the vSwitch port group names have to match exactly (the match is case-sensitive). 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 381 CPU Constraints on VMotion CPU Characteristics Clock speeds, cache sizes, hyperthreading, and number of cores Manufacturer (Intel or AMD) Family (P3, P4, Opteron) Presence or absence of SSE3 or SSSE3 instructions Virtualization Hardware Assist No Exact Match Required? Why or why not? Virtualized away by VMkernel Yes Instruction sets contain many small differences Yes Multimedia instructions usable directly by applications Virtualized away by VMkernel VMware’s Intel 64-bit implementation leverages VT Guest OS relies on NX/XD bit if detected For 32-bit VMs: No For 64-bit VMs on Intel: Yes Yes (but customizable) Execution-Disable CPU compatibility between the source and target host is a VMotion requirement that must be met. This table lists various CPU characteristics and identifies whether or not an exact match is required. This table also provides a brief explanation why. For example, if hyperthreading is enabled on the source host and disabled on the destination host, the VMotion migration will continue because the VMkernel handles this difference in characteristic. Additional Information: For a description of migration options, consult the VI3 online library: http:// pubs.vmware.com/vi3/bsa/wwhelp/wwhimpl/common/html/ wwhelp.htm?context=bsa&file=BSA_Migration.17.3.html At present, we are not using AMD Pacifica (aka AMD-V). We are using Intel VT in order to run 64-bit guests on Intel hardware. Generally speaking, we do not use these HW assist technologies because the performance using our binary translation and direct execution is generally superior to this first generation of hardware assist. For more details on why that is, please see http://www.vmware.com/pdf/asplos235_adams.pdf. 382 VMware Infrastructure 3: Install and Configure Enable or Disable Nx/xD Choose between Nx/xD security features or broadest VMotion compatibility For future CPU features, edit mask at the bit level A CPU feature's effects on compatibility are dependent on whether or not ESX Server exposes or hides them from virtual machines: Features that are exposed to virtual machines are not compatible when they are mismatched; features that are not exposed to virtual machines are compatible regardless of mismatches. VirtualCenter compares the CPU features of two hosts to determine whether to allow or disallow migrations with VMotion. CPU compatibility masks allows per-virtual machine, advanced customization of the CPU features that a virtual machine should require for CPU compatibility during a VMotion migration. Default values for the CPU compatibility masks are set by VMware to guarantee the stability of virtual machines after a VMotion migration. Changes to these default masks are made very conservatively by VMware, and only when new CPU features are introduced and versions of ESX Server are updated to expose or hide them from virtual machines. In some cases, where a choice between CPU compatibility or guest operating system features (such as NX/XD) exists, the VI Client provides check-box options to configure individual virtual machines. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 383 Identifying CPU Characteristics • In most cases, use server & CPU family/model specifications • Use VMware’s CPU bootable utility Other ways to determine CPU compatibility are the following: A freeware tool named cpu-z, available at www.cpuid.com, the VMotion compatibility tool, available at www.runvirtual.com, or using the service console command line, cat /proc/ cpuinfo. VMware provides you with a CPU Compatibility tool that allows you to check CPU compatibility of hosts participating in a VMotion migration (as well as 64-bit support.) If CPU features cannot be determined directly from the server/CPU specifications, a bootable CD can be created using the CPU Compatibility tool to help you identify the characteristics of CPUs installed in a host. It can also verify whether your hardware supports 64-bit guests in ESX 3, and whether ESX Servers upgraded to ESX 3 will remain compatible for VMotion. Download this tool from the VMware Web site:http://www.vmware.com/ download/vi/drivers_tools.html. 384 VMware Infrastructure 3: Install and Configure Verify VMotion Layout: Use Maps Panel (1 of 2) To verify that the source and target ESX Servers satisfy the VMotion requirements that pertain to shared datastores and networks, display a map that shows the relationships between the hosts, datastores and networks. In the example above, we are verifying that the ESX Servers participating in the VMotion migration are attached to the same shared storage. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 385 Verify VMotion Layout: Use Maps Panel (2 of 2) You can also use maps to display the relationship between the virtual machine being migrated and the ESX Servers. In the example above, we are verifying that the virtual machine to be migrated is using a datastore that is visible to both the source and target host participating in the VMotion migration. 386 VMware Infrastructure 3: Install and Configure Verify VMotion Layout: Use Maps Tab Every virtual machine has a VMotion Resource Map that you can use to check if a virtual machine can be VMotion'ed between ESX Servers. From this map, you can determine the relationships between the virtual machine's networks, datastores and ESX Servers. To view the VMotion Resource Map, select the virtual machine in the inventory, then select the Maps tab In the example above, the virtual machine named Prod06-1 is located on sc-gallium03. Both sc-gallium03 and sc-gallium06 have access to the Production network as well as the datastore named Shared, where the files of Prod06-1 reside. Notice that sc-gallium06 has a red X. This means that a VMotion requirement has not been met. View the Events section of the Tasks & Events tab for sc-gallium06 to find out more information. If the Events screen does not help, then the validation performed with the Migration wizard probably will. 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 387 Checking VMotion Errors To initiate a VMotion migration, right-click a virtual machine that is powered on and select Migrate from the drop-down menu. In the Migrate Virtual Machine wizard, select the host to migrate to. A validation of that host is performed. If the validation does not succeed, a list of VMotion errors and/or warnings will display in the wizard’s screen. Warnings display with yellow icons and errors display with red icons. Remember that warnings will still allow you to perform a VMotion migration. VMotion errors will not allow you to continue. You must fix the error and retry the migration. 388 VMware Infrastructure 3: Install and Configure Problem: VMotion Fails • Is a problem with the virtual machine configuration preventing VMotion from working properly? • Is a problem with the source or target host configuration preventing VMotion from working properly? • Do both ESX Servers have a VMKernel port that is enabled for VMotion? If you are having problems with the VMotion migration failing, make sure that all host requirements and virtual machine requirements are met. Is a problem with a virtual machine's configuration preventing VMware VMotion from working? • VMware VMotion is designed to migrate RUNNING virtual machines. It involves moving a copy of the RAM image from one ESX Server to another. This means that you might have a virtual machine that you can "power-on" in the VI Client but, if the virtual machine is constantly crashing and rebooting, the RAM image is never stable enough for VMware VMotion to work. • An example of this would be a virtual machine that you installed as default with a guest OS of Windows 2003. By default, this virtual machine will get an LSI adapter. If you were to change this to a BusLogic adapter manually, the virtual machine would not be able to boot. It would power on, but it would continuously blue screen and reboot. • How can you tell if a virtual machine configuration problem is preventing VMware VMotion from working? Use the Remote Console to make sure that the virtual machine is actually powering-on and that you are able to login to it. Is there a VMKernel port on both ESX Servers and is it enabled for VMware VMotion? 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 389 • Look for a VMKernel port that might be configured for VMware VMotion (remember, the name of the VMkernel port may not necessarily be named “VMotion”. Are the NICs that are being used for VMware VMotion on the same physical LAN? • Remember that the vmnic label is a logical label. What is labeled vmnic4 on one ESX Server might provide the same network connectivity as vmnic2 on a different ESX Server, even if both ESX Servers share the same hardware configuration. It all depends on which vmnic is selected for the service console during installation. As a rule, if two ESX Servers have exactly the same physical configuration and if you select the same NIC on both of them for the first service console connection, then both of them will use identical logical vmnic labels for physical NICs in both hosts. • What really matters is that the physical network that these vmnics are tied to is the same LAN on both ESX Servers. Are you seeing "Error" or "Warning" messages during validation when you attempt to use VMware VMotion to move a virtual machine? • Errors are caused when VMware VMotion is attempted, but VMware VMotion does not work due to some configuration problem. Warnings are caused when VMware VMotion is attempted and is successful, but there is still something that could have been a problem. In neither case does the virtual machine crash. These messages appear on your VI console during the validation process Is there a physical network problem between the two ESX Servers preventing VMware VMotion from working? • Your configuration may be perfect, but broken network cables, disconnected network cables, failed physical switches, or failed physical NICs in the ESX Server can all cause a network connectivity problem between the two ESX Servers. Any of these problems can prevent VMware VMotion from working. If you are positive that your configuration is correct, double-check all physical components. • Note: For fault-tolerance, you may want to team two NICs on the ESX Servers when using VMware VMotion. Connect the virtual switch that has the VMKernel port for use with VMware VMotion to two or more vmnics. 390 VMware Infrastructure 3: Install and Configure Lab for Lesson 2 • Migrate Virtual Machines Using VMotion • In this lab, you will perform the following tasks: •Create a VMkernel port for VMotion •Migrate a virtual machine using VMotion ESX Server of highernumbered team must be added to VirtualCenter Server of lower-numbered team New lab requirement: Join another team’s VirtualCenter! VirtualCenter Server #3 VirtualCenter Server #4 ESX Server #3 ESX Server #4 Student 03a Student 03b Student 04a Student 04b EMPHASIZE the following before students start this lab: • The instructor, will pair up ESX Server lab teams. • The team with the higher-numbered ESX Server must remove their ESX Server from their VirtualCenter Server (steps are found in the lab) and add it to their partner ESX Server lab team. For example, if the Kentfield01 ESX Server team is paired up with the Kentfield02 ESX Server team, the Kentfield02 ESX Server team must remove their ESX Server from their VirtualCenter Server and add it to the VirutalCenter Server that Kentfield01 is using. • There are parts of the lab where students will be performing on their own ESX Server and other parts of the lab that they must perform with their "partner ESX Server team". 9 Resource Management Module 9 Resource Management: Migrate VMs with VMotion 391 Lesson 3 VMware DRS (Distributed Resource Scheduler) Lesson Topics • What is a VMware DRS cluster? • Creating a VMware DRS cluster • VMware DRS cluster settings • Automation level • Migration threshold • Placement constraints • VM swapfile location • VMware DRS best practices : 392 VMware Infrastructure 3: Install and Configure What is a DRS Cluster? • Cluster • A collection of ESX Server hosts and associated VMs • DRS-enabled cluster • Managed by VirtualCenter • Balances virtual machine load across hosts in the cluster • Enforces resource policies accurately (reservations, limits, shares) • Respects placement constraints • Affinity and anti-affinity rules • VMotion compatibility Cluster When you enable a cluster for DRS, VirtualCenter continuously monitors the distribution of CPU and memory resources for all hosts and virtual machines in the cluster. DRS compares these metrics to what resource utilization ideally should be given the attributes of the resource pools and virtual machines in the cluster and the current demand and makes migration recommendations accordingly A maximum of 32 hosts per cluster is supported. One goal of VMware DRS is to balance the load of virtual machines across all hosts in the cluster. VMware DRS considers resource policies of the virtual machines as well as any placement constraints that exist, such as anti-affinity or affinity rules as well as VMotion compatibility constraints. 9 Resource Management What is not covered in this course: VMware Distributed Power Management. VMware DPM reduces power consumption by intelligently balancing a datacenter's workload. VMware DPM, which is part of VMware DRS, automatically powers off servers whose resources are not immediately required and returns power to these servers when the demand for compute resources increases again. There is only experimental support for VMware DPM. Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 393 Create a DRS Cluster • Right-click your datacenter • Select New Cluster Name your cluster, then enable VMware DRS by selecting the check box To create a DRS cluster, right-click your datacenter, then select New Cluster from the drop-down menu. The New Cluster Wizard appears. Give your cluster a descriptive name, then select the check box next to VMware DRS to create a VMware DRS-enabled cluster. 394 VMware Infrastructure 3: Install and Configure DRS Cluster Settings - Automation Level Configure the automation level for initial placement of VMs and dynamic balancing while VMs are running Automation level Manual Partiallyautomated Fullyautomated Initial VM placement Manual Automatic Automatic Dynamic balancing Manual Manual Automatic After creating the VMware DRS cluster, define the automation level. The automation level determines how much of the decision-making process you would like to grant VMware DRS when it needs to initially place virtual machines that are powered on and when it needs to dynamically balance the load of virtual machines across hosts in the cluster. Choose from the following levels of automation: • Manual: When you power on a virtual machine, VMware DRS displays a list of recommended hosts. When the cluster becomes unbalanced, DRS displays recommendations for virtual machine migration • Partially automated: When you power on a virtual machine, VMware DRS places it on the best-suited host. When the cluster becomes unbalanced, VMware DRS displays recommendations for virtual machine migration • Fully automated: When you power on a virtual machine, VMware DRS places it on the best-suited host. When the cluster becomes unbalanced, VMware DRS migrates virtual machines from overutilized hosts to underutilized hosts to ensure a balanced use of cluster resources. 9 Resource Management Initial placement is a simplified form of dynamic balancing. Initial placement does not use VMotion because you are powering on or resuming a VM for the first time. When you power on a VM, you power it on in a resource pool. By default, DRS will automatically decide how many resources that VM is entitled to and will pick the appropriate host for it. If your automation level is manual, you must manually perform the initial Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 395 placement. A prioritized list of recommendations is presented to you to help you make good decisions. VMware DRS performs both dynamic balancing and initial placement. For dynamic balancing, VMware DRS monitors key metrics associated with virtual machines, resource pools and hosts. This information, along with the associated resource policies, are used to determine the resource allocations entitled to the virtual machines. 396 VMware Infrastructure 3: Install and Configure DRS Cluster Settings - Migration Threshold The migration threshold levels determine how quickly virtual machines are migrated Level 1 – Most conservative 2 – Moderately conservative 3 – Midpoint (default) 4 – Moderately aggressive 5 – Aggressive Apply all recommendations… with five stars only with four or more stars with three or more stars with two or more stars with one or more stars There are five migration threshold levels: • Level 1, most conservative: Applies only five-star recommendations. This level applies recommendations that must be followed to satisfy constraints such as affinity rules and host maintenance. • Level 2, moderately conservative: Applies recommendations with four or more stars. This level includes Level 1 plus recommendations that promise a significant improvement in the cluster's load balance. • Level 3, midpoint (the default): Applies recommendations with three or more stars. This level includes Level 1 and 2 plus recommendations that promise a good improvement in the cluster's load balance. • Level 4, moderately aggressive: Applies recommendations with two or more stars. This level includes Level 1-3 plus recommendations that promise a moderate improvement in the cluster's load balance. • Level 5, aggressive: Applies all recommendations. This level includes Level 1-4 plus recommendations that promise a slight improvement in the cluster's load balance. A strong 5-star recommendation should always be applied but a list of several 1-star recommendations could also collectively affect the cluster negatively if not applied. 9 Resource Management Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 397 DRS Cluster Settings - Placement Constraints • Affinity rules • Run virtual machines on same host • Use for multi-VM systems where performance benefits • Anti-affinity rules • Run virtual machines on different hosts • Use for multi-VM systems that load balance or require high availability After you have created a DRS cluster, you can edit its properties to create rules that specify affinity. You can use these rules to determine that: • DRS should try to keep certain virtual machines together on the same host (for example, for performance reasons) • DRS should try to make sure that certain virtual machines are not together (for example, you might want to guarantee certain virtual machines are always on different physical hosts, so if there is a problem with one host, you do not want to lose both virtual machines) The example above shows an anti-affinity rule that requires two database servers to be placed on different hosts, most likely for availability and perhaps performance reasons. Conversely, there are affinity rules, where you might want to keep certain virtual machines on the same host because of increased locality or performance benefits, for example, VM-to-VM networking that uses internal-only (instead of physical) networking. 398 VMware Infrastructure 3: Install and Configure Examples of affinity and anti-affinity rules: • Example for an anti-affinity rule: A VM that uses lots of resources. Customers are now virtualizing large systems for the purpose of easier DR. Consolidation is not the driving factor in this case. Large virtualized hosts would best be kept on separate ESX Servers to preserve at least some ability to consolidate other small VMs with it on the same ESX Server. • Another use of an anti-affinity rule is availability. Configure DRS to never run two critical applications on the same host. • An affinity rule might be useful to keep two memory intensive applications with similar working sets located on the same host in order to derive maximum benefit from transparent page sharing. 9 Resource Management Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 399 DRS Cluster Settings - Automation Level per VM • Optionally set automation level per VM You can customize the automation level for individual virtual machines in a DRS cluster to override the automation level set on the entire cluster. This allows you to fine tune automation to suit your needs. For example, there may be a virtual machine that is especially critical to your business and you would like more control over its placement, therefore set its automation level to Manual. If a virtual machine is set to Disabled, VirtualCenter does not migrate that virtual machine or provide migration recommendations for it. 400 VMware Infrastructure 3: Install and Configure DRS Cluster Settings - VM Swapfile Location • Store VM’s swapfile with VM or in a specified datastore By default, swapfiles for a virtual machine are located on a VMFS datastore in the folder containing the other virtual machine files. However, you can instead configure the hosts in your cluster to place virtual machine swapfiles on an alternative datastore of your choice. You might use this option to place virtual machine swapfiles on either lower cost or higher performance storage, depending on your needs. If the swapfile location specified on the destination host differs from the swapfile location specified on the source host, the swapfile is copied to the new location. This can result in slower migrations with VMotion. For best VMotion performance, store virtual machine swapfiles in the same directory as the virtual machine. 9 Resource Management Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 401 Add Hosts to Cluster • Drag-and-drop ESX Server onto cluster Drag-and-drop • Use the Add Host Wizard to complete the process To add a host to a VMware DRS cluster, drag-and-drop an ESX Server onto the cluster object in the inventory. The Add Host Wizard appears. Work through the wizard to complete the process of adding a host to the cluster. 402 VMware Infrastructure 3: Install and Configure Best Practices for DRS • When DRS makes strong recommendations (typically 4- or 5star), follow them • Otherwise, balance and fairness may deteriorate • Some VMotion is necessary • Enable automation • Choose default based on environment, comfort level • Let DRS autonomously manage most VMs • Use per-VM automation level overrides to accommodate sensitive VMs It is important to follow any strong recommendations that DRS recommends. Otherwise if you leave DRS in manual mode and you do not follow any of its recommendations, balance and fairness in the cluster may deteriorate. Another best practice is to enable some level of automation. The default that you choose will be based on your experience with DRS, as well as the knowledge you have about your environment. Note that there are cluster-wide controls and per-VM controls. It is recommended that DRS autonomously manage most of your VMs. However, for any critical VMs, keep a human in a loop to approve all VMotion operations for that VM. For example, use a default of manual for your critical VMs. For your non-critical VMs, such as test/development VMs, specify automatic movement by DRS. 9 Resource Management Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 403 Lab for Lesson 3 • Create a DRS Cluster • In this lab, you will perform the following tasks: •Create a DRS cluster •Add ESX Servers to the DRS cluster Two ESX Server teams belong to one Cluster team VirtualCenter Server #3 VirtualCenter Server #4 ESX Server #3 ESX Server #4 Student 03a Student 03b Student 04a Student 04b Cluster Team 404 VMware Infrastructure 3: Install and Configure Lesson Summary • DRS applies intelligence to the location of VMs • Upon initial power-on • Dynamically (using VMotion) • Accepting DRS’s recommendations leads to balanced resource utilization 9 Resource Management Module 9 Resource Management: VMware DRS (Distributed Resource Scheduler) 405 Lesson 4 Resource Pools in a VMware DRS Cluster Lesson Topics • The role of resource pools in DRS clusters • Using pools for delegated administration • Monitoring the state of resource use in a pool • Adding hosts with resource pools to a cluster : 406 VMware Infrastructure 3: Install and Configure Resource Pools in a DRS Cluster Resource pools are used to subdivide the computing resources in a cluster Root Resource Pool: CPU = 20 GHz (10 x 2 GHz) Memory = 20 GB Cluster Resource Pool 1 (CPU = 12 GHz, Memory = 12 GB) CPU Shares: 2000 Reservation: 4 GHz Limit: 12 GHz VM CPU Shares: 4000 Reservation: 0 Limit: 4 GHz VM CPU Shares: 1000 Reservation: 0 GHz Limit: 2 GHz Resource Pool 2 (CPU = 8 GHz, Memory = 4 GB) CPU Shares: 4000 Reservation: 0 Limit: 8 GHz VM VM VM Resource pools can be used to divide the CPU and memory resources of a standalone host. Resource pools can also be used with a VMware DRS cluster, which allows you to manage the resources of all hosts in the cluster as a single pool of resources. With resource pools, you can hierarchically organize virtual machines and isolate resource pools so that you can control the amount of resources for a whole collection of virtual machines. Resource pools can be created only on ESX standalone hosts or VMware DRS-enabled clusters. Clusters that have only VMware HA-enabled (and not VMware DRS) cannot use resource pools. 9 Resource Management Module 9 Resource Management: Resource Pools in a VMware DRS Cluster 407 Delegated Administration • Joe administers cluster • Has “Datacenter Administrator” VC role • Carves up cluster Joe resources into pools, provides bulk allocations to pool admins Cluster (Root Resource Pool) • Jane administers Resource Pool 1 • Has “Resource Pool Administrator” VC role • Carves up pool resources into smaller pools for users Jane Resource Pool 1 (CPU = 12 GHz Mem = 12 GB) Resource Pool 2 (CPU = 8 GHz Mem = 4 GB) Resource Pool 3 (CPU = 8 GHz Mem = 4 GB) Resource Pool 4 (CPU = 4 GHz Mem = 4 GB) VM VM VM • Ted administers VMs in Resource Pool 3 • Has “Virtual Machine Power User” VC role • Allocates resources to VMs Ted VM VM VM VM A pool can reflect any organizational structure that makes sense to you, such as a pool for each department, or a project or a client, etc. You can associate access control and permissions to different levels in the resource pool hierarchy. For example, you can have a cluster-wide administrator defined at the cluster level (which is the root resource pool.) The cluster administrator can then carve up the aggregate resources of the entire cluster into pools and provide bulk allocations to sub-administrators or administrators for individual resource pools. A cluster administrator is given at least the Datacenter Administrator role. Each pool administrator can take the resources that he or she has been allocated and carve them up into smaller resource pools for end users. A pool administrator is given the role of Resource Pool Administrator. Finally, each end user can allocate resources from his or her pool to the virtual machines that they care about. An end user is given at least the Virtual Machine Power User role. The key to understanding and using delegation is to understand roles and their privileges. It will be very beneficial to use the VI Client to explore and gain familiarity with the privileges assigned to each role. 408 VMware Infrastructure 3: Install and Configure Monitor Cluster Usage • View the inventory hierarchy for the cluster state • View the cluster’s Tasks & Events tab for further information The VirtualCenter inventory hierarchy indicates whether a cluster is valid, overcommitted (yellow), or invalid (red): • Valid: A cluster is valid unless something happens that makes it overcommitted or invalid. In a valid cluster, there are enough resources to meet all reservations and to support all running virtual machines. • Overcommitted (Yellow): A cluster becomes yellow if it does not have enough capacity to satisfy the constraints it was originally configured with. A cluster typically turns yellow when cluster capacity is suddenly reduced, for example, when a host in the cluster goes down. It is recommended that you leave adequate additional resources in the cluster to avoid having your cluster turn yellow. 9 • Invalid (Red): A cluster enabled for DRS becomes red when the tree is no longer internally consistent and does not have enough resources available. The total resources in the cluster have nothing to do with whether the cluster is yellow or red. It is possible for the cluster to be DRS red even if there are enough resources at the root level, if there is an inconsistency at a child level. For example, a DRS cluster turns red if the virtual machines in a fixed resource pool use more resources than the Reservation of that resource pool allows. For more information on cluster states, see the VI3 Resource Management Guide (http://www.vmware.com/pdf/vi3_301_201_resource_mgmt.pdf). Resource Management Module 9 Resource Management: Resource Pools in a VMware DRS Cluster 409 When using DRS clusters, we discourage bypassing VirtualCenter and making changes to the resource pool directly on the host. So why would someone want/need to do that in the first place? • If the VirtualCenter Server goes down, then you can access your hosts by pointing the VI Client directly to your ESX Server. Again, this is discouraged, specifically if this ESX Server is part of DRS Cluster. If it is, always try to make resource pool changes from VirtualCenter. More information on the Red DRS Cluster State: • You can resolve a red DRS cluster problem either by powering off one or more virtual machines, moving virtual machines to parts of the tree that have sufficient resources, or editing the resource pool settings in the red part. Adding resources typically helps only when you're in the yellow state, not in the red state. A cluster can also turn red if you reconfigure a resource pool while a virtual machine is in the process of failing over. A virtual machine that is in the process of failing over is disconnected and does not count toward the reservation used by the parent resource pool. So it is possible that you reduce the reservation of the parent resource pool before the failover completes. Once the failover is complete, the virtual machine resources are again charged to the parent resource pool. If the pool's usage becomes lager than the new reservation, the cluster turns red 410 VMware Infrastructure 3: Install and Configure Adding Host to DRS Cluster • When adding a new host or moving an existing host into the DRS cluster, you have the option of keeping the resource pool hierarchy of the existing host, if one exists • For example, add kentfield04 to Lab Cluster When adding the host, choose to create a new resource pool for this host’s virtual machines and resource pools. When you add a host with resource pools to a DRS cluster, you must decide on resource pool placement. By default, the resource pool hierarchy is discarded and the host is added at the same level as the virtual machines. You can choose to graft the host's resource pools onto the cluster's resource pool hierarchy and choose a name for the resource pool created to represent the host’s resources. By default, the resource pool created to represent the host’s resources is named “Grafted from host_name", but you can choose a different name. The term grafted was chosen because the branches of the host's tree are added to the branches of the cluster's tree, just as fruit tree branches are grafted onto rootstock. 9 Resource Management Module 9 Resource Management: Resource Pools in a VMware DRS Cluster 411 Planned Downtime: Maintenance Mode • Maintenance mode restricts VM operations on the host so that VMs can be shut down or VMotion’ed in preparation for host shut down or removal from a cluster • Applies to both standalone hosts and hosts within a cluster • As a host goes from normal to maintenance mode, VM operations become restricted: Normal mode You can power on VMs as needed, and VMs can be migrated to this host All running VMs must either be shut down or migrated to other hosts; no new VMs can be powered on; no VMs will be migrated to this host Maintenance mode All VMs have been manually powered off or migrated to other hosts, and no new VMs can be powered on; no VMs will be migrated to this host Maintenance mode restricts the virtual machine operations on the ESX Server to allow you to conveniently shut down running virtual machines, or VMotion virtual machines to other ESX Servers. Place an ESX Server into maintenance mode if you are going to: • Shut down the ESX Server • Add the ESX Server to a cluster • Remove the ESX Server from a cluster Before entering maintenance mode, all virtual machines on that host must either be shut down or VMotion'ed to other hosts in the cluster. When a host is in maintenance mode, no new virtual machines can be powered on and no virtual machines will be migrated to this host. If a DRS cluster is set to the fully automated level, the VMs on the server that is placed in maintenance mode will automatically be moved off that server onto the remaining host(s) in the cluster. If the DRS cluster is set to the partially automated level, the administrator has to manually move the VMs to a new host or power them down. To place a host in maintenance mode, select the host in the inventory, then click Enter Maintenance Mode in its Summary tab. After the host is placed in maintenance mode, its icon changes to reflect this state. Once the host is in maintenance mode, you can safely shut down the host. Both standalone hosts and hosts within a cluster support maintenance mode. 412 VMware Infrastructure 3: Install and Configure Problem: Cannot Power on VM (1 of 2) • Error: Insufficent memory resources • What does the failing feature depend on? • Memory resources: Of the cluster, ESX Server, resource pool, VM? The example above shows the virtual machine named Prod03 is failing to power on. If a user tries to power on a virtual machine, but the task fails with the error, "Insufficient memory resources”, the virtual machine is failing because there is not enough memory to power it on. What memory is the error referring to? The physical memory of the ESX Server? The maximum memory size of the virtual machine? The memory reservation of the virtual machine? The memory reservation or limits of the resource pool in which the virtual machine is located? Once you determine this, you can then decide how to resolve the problem. Here are ways to check various memory values: 9 • To view physical memory size and memory usage of an ESX Server: select the Memory link in its Configuration tab • To view memory size and memory overhead of a virtual machine: view the virtual machine's Summary tab • To view memory reservation and limit of a virtual machine, view the virtual machine's memory resources in its Properties settings • To view memory reservation and limit of a resource pool: view its Summary tab and Resource Allocation tab • To view memory reservation and limits of all virtual machines in a resource pool, view the resource pool's Resource Allocation tab • To view memory total, reservation and limit of a cluster: view its Summary tab and Resource Allocation tab Resource Management Module 9 Resource Management: Resource Pools in a VMware DRS Cluster 413 Problem: Cannot Power on VM (2 of 2) • View VMs’ memory reservations • Check amount of unreserved memory in resource pool • Modify memory reservation of one or more VMs, or of the resource pool To continue with this example, let’s look at the Resource Allocation tab of the Production resource pool. The virtual machine, Prod03, has a memory reservation of 128 MB. The Production resource pool has a memory reservation of 1024 MB. Of that reservation, 142 MB is unreserved memory. It would seem that Prod03 should be able to power on, since there appears to be enough unreserved memory available. However, every virtual machine that is powered on incurs some amount of memory overhead. A virtual machine’s memory overhead is listed in its Summary tab. In this example, the amount of memory overhead for Prod03 is 64.36 MB (see graphic on previous page). Therefore, Prod03 needs approximately 193 MB to power on. To resolve the problem, you can lower the memory reservation of one or more of the virtual machines in the Production resource pool, or increase the memory reservaton of the Production resource pool itself. 414 VMware Infrastructure 3: Install and Configure Lab for Lesson 4 • Resource Pools in a DRS Cluster • In this lab, you will perform the following tasks: •Create two resource pools in a DRS cluster •Cause DRS to make resource balancing recommendations Two ESX Server teams belong to one Cluster team VirtualCenter Server #3 VirtualCenter Server #4 ESX Server #3 ESX Server #4 Student 03a Student 03b Student 04a Student 04b Cluster Team 9 Resource Management Module 9 Resource Management: Resource Pools in a VMware DRS Cluster 415 Lesson Summary • Use DRS clusters to delegate the right to allocate resources • When DRS clusters are in use, do not manually make changes to individual hosts’ resource pools 416 VMware Infrastructure 3: Install and Configure Module Summary • A resource pool has three attributes – reservation (expandable), limit and shares • Resource pools can be created on standalone hosts or in DRS clusters • VMotion is the underlying technology of VMware DRS • A DRS cluster provides initial placement of VMs at power on and dynamic load balancing of running VMs 9 Resource Management Module 9 Resource Management: Resource Pools in a VMware DRS Cluster 417 Questions? Questions? 418 VMware Infrastructure 3: Install and Configure MODULE 10 Resource Monitoring 10 Importance • Although the VMkernel works proactively to avoid resource contention, maximizing performance requires both analysis and ongoing monitoring Objectives for the Learner • To monitor a VM’s performance • To determine whether a VM is constrained by a resource, and solve the problem if one exists Module Lessons • Tools for Resource Optimization • Monitor VM Performance • Monitoring Using Performance-based Alarms 10 Resource Monitoring VMware Infrastructure 3: Install and Configure 419 Lesson 1 Tools for Resource Optimization Lesson Topics • Virtual CPU concepts • Virtual memory concepts • Transparent page sharing • Balloon-driver mechanism • VMkernel swap file : 420 VMware Infrastructure 3: Install and Configure Systems for Optimizing VM Resource Use These are the different parameters and features that we can use to control a virtual machine's access to CPU, memory, disk bandwidth and network bandwidth. We will discuss allocating each of these resources in this module. The mechanisms in the left column are those automatically managed by the VMkernel. Those in the middle column are used at the discretion of each virtual machine's owner. Those in the right column are those used by the administrator to set virtual machine-wide policies. The term "VMkernel swap" refers to the VMkernel swap file that the VMkernel creates for each VM that is powered on. Try to avoid referring to this file as the "virtual machine swap file" because students might get that confused with the swap file used by the guest OS (e.g. pagefile.sys) within the VM itself. These swap files are entirely different from each other. 10 Resource Monitoring Module 10 Resource Monitoring: Tools for Resource Optimization 421 Virtual CPUs • A virtual machine can have 1, 2 or 4 virtual CPUs (VCPUs) • When a VCPU needs to be scheduled, the VMkernel maps a VCPU to a “hardware execution context” • A “hardware execution context” is a processor’s capability to schedule one thread of execution H.E.C. H.E.C. H.E.C. H.E.C. H.E.C. H.E.C. H.E.C. A virtual machine can be configured with 1, 2 or 4 virtual CPUs (VCPUs). When a VCPU needs to be scheduled, the VMkernel maps a VCPU to a hardware execution context (H.E.C.). A hardware execution context is a processor's capability to schedule one thread of execution. A single-CPU VM gets scheduled on one hardware execution context at a time. A 2VCPU VM gets scheduled on two hardware execution contexts at a time, or none. A 4-VCPU VM gets scheduled on four hardware execution contexts at a time, or none. You may run 2-VCPU VMs only on physical machines with 2 or more H.E.C.'s. Likewise, you may run 4-VCPU VMs only on physical machines with 4 or more H.E.C.'s. 422 VMware Infrastructure 3: Install and Configure Hardware Execution Contexts Different systems provide different numbers of hardware execution contexts Single-Core, Dual-Socket System Dual-Core, Single-Socket System (Hyper-Threading Not Enabled) Quad-Core, Single-Socket System The number of hardware execution contexts available for scheduling depends on the type of system being used. For example, a single-core, dualsocket system has two cores and therefore, without Hyper-Threading enabled, has two hardware execution contexts. In general, a socket is another term for the entire physical processor package. A socket contains one or more CPUs in the same package. Each of these CPU equivalents is a core. For example, a single-core, dual-socket system has two sockets with one core in each socket, and a dual-core, single-socket system has one socket containing two cores. In relation to hardware execution contexts, a dual-core, single-socket system has two cores and therefore, two hardware execution contexts (without Hyper-Threading enabled.) A quad-core, single-socket system has four cores and therefore, four hardware execution contexts (without HyperThreading enabled.) 10 Resource Monitoring Module 10 Resource Monitoring: Tools for Resource Optimization 423 Hyper-Threading • Enables a core to execute two threads, or sets of instructions, at the same time • Provides more hardware execution contexts for VCPUs to be scheduled • However, it does not double the power of the core Single-Core, Dual-Socket System Dual-Core, Single-Socket System Hyper-Threading is a technology developed by Intel that enables a core to execute two threads, or sets or instructions, at the same time. The benefit of Hyper-Threading is more scheduler throughput, i.e. Hyper-Threading provides more hardware execution contexts on which VCPUs can be scheduled. The downside of Hyper-Threading is that it does not double the power of a core. Therefore, if both threads of execution need the same onchip resources (for example, the floating-point unit) at the same time, one thread will have to wait. For best performance, run 2-VCPU VMs only on physical machines with more than 2 H.E.C.'s, and run 4-VCPU VMs only on physical machines with more than 4 H.E.C.'s. If these virtual machines are CPU-intensive, ignore the fact that hyper-threading is enabled, if it is. For example, let's say that you have a dual-core, single-socket system with Hyper-Threading enabled. This system provides 4 H.E.C.'s. If a virtual machine is CPU intensive, the VMkernel will dynamically try to refrain from using the other thread in the core. Therefore, a 2-VCPU VM that is CPU-intensive will fare better on this system than a 4-VCPU, CPU-intensive VM. Hyper-Threading must be enabled in your server's BIOS. On some server models, the option is named "Enable Logical Processors." There is a Hyper-Threading whitepaper, available at http://www.vmware.com/support/resources/esx_resources.html 424 VMware Infrastructure 3: Install and Configure VMkernel CPU Load Balancing • VMkernel dynamically schedules virtual machines and the service console • Service console always runs on the first hardware execution context • For multi-VCPU, CPU-intensive VMs, the VMkernel tries to avoid scheduling their VCPUs on hardware execution contexts in the same core Hyper-Threaded, Dual-Core, Dual-Socket System The VMkernel dynamically schedules virtual machines and the service console onto the hardware execution contexts. By default, the VMkernel looks every 20 milliseconds for virtual machines to migrate from one hardware execution context to another. The service console always runs on the first hardware execution context and is never migrated to another one. The VMkernel decides on what hardware execution context a VCPU runs. In general, when mapping VCPUs to hardware execution contexts, the VMkernel's main goal is to balance the load. With multiple-VCPU VMs, a VMkernel may decide to map the VM's VCPUs to hardware execution contexts on different sockets, on different cores in the same socket, or on different threads in the same core. The VMkernel tries its best to avoid scheduling the VCPUs of a CPU-intensive, multi-VCPU VM on threads (i.e. hardware execution contexts) in the same core. However, if necessary, the VMkernel could map two VCPUs from the same VM to threads on the same core. 10 Resource Monitoring Module 10 Resource Monitoring: Tools for Resource Optimization 425 Transparent Memory Page Sharing • VMkernel detects identical pages in VMs’ memory and maps them to the same underlying physical page • No changes to guest OS required • VMkernel treats the shared pages as copy-on-write • • Read-only when shared Private copies after write • Page sharing is always active unless administratively disabled In this example, a page of physical memory is in common, but marked readonly at the hardware level, across all VMs shown in the slide. If any individual VM tries to write to the page, the VMkernel detects that as a fault, it takes a private copy of the page and breaks the share (pointer). For example, let's say the VM in the middle tried to write to this page; the arrow would go away, the VMkernel allocates another page created in real physical memory, copies the content, swings the pointer over, then continues computing. The VMkernel detects when different VMs have memory pages with identical content, and arranges for those pages to be shared. That is, a single physical page is mapped into each VM's address space. If any VM tries to modify a page that is (unbeknownst to it) shared, the VMkernel will create a new, private copy for that VM, and then map that page into the address space of that VM only. The other VMs continue to share the original copy. Transparent page sharing is enabled by default. The system dynamically scans memory looking for duplicate pages. This mechanism is a way in which ESX Server tries proactively to conserve physical memory, so that it will not have to resort to any of the other techniques. When a virtual machine has been suspended and gets resumed, it does not participate right away in the memory-sharing system. Its pages become shared over time. So if you plan to suspend and resume large batches of VMs, don't scrimp on memory. 426 VMware Infrastructure 3: Install and Configure vmmemctl: The Balloon-Driver Mechanism • Deallocate memory from selected virtual machines when RAM is scarce ample memory; balloon remains uninflated inflate balloon (driver demands memory from guest OS) guest is forced to page out to its own paging area; VMkernel reclaims memory deflate balloon (driver relinquishes memory) guest may page in; ESX Server grants memory When a VM needs to yield memory, it's in everyone's best interest to let the guest OS in that VM pick which pages of memory to give up. It knows which pages have been least recently used and which pages can easily be refreshed from some backing store on disk. This is what vmmemctl achieves; a balloon driver is installed in the guest OS when you install VMware Tools. The balloon driver installs as a device driver, but its only function is to demand memory from the guest OS and later to relinquish it, under the control of the VMkernel. VMs are ignorant of this entire mechanism. This mechanism is out of their view. When a system is not under memory pressure, no VM's balloon is inflated. But when memory becomes scarce, the VMkernel chooses a VM and inflates its balloon: that is, it tells the balloon driver in that VM to demand memory from the guest OS. The guest OS complies by yielding memory, according to its own algorithms; the relinquished pages can be assigned by the VMkernel to other VMs. Whether a VM loses memory because of the balloon driver is determined by its relative share allocation. The term "balloon" driver is an informal term often used to refer to the vmmemctl device driver, which is used to perform memory deallocation/ reallocation. 10 Resource Monitoring Module 10 Resource Monitoring: Tools for Resource Optimization 427 VMkernel Swap • Each powered-on VM needs its own VMkernel swap file • Automatically allocated on first poweron • Default location: same VMFS volume as virtual machine’s boot disk • Size equal to the difference between the memory guaranteed to it, if any, and the maximum it can use • This file lets the VMkernel swap the VM out entirely if memory is scarce • Use of VMkernel swap is a last resort • Performance will be noticeably slow When a virtual machine is powered on for the first time, the system allocates a VMkernel swap file for it. This file will serve as backing store for the virtual machine's RAM contents. In the event that the VMkernel needs to reclaim some or all of this virtual machine's memory, and if the balloon driver cannot free enough memory, the VMkernel will copy pages' contents to the VMkernel swap file before giving them to other virtual machines. The size of the VMkernel swap file is determined by the difference between how much memory the virtual machine can use (its limit, if no limit is defined, or the amount configured into the virtual hardware) and how much RAM is reserved for it (its reservation). Whenever VMkernel swap is being actively used, performance is not optimal. Configure your server systems so that all virtual machines' normal running memory needs (as determined by monitoring under load) can be accommodated using physical memory. When you power off the VM, the VMkernel swap file of the VM is deleted. When the VM is powered back on, the VMkernel swap file for the VM is recreated. 428 VMware Infrastructure 3: Install and Configure Ballooning vs. VMkernel Swapping Limit MB 100% Balloon Limit* Reservation MB 35% 30% 0 MB 0% *Up to 65% or Reservation, whichever comes first By default, up to 65% of a VM's memory can be taken away during the ballooning process, subject of course to the memory reservation setting. An advanced VMkernel setting named Mem.CtlMaxPercent controls this value. By default, it is 65% but can be set between 0-75%. In the example above, the VM's memory reservation is set equal to 30% of the VM's memory. Under heavy contention, the VMkernel could request up to 70% of this VM's memory to be reclaimed and given to other VMs. But only 65% could be ballooned away, which means the last 5% would have to be VMkernel-swapped. Swapping is less desirable than ballooning. The drawing illustrates that by default a maximum of 65% of the VM can be paged out via the ballooning mechanism. If 65% of the VM's memory was ballooned out that would leave 35% of VM memory in physical memory. If then, the reservation is set to anything under that 35%, then VMkernel swapping would have to remove the rest to the swap file. One of the main points is that the administrator should not set the reservation too low as that might force VMkernel swapping during periods of contention. 10 Resource Monitoring Module 10 Resource Monitoring: Tools for Resource Optimization 429 Lesson Summary • A hardware execution context is a processor’s capability to schedule one thread of execution • Transparent page sharing is a way for the ESX Server to proactively conserve physical memory • Because performance will be noticeably slow, use of VMkernel swap is a last resort 430 VMware Infrastructure 3: Install and Configure Lesson 2 Monitor VM Performance Lesson Topics • Virtual machine performance graphs • Monitoring a VM’s • CPU • Memory • Disk • Network : 10 Resource Monitoring Module 10 Resource Monitoring: Monitor VM Performance 431 Performance Tuning Methodology Assess performance • Record a numerical benchmark before changes Identify the limiting resource Make more resource available • Allocate more • Reduce competition • Log your changes! Benchmark again Don’t make casual changes to production systems! The best practice for performance tuning is to take a logical step-by-step approach, especially when working on production systems. An ESX Server is well-tuned when high-priority VMs are running with maximum performance, possibly at the expense of lower-priority VMs. 432 VMware Infrastructure 3: Install and Configure Monitoring VM Resource Use with Performance Graphs The target (host or VM) Export to Excel Tear off this chart Units Modify what is graphed Items being graphed Statistics for displayed range For each host and virtual machine, the Virtual Infrastructure client offers a Performance tab. This tab offers both a real-time view and a historical view of many performance counters. For more formatting and analysis options, you may export the data being graphed to Microsoft Excel. For side-by-side comparisons of several virtual machines or hosts, tear off each's performance graph. It will be dynamically refreshed. 10 Resource Monitoring Module 10 Resource Monitoring: Monitor VM Performance 433 Tools for Improving VMs' CPU and Memory Performance Broad Add capacity to DRS cluster Modify resource pool’s CPU and memory limits and reservations Fine Modify VM’s CPU and memory limits and reservations You may control a virtual machine's access to CPU and memory at three levels. You may define limits, reservations, and shares on individual virtual machines; however, you are likely to find this difficult to manage as you have more and more virtual machines. A more scalable approach is to organize your virtual machines into resource pools, placing virtual machines with similar needs and levels of criticality into the same resource pool. You may then define limits, reservations, and shares on the resource pool itself. If you have an active VMware DRS cluster in fully automated mode, you have one still-higher point of control. You can add ESX Server instances to your cluster; the system will automatically VMotion virtual machines so as to reduce contention. The more CPU and memory resources in your cluster, the higher CPU and memory reservations you can define, and the more your virtual machines are insulated from competition. 434 VMware Infrastructure 3: Install and Configure Are VMs Being CPU-Constrained? Task Manager inside VM VM’s CPU ready graph in VI Client • If VM is constrained by CPU • Add shares or increase CPU reservation • VMotion this virtual machine • Shut down, VMotion, or remove shares from other VMs The key indicator of a virtual machine losing competition for CPU time is "CPU ready" time in its CPU resource graph. Ready time refers to the interval when a virtual machine is ready to execute instructions, but cannot because it cannot get scheduled onto a CPU. Note that CPU Ready values only show up in the "Real Time" graph and not in any of the historical graphs (in other words, the day, week, month, or year graphs.) Several factors affect the amount of ready time seen: • Overall CPU utilization: You're more likely to see ready time when utilization is high, because the CPU is more likely to be busy when another VM becomes ready to run. • Number of resource consumers (in this case, guest OSes): When a host is running a larger number of VMs, the scheduler is more likely to need to queue a VM behind one or more that are already running or queued. • Load correlation: If loads are correlated, for example, if one load wakes another one when the first load has completed its task, ready times are unlikely. If a single event wakes multiple loads, high ready times are likely. • Number of virtual CPUs in a virtual machine: When co-scheduling for n-way Virtual SMP is required, the virtual CPUs can be scheduled only when n physical CPUs are available to be preempted. A good ready time value varies from workload to workload. To find a good ready time value for your workload, collect ready time data over time for each virtual machine. Once you have this ready time data for each virtual machine, estimate how much of the observed response time is ready time. Module 10 Resource Monitoring: Monitor VM Performance 435 10 Resource Monitoring If the shortfalls in meeting response time targets for the applications appear largely due to the ready time, then take steps to address the excessive ready time, as mentioned in the slide above. CPU time is tabulated on a per-virtual-CPU basis. To display it, choose one or more of the virtual CPUs in the virtual machine. In our example above, we chose our uniprocessor virtual machine's only VCPU, number 0. For more information, consult the technical paper, "VMware ESX Server 3 Ready Time Observations", available on the VMware website at http://www.vmware.com/pdf/esx3_ready_time.pdf. 436 VMware Infrastructure 3: Install and Configure Are VMs Being Memory-Constrained? Task Manager inside VM • If VM is constrained by memory • Add shares or raise memory reservation • VMotion this virtual machine • Shut down, VMotion, or remove shares from other virtual machines When a virtual machine is losing the competition for memory, the balloon driver will force it to yield memory. Trace this amount using a memory resource graph. Note that the ballooning values only show up in the "Real Time" graph and not in any of the historical graphs (in other words, the day, week, month, or year graphs.) Check for high ballooning activity 10 Resource Monitoring Module 10 Resource Monitoring: Monitor VM Performance 437 Are VMs Being Disk-Constrained? • Disk-intensive applications can saturate the storage or the path • If you suspect that a VM is constrained by disk access • Measure the effective bandwidth between VM and the storage • Measure the resource consumption using performance graphs • To improve disk performance • Ensure VMware Tools is installed • Reduce competition • Move other VMs to other storage • Use other paths to storage • Reconfigure the storage • Ensure that the storage’s RAID level and cache configuration suit the application Disk performance problems are commonly caused by saturating the underlying physical storage hardware. Use a tool like IOMETER (shown) to measure the maximum throughput via the current path to the storage. Note that disk access values only show up in the "Real Time" graph and not in any of the historical graphs (in other words, the day, week, month, or year graphs.) 438 VMware Infrastructure 3: Install and Configure Are VMs Being Network-Constrained? • Network-intensive applications will often bottleneck on path segments outside ESX Server • Example: WAN links between server and client • If you suspect that a VM is constrained by the network • Confirm VMware Tools is installed • Measure the effective bandwidth between VM and its peer system • Examine performance graphs • To improve network performance • Move VMs to another physical NIC • Traffic-shape other VMs • Reduce overall CPU utilization Like disk performance problems, network performance are commonly caused by saturating some network link between client and server. Use a tool like IOMETER, or a large file transfer, to measure the effective bandwidth. 10 Resource Monitoring Module 10 Resource Monitoring: Monitor VM Performance 439 Lab for Lesson 2 • Monitor Virtual Machine Performance • In this lab, you will perform the following tasks: •Add a second disk to a virtual machine •Monitor the second disk activity using VirtualCenter •Monitor CPU Ready time using VirtualCenter Student 03a Student 03b Student 04a Student 04b This lab will be performed by each ESX Server team separately VirtualCenter Server #3 ESX Server #3 ESX Server #4 ESX Server Team #3 ESX Server Team #4 440 VMware Infrastructure 3: Install and Configure Lesson Summary • The VI Client offers both real-time and historical views of many performance counters • The key indicator of a virtual machine losing competition for CPU time is "CPU ready" time • High ballooning activity can indicate that a virtual machine is memory-constrained 10 Resource Monitoring Module 10 Resource Monitoring: Monitor VM Performance 441 Lesson 3 Monitoring Using Performancebased Alarms : Lesson Topics • VM-based alarms • Host-based alarms 442 VMware Infrastructure 3: Install and Configure What is an Alarm? • VirtualCenter alarms report changes in host or VM state Alarms are indicated in the inventory Status determined by threshold levels in alarm definition View of VMs’ CPU and memory utilization on selected host Alarms are asynchronous notifications of changes in host or virtual-machine state. When a host or virtual-machine's load passes certain configurable thresholds, the VI Client will display messages to this effect. You can also configure VirtualCenter to transmit these messages to external monitoring systems. 10 Resource Monitoring Module 10 Resource Monitoring: Monitoring Using Performance-based Alarms 443 Creating a VM-Based Alarm • Right-click on a VM and choose “Add Alarm…” Name and describe the new alarm Click any field to modify Percentages Powered on, powered off, suspended When you right-click on a virtual machine and choose Add Alarm..., the resulting window has four panels. Visit the General panel to name this alarm. Visit the Triggers panel to control which load factors are monitored, and what the threshold for the yellow and red states are. We will discuss the Reporting and Actions panels in upcoming slides. 444 VMware Infrastructure 3: Install and Configure Creating a Host-Based Alarm • Right-click on a host and choose “Add Alarm…” Name and describe the new alarm Click any field to modify Percentages Connected, disconnected, not responding The dialogue box displayed when you right-click on a host and choose Add Alarm... is very similar to that for a virtual machine. The key difference is the list of available triggers. 10 Resource Monitoring Module 10 Resource Monitoring: Monitoring Using Performance-based Alarms 445 Alarm Reporting Options • Use the Reporting pane to avoid needless re-alarms Avoid small fluctuations Avoid repeats If you plan to transmit alarms to some external monitoring system, such as an SNMP monitoring tool, someone's email, or someone's pager, you probably want to avoid generating a flood of duplicate alarms. Use the controls on the Reporting pane to avoid such a flood. 446 VMware Infrastructure 3: Install and Configure Actions to Take When an Alarm is Triggered • Use the Actions pane to send external messages or to respond to problems proactively Only available for VM-based alarms You may specify one or more actions to occur when an alarm is triggered (other than simply displaying it in the VI Client). 10 Resource Monitoring Module 10 Resource Monitoring: Monitoring Using Performance-based Alarms 447 Using Alarms to Monitor CPU and Memory Usage • Default alarms, defined at the top of the inventory • Add custom alarms anywhere in the inventory The highest point in the VirtualCenter inventory, Hosts and Clusters, is the location of the default alarms. You may modify these alarms in place. You may also define finer-grained alarms. For example, you might organize several hosts or clusters into a folder and apply an alarm to that folder. 448 VMware Infrastructure 3: Install and Configure Configure VirtualCenter Notifications • Choose Administration Server Configuration • Click Mail to set SMTP parameters VirtualCenter Management • Click SNMP to specify trap destinations If you wish to transmit SNMP or email alarms, you must supply the IP address of the destination server. If your SNMP community string is not public, specify it here. Specify the email address to be used for the From: address of email alerts. 10 Resource Monitoring Module 10 Resource Monitoring: Monitoring Using Performance-based Alarms 449 Lab for Lesson 3 • Host-Based and VMBased Performance Alarms • In this lab, you will perform the following tasks: •Create Host-based and VM-based alarms in VirtualCenter •Monitor CPU Usage alarms in VirtualCenter This lab will be performed by each ESX Server team separately VirtualCenter Server #3 ESX Server #3 ESX Server #4 Student 03a Student 03b Student 04a Student 04b ESX Server Team #3 ESX Server Team #4 450 VMware Infrastructure 3: Install and Configure Lesson Summary • The VI Client reports changes in host or VM state in its inventory panel • To be proactively notified of performance problems: • Configure alarms to watch for high resource consumption • Configure notifications so that personnel are notified appropriately • Alarm reporting options can be set to avoid needless realarms 10 Resource Monitoring Module 10 Resource Monitoring: Monitoring Using Performance-based Alarms 451 Module Summary • Understanding how CPU and memory resources are allocated to virtual machines is key to maximizing virtual machine performance • Be proactive and monitor your virtual machines’ performance graphs periodically • The VI Client inventory provides a convenient view of the state of your ESX Servers and virtual machines 452 VMware Infrastructure 3: Install and Configure Questions? Questions? 10 Resource Monitoring Module 10 Resource Monitoring: Monitoring Using Performance-based Alarms 453 454 VMware Infrastructure 3: Install and Configure M O D U L E 11 Data and Availability Protection 11 11 Data and Availability Protection Importance • Administrators have the very important tasks of protecting their systems against data loss and to make data continuously available to their end users Objectives for the Learner • Discuss general backup strategies for the Virtual Infrastructure • Implement a VMware HA Cluster Module Lessons • Backup Strategies • Virtual Machine High Availability VMware Infrastructure 3: Install and Configure 455 Lesson 1 Backup Strategies Lesson Topics • Backup strategies for virtual machines • Backup strategies for the ESX Server service console : This lesson discusses general strategies for backing up your virtual machines and service console. For details, consult the Virtual Machine Backup Guide, available on the VMware Web site. 456 VMware Infrastructure 3: Install and Configure What to Back Up 11 Data and Availability Protection • Within the ESX Server environment: • Virtual machine contents • Service console Within the ESX Server environment, you need to back up the following major items: • Virtual machine contents: The virtual machine data you back up can include virtual disks or Raw Device Mappings (RDMs), configuration files, and so on. As with physical machines, virtual machine data needs to be backed up periodically to prevent its corruption and loss due to human or technical errors. Generally, use the following backup schedule for your virtual machines: • At the image level, perform backups periodically for Windows and Linux. For example, back up a boot disk image of a Windows virtual machine once a week. • At the file level, perform backups once a day. For example, back up files on drives D, E, and so on every night. • Service Console: The service console, a customized version of Linux, is the ESX Server 3 command-line management interface. It provides tools and a command prompt for more direct management of ESX Server 3. With the VI Client being the main interface to the ESX Server 3 host, you should use the service console to perform only advanced administration operations During its lifetime, the service console doesn't experience any major changes other than periodic upgrades. In case of a failure, you can easily recover the state of your service console by reinstalling ESX Server 3. Therefore, although you might consider backing up the Module 11 Data and Availability Protection: Backup Strategies 457 ESX Server 3i does not have a service console. service console, it doesn't need to be backed up as frequently as the virtual machines and their date. 458 VMware Infrastructure 3: Install and Configure General Guideline for VM Backups • Store application data in separate virtual disks from system images • Use backup agents inside guest OSes for application data • If Windows, perform VCB file-level backups 11 Data and Availability Protection • Use full virtual machine backups for system images • Or plan to redeploy from template Storing data in separate physical disks not only makes backups more flexible, but it also allows for more flexible deployment from templates. You can configure the applications in your templates to keep their data on separate disk drives, and then provide a new blank disk for data whenever the VM is deployed. Backups from within the virtual machine, using a backup agent, are best for application data because no system shutdown is required. In contrast, virtual disk backups are best for system images, because they always result in a bootable virtual disk, suitable for rapid redeployment. Note that in addition to the 3rd party backup agents that VMware supports with ESX Server, there could be other 3rd party backup agents out there that are supported directly by the vendor. Module 11 Data and Availability Protection: Backup Strategies 459 Strategies for VM Backups • Perform a VM file-level backup using a backup client in the VM • Perform a full virtual machine backup from the service console • Perform a Windows VM file-level backup using VMware Consolidated Backup (VCB) • Perform a full virtual machine backup using VCB There are several backup strategies for backing up virtual machines. These strategies use traditional backup methods or VMware Consolidated Backup (VCB): • VM file-level backup using backup client: Because a virtual machine is just like a physical machine, you can back it up in the same manner as a physical machine, using backup software running inside a virtual machine. There are two methods to do this: • Method 1: Deploy your backup client in one virtual machine while the backup server is in another virtual machine. VMware recommends that you run both virtual machines on the same ESX Server system. In this case, data between the two virtual machines moves through the virtual Ethernet that connects these virtual machines, but does not have to be transferred over a physical Ethernet connection. • Method 2: Deploy the backup client in a virtual machine while the backup server runs on a physical machine. NOTE Instead of Method 2, consider using VCB. • Run a Backup Client in the service console: Because an entire virtual machine is encapsulated in only a few files, you can treat virtual machines as files on an ESX Server and back up these files from the service console. With this approach, you deploy your backup client in the service console and back up the files to backup servers, deployed in other virtual machines or inside physical machines. 460 VMware Infrastructure 3: Install and Configure When you run backup clients in the service console, do one of the following to perform a backup of your virtual machines: • Power off your virtual machines. • Use snapshots to backup running virtual machines. • VMware Consolidated Backup (VCB): VCB addresses most of the problems you encounter when performing traditional backups. Consolidated Backup helps you to: • Reduce the load on your ESX Servers by moving the backup tasks to one or more dedicated backup proxy servers. • Eliminate the need for a backup window by moving to a snapshotbased backup approach. • Simplify backup administration by making optional the deployment of backup agents in each virtual machine you back up. • Back up virtual machines that are powered on. VCB allows supports file-level backups for virtual machines running Microsoft Windows operating systems and full virtual machine (imagelevel) backups for virtual machines running any guest operating system. For more details on the traditional backup methods and VMware Consolidated Backup, consult the Virtual Machine Backup Guide, available on the VMware web site. For a complete list of backup clients and backup servers supported in a virtual machine, consult the ESX Server 3.x Backup Software Compatibility Guide, available on the VMware web site. 11 Data and Availability Protection Module 11 Data and Availability Protection: Backup Strategies 461 Strategies for Service Console Backups • Perform a VM file-level backup using a backup agent in the VM • Perform a Windows VM file-level backup using VMware Consolidated Backup (VCB) • Perform a full virtual machine backup using VCB Because the ESX Server 3 service console doesn't experience any major changes during its lifetime and its state is easily recoverable in case of a failure, you might decide against backing it up. If you choose to back up the service console, you don't need to do it frequently. Use the following methods when backing up the service console: • File-Based: Treat the service console as a physical machine with a deployed backup agent. To restore the service console, reinstall it, reinstall the agent, and then restore the files that you backed up. This approach makes sense if management agents that are hard to set up have been deployed in the service console. Otherwise, this approach provides no advantage over not backing up the service console. • Image-Based: Use third-party software to create a backup image that you can restore quickly. Use your boot CD or whatever the backup software created to restore the service console. For a complete list of backup clients supported in the service console, consult the ESX Server 3.x Backup Software Compatibility Guide, available on the VMware web site. 462 VMware Infrastructure 3: Install and Configure Lesson Summary 11 Data and Availability Protection • Performing backups using VCB takes the burden off the ESX Server and places it onto the backup proxy server • Service console backups do not need to be taken as frequently as virtual machine backups • VMware supports a number of different backup agents for the virtual machine and the service console Module 11 Data and Availability Protection: Backup Strategies 463 Lesson 2 Virtual Machine High Availability Lesson Topics • Strategies for clustering VMs using third-party software products • Clustering VMs using VMware HA (High Availability) : This lesson focuses on high availability, not fault tolerance (continuous availability). From the perspective of the user experience, a user will experience no disconnection (no disruption of service) in a fault tolerant system. In a highly available system, the user will be disconnected and will have to reconnect. A highly available system will not be 100% available but will be available in percentages approximating 100%, depending on the architecture. 464 VMware Infrastructure 3: Install and Configure Clustering Inside VMs for High Availability • Cluster-in-a-box • Protects against operator error, application and OS crashes 11 Data and Availability Protection • Cluster-across-boxes • Protects against operator error, application and OS crashes, hardware failures • Shared storage required • Cluster between physical and virtual machines • Low-cost N+1 redundancy • Shared storage required There are three main implementation schemes for clustering in ESX Server: • Cluster-in-a-box: this provides simple clustering to deal with software crashes or administrative errors. The cluster consists of multiple virtual machines on a single ESX Server. • Cluster-across-boxes: this allows you to deal with the crash of an ESX Server, since the virtual machines in the cluster are located across multiple ESX Servers. • Physical-to-virtual cluster (N+1 clustering): this provides a standby host for multiple physical machines on one standby box with multiple virtual machines. In other words, a physical machine is clustered with a virtual machine on an ESX Server (the standby host). For details on how to implement these schemes, consult the document, Setup for Microsoft Cluster Service, available on the VMware Web site at http://www.vmware.com/pdf/vi3_30_20_mscs.pdf. To help students with no cluster background, you might want to describe the hardware requirements for cluster briefly, making clear that there is no special hardware required, usually only additional NICs. Off-the-shelf hosts are used for clusters and are connected in every way possible: through the public net, through shared storage and through a "heartbeat" network or private network so the cluster software can manage shared resources and cluster nodes can communicate and coordinate with one another. Module 11 Data and Availability Protection: Virtual Machine High Availability 465 What is VMware HA? • Automatic restart of virtual machines in case of physical server failures • Provides high availability while reducing the need for passive stand-by hardware and dedicated administrators • A VirtualCenter feature • Configuration, management and monitoring done through the VI Client • Provides experimental support for VM failures VMware High Availability (HA) provides easy-to-use, cost effective high availability for applications running in virtual machines. In the event of server failure, affected virtual machines are automatically restarted on other production servers with spare capacity. VMware HA allows IT organizations to minimize downtime and IT service disruption while eliminating the need for dedicated stand-by hardware and installation of additional software. VMware HA continuously monitors all servers in a cluster and detects server failures. An agent placed on each server maintains a “heartbeat” with the other servers in the cluster. ESX Server heartbeats are sent every 5 seconds. If a heartbeat is lost, the agent initiates the restart process of all affected virtual machines on other servers. The heartbeat timeout is 15000 milliseconds or 15 seconds. VMware HA ensures that sufficient resources are available in the cluster at all times to be able to restart virtual machines on different physical servers in the event of server failure. Restart of virtual machines is made possible by the distributed locking mechanism in VMFS which gracefully coordinates read-write access to the same virtual machine files by multiple ESX Servers. VMware HA is easily configured for a cluster through VirtualCenter. Virtual Machine Failure Monitoring An additional VMware HA function named Virtual Machine Failure Monitoring allows VMware HA to monitor whether a virtual machine is available or not. VMware HA uses the heartbeat information that VMware Tools captures to determine virtual machine availability. 466 VMware Infrastructure 3: Install and Configure On each virtual machine, VMware Tools sends a heartbeat every second. Virtual Machine Failure Monitoring checks for a heartbeat every 20 seconds. If heartbeats have not been received within a specified (userconfigurable) time interval, Virtual Machine Failure Monitoring declares that virtual machine as failed and resets the virtual machine. Virtual Machine Failure Monitoring can distinguish between a virtual machine that was powered on but has stopped sending heartbeats and a virtual machine that is powered-off, suspended, or migrated. Virtual Machine Failure Monitoring is experimental and not supported for production use. By default, Virtual Machine Failure Monitoring is disabled. For more details on how to configure Virtual Machine Failure Monitoring, consult the Technical Note named Virtual Machine Failure Monitoring, available on the VMware web site. 11 Data and Availability Protection Module 11 Data and Availability Protection: Virtual Machine High Availability 467 VMware HA in Action If an ESX Server is a member of a VMware HA cluster, each of the VMs formerly running on it will get booted up again on some other surviving ESX Server in the cluster. Downtime depends in every cluster on how long it takes whatever is running to restart when the VM is failed over. The answer to how long it will take to restart the VM is "it depends". This slide builds: • We start with an HA cluster of 3 ESX Servers, VM A and VM B are on the first ESX Server, VM C and D are on the second ESX Server and VM E and F are on the third ESX Server. • Press Enter. • The first ESX Server fails. • Press Enter. • VMware HA will place VM A and VM B on the remaining servers in the cluster. 468 VMware Infrastructure 3: Install and Configure VMware HA Prerequisites 11 Data and Availability Protection • You should be able to power-on a VM from all hosts within the cluster • Access to common resources (shared storage, VM network) • Host should be configured for DNS • DNS resolution of all hosts within cluster is needed for initial configuration In order for the HA cluster to work properly, there are two prerequisites: Each host in the cluster should have access to the virtual machines' files and should be able to power on the VM with no problem. For that matter, all the VMotion requirements should be met in the cluster as well. Also, an important requirement is to make sure that each ESX Server in the cluster is configured to use DNS and DNS resolution of the host's fully qualifed domain name is successful because VMware HA relies on that name. Proper DNS & Network settings are needed for initial configuration. After configuration, DNS resolutions are cached to / etc/FT_HOSTS (minimizing the dependency on DNS server availability during an actual failover). DNS on each host is preferred (manual editing of /etc/ hosts is error prone). Module 11 Data and Availability Protection: Virtual Machine High Availability 469 VMware HA Host Network Configuration Networking should be set up to remove single points of failure and therefore, it is recommended to have two network paths for cluster server heartbeating. This can be accomplished using one of the following methods: • Define two service console ports, each one on a different virtual switch • Define a single service console port, and configure NIC teaming for the virtual switch on which this port is located. Service console network(s) are used for heartbeats and state synchronization. There is minimal network activity in a steady state (5 second heartbeat intervals). Additional light traffic occurs on this network during node configuration and VM power operations Incoming ports used: TCP/UDP 8042-8045 Outgoing ports used: TCP/UDP 2050-2250 NOTE The only way to eliminate single point of failure is to have an entirely redundant infrastructure including redundant switches. NIC teaming alone will not eliminate single point of failure if the NICs are connected to the same switch. 470 VMware Infrastructure 3: Install and Configure Create Cluster Configure cluster for VMware HA and/or DRS 11 Data and Availability Protection Creating a VMware HA cluster is very similar to creating a DRS cluster. The first step is to select the cluster type. It is best to create a cluster that has both VMware HA and DRS implemented, VMware HA for the reactive solution and DRS for the proactive solution. The job of DRS is to VMotion VMs to balance servers' CPU and memory loads. The job of VMware HA is to reboot VMs on a different ESX Server when an ESX Server crashes. No VMotion is involved in VMware HA. Why enable both VMware HA and DRS? The decision of initial placement of the VMs is done only for DRS clusters. The users can use DRS not just for initial placement, but for overall cluster balance. VMware HA is a reactive system, reacting to host failures. DRS is a proactive solution, and gives you better utilization for running VMs by balancing the cluster. Thus VMware HA+DRS is a reactive+proactive system, an ideal situation. Module 11 Data and Availability Protection: Virtual Machine High Availability 471 Configure Cluster Configure host failures and admission control settings How much redundant capacity will we have? Cluster-wide settings Which is more important: uptime or resource fairness? VMware HA cluster configuration is composed of two steps: Cluster-wide policies and individual VM customizations. There are two cluster-wide policy settings: number of host failures allowed and admission control. The number of host failures to tolerate can be from 1 to 4. For example, If 1 host fails in the cluster, there should be enough resources on the remaining hosts in the cluster on which to run the virtual machines that were on the failed host. Admission control policies for VMware HA define when or when not to power on a VM. By default, if a virtual machine violates availability constraints, then the virtual machine will not be powered on. Availability constraints refer to the cluster's resource reservations as well as the constraint specifying the number of host failures to tolerate. VMware HA tries to maintain enough spare capacity across the cluster based on these values. The actual spare capacity available can be monitored in the "current failover capacity" field in a VMware HA cluster's Summary tab (in the VI Client). 472 VMware Infrastructure 3: Install and Configure Failover Capacity Examples Failover capacity: 1 host failure VMware HA cluster 11 Data and Availability Protection Failover capacity: 2 host failure VMware HA cluster In the first example, the VMware HA cluster has been set up to allow 1 host to fail. Therefore, if any single ESX Server fails in the cluster, the remaining ESX Servers should have enough capacity to run the virtual machines that are on the failed server. This example assumes that all virtual machines require an equal amount of resources. In the second example, the VMware HA cluster has been set up to allow up to 2 hosts to fail. Therefore if two ESX Servers fail, the remaining ESX Server in the cluster should have enough capacity to run all virtual machines. Again, this example assumes that all virtual machines require the same amount of resources. Module 11 Data and Availability Protection: Virtual Machine High Availability 473 Add Host to Cluster To add a host to the cluster, there are two ways: First, you could right-click over the HA cluster and select Add Host from the menu. Or, you can drag and drop an existing standalone host into the HA cluster. 474 VMware Infrastructure 3: Install and Configure Which VMs Should Be Restarted First? 11 Data and Availability Protection If there is insufficient spare capacity during failover, VMs with higher priority get failed over first Restart priority is based on criticality of virtual machines and factor in dependencies. For example, in a Windows environment, DNS and Domain Controllers would normally be specified as the highest restoration priority, due to other servers depending on those infrastructure services. This priority decision may be influenced if you have redundant DNS and domain controller elements that are forced to be resident on different servers at all times, such as if an anti-affinity rule is applied at a DRS level. Note that this will not prevent someone from manually invoking migrations that cause these virtual machines to be on the same ESX Server. There are also some virtual machines that are not essential in the event of a failure, and may be disabled from being restored. This means that, if the HA cluster will have drastically reduced available resources, shedding these less essential resource consumers will reduce contention for these limited resources. Use Low/Medium/High restart priorities to customize failover ordering. The default is medium. High priority VMs are restarted first. Non-essential VMs should be set to “Disabled” (automated restart will skip them). Module 11 Data and Availability Protection: Virtual Machine High Availability 475 Architecture of a VMware HA Cluster VC Server In general, cluster software is composed of layers of code, “agents” that can start, stop and/or monitor cluster components. A host agent monitors hosts within the cluster, if a host fails the host agent can restart services (or in our case, VMs) on surviving nodes in the cluster. A key component to the VMware HA architecture is the cluster of hosts. In this example, the cluster consists of three hosts. When each host was added to the cluster, the VMware HA agent was uploaded to the host. The VMs' files are located on shared storage and therefore, each host in the cluster needs access to the same resources. You must be able to power on the VM on every host in the cluster. Distributed locking prevents simultaneous access to VMs, thus protecting data integrity. HA agents maintain a heartbeat network and therefore, their ability to perform failovers is independent from VirtualCenter availability. 476 VMware Infrastructure 3: Install and Configure During a failover, quick restart is the primary goal of VMware HA. DRS algorithms balance workloads after HA has recovered virtual machines. DRS uses VMotion to automatically rebalance the overall cluster load. Cluster nodes are designated as Primary or Secondary nodes. Primary nodes maintain a synchronized view of the entire cluster. There can be up to five primary nodes per cluster. Secondary nodes are managed by the primary nodes. 11 Data and Availability Protection Additional Information: VMware HA is based on EMC Autostart Manager (formerly known as Legato's AAM, Automated Availability Management) product for high availability. EMC Autostart Manager has the concept of primary and secondary nodes. All hosts in an EMC Autostart Manager cluster are either primary or secondary, and their roles are assigned dynamically. A primary node acts as a rule interpreter and maintains a distributed database. There are usually 2-5 primary nodes per cluster. Adding a new node requires that at least one primary node be up. A secondary node is a somewhat lighter-weight (less overhead) version of the primary node. Configuring the "Number of host failures allowed" in essence defines the number of primary nodes in the cluster. There is no parameter other than this one that allows you to configure the number of primary nodes in the cluster. More on primary nodes: Number of primary nodes = number of host failures to tolerate + 1. In a cluster, if a primary fails, another (secondary) node would be promoted to primary, to maintain the number of primaries in the system. In the worst case, if all the primaries were to fail, then you don't have enough information to recover from that situation. We have up to 5 primaries in our clusters. Having more than 5 would increase network traffic, and EMC Autostart Manager thinks that 3-5 is the sweet spot. So we allow up to 4 host failures - you can specify the number of host failures to tolerate between 1 and 4. If you want to have more host failures to tolerate, then you need to maintain more spare capacity across your cluster. It would be better at that point to just split the cluster up into smaller clusters. Module 11 Data and Availability Protection: Virtual Machine High Availability 477 What if a Host is Running but Isolated? • A network failure might cause a “splitbrain” condition • VMware HA waits 15 seconds before deciding that a host is isolated Network failures can cause "split-brain" conditions. In such cases, hosts are unable to determine if the rest of the cluster has failed or has become unreachable. A different isolation address can be specified using the the advanced HA option Isolation response is used to prevent split-brain conditions and is started when: • A host has stopped receiving heartbeats from other cluster nodes AND the isolation address cannot be pinged • The default isolation address is the service console gateway, and the default isolation response time is 15 seconds. Powering virtual machines off releases VMFS locks and enables other hosts to recover. When the "Leave power on" option is set, virtual machines may require manual power-off / migration in case of an actual network isolation. das.isolationaddre ss. A different isolation response time can also be specificed using the advanced HA option das.failuredetecti ontime. These are cluster-wide settings, which can be set in the Advanced Options menu of the VMware HA properties. More information on split-brain taken from the Internet: A split-brain condition occurs when a single cluster has a failure that results in reconfiguration of the cluster into multiple partitions; each partition forms its own sub-cluster without knowledge of the existence of the other. This leads to data collision and the corruption of shared data, because each sub-cluster assumes ownership of shared data. As an example, when two systems have access to the shared storage, the integrity of the data depends on the communication of heartbeats through the private interconnects. When the private links fail, or if one of the systems is hung or too busy to transmit heartbeats, each system thinks the other system has exited the cluster. Each system then tries to become master (or form a sub-cluster), and claim exclusive access to the shared storage. This condition leads to split-brain. 478 VMware Infrastructure 3: Install and Configure Choose Isolation Response per VM Power VM off to release lock on its disks 11 Data and Availability Protection Allow VM to continue to run while host isolated The user can also determine whether to power down the VMs or not, on node isolation. This is set using the Isolation Response. The isolation response of "Power off" does just that; VMware HA does not do a clean shutdown of the VM. Isolation Response is initiated when a host experiences network isolation from the rest of the cluster. “Power off” is the default response. “Leave power on” is intended for cases where: • Lack of redundancy and environmental factors make outages likely • VM networks are separate from service console (and more reliable) Isolation events can be prevented if proper network redundancy is employed from the start. Module 11 Data and Availability Protection: Virtual Machine High Availability 479 Troubleshooting VMware HA • Ensure IP connectivity, DNS resolution • Ensure that storage and networks are visible throughout the cluster • Service consoles have valid and reachable gateways • Re-initialize HA cluster configuration • Per host: Select ESX host -> Summary Tab -> Reconfigure for HA • Per cluster: Select Cluster -> Edit Settings -> Uncheck HA enabled, wait for reconfiguration task to complete, and then check to re-enable • Ensure that no one has managed hosts directly, bypassing VC • Check logs: /opt/LGTOaam512/log/* /opt/LGTOaam512/vmsupport/* If students are interested: Consider extending timeout values & adding multiple isolation addresses. Timeouts of 30-60 seconds will slightly extend recovery times, but will also allow for intermittent network outages. Modifying these options will be covered in the VI3: Deploy, Secure and Analyze course. In general, for any type of cluster (DRS, VMware HA, or combination of both), it is possible to damage the cluster by managing the ESX Server directly (bypassing VirtualCenter) and tweaking resource reservations. DRS and VMware HA are both VirtualCenter-level concepts, and they both believe that all changes to resource reservations are done at the VirtualCenter level. Changing resource reservations at the host level will cause the cluster to go into a red state and cease to do its job until any resource problems associated with that cluster have been fixed. For more information on VMware HA Best Practices, refer to the Knowledge Base article 1002080, Setting Failure and Isolation Detection Timeout and Multiple Isolation Response Addresses. (http:// kb.vmware.com/kb/1002080). 480 VMware Infrastructure 3: Install and Configure Lab for Lesson 2 • Using VMware HA • In this lab, you will perform the following tasks: •Add VMware HA functionality to an existing cluster •Cause VMware HA to restart virtual machines following the “crash” of a physical server 11 Data and Availability Protection Two ESX Server teams belong to one Cluster team VirtualCenter Server #3 VirtualCenter Server #4 ESX Server #3 ESX Server #4 Student 03a Student 03b Student 04a Student 04b DRS/HA Cluster Team Module 11 Data and Availability Protection: Virtual Machine High Availability 481 Module Summary • Use VCB to perform file-level backups of Windows virtual machines • Use VCB to perform virtual disk backups of all virtual machines • Check prerequisites for hosts and VMs in a VMware HA cluster • Plan your HA cluster • Failover level and admission control • Hosts and VMs’ resource availability and requirements 482 VMware Infrastructure 3: Install and Configure Questions? 11 Data and Availability Protection Questions? Module 11 Data and Availability Protection: Virtual Machine High Availability 483 484 VMware Infrastructure 3: Install and Configure MODULE 12 Planning VI Deployment 12 12 Planning VI Deployment Importance • Planning your VMware Infrastructure deployments properly from the very start can prevent problems that could occur when your VMware Infrastructure is put into production Objectives for the Learner • Size ESX Server and VirtualCenter deployments appropriately • Understand the process for booting the ESX Server from a SAN • Understand design principles for virtual disk storage Module Lessons • Plan VMware Infrastructure Deployment • Storage Considerations VMware Infrastructure 3: Install and Configure 485 Lesson 1 Plan VMware Infrastructure Deployment Lesson Topics • Using qualified hardware • Sizing VMkernel and service console resources • Booting ESX Server from a SAN • VirtualCenter resource sizing • VirtualCenter inventory guidelines : 486 VMware Infrastructure 3: Install and Configure ESX Server 3 Hardware Support Check the compatibility guides before deploying hardware! 12 Planning VI Deployment • ESX Server 3.x Systems Compatibility Guide • ESX Server 3.x I/O Compatibility Guide • ESX Server 3.x Storage/SAN Compatibility Guide http://www.vmware.com/support/pubs/vi_pubs.html Located on VMware's web site are many documents designed to provide you as much information as possible to ensure a successful deployment. Prior to installing the product, it is important to make sure that all of the equipment planned for supporting the Virtual Infrastructure is on the supported, tested compatibility documents. In addition to the ones listed above are many other sources of information including knowledge base articles, whitepapers, etc. It is important to consistently check the online documentation for supported hardware since VMware is constantly evaluating new hardware. Module 12 Planning VI Deployment: Plan VMware Infrastructure Deployment 487 ESX Server Sizing: Core Resources • Consider the peak load that virtual machines place on the “core” resources RAM Disk CPU Network When determining the hardware requirements of the ESX Server it is important in to consider the four core resources. Consider the peak load that is placed on these resources so they are not undersized. Don’t use a tool and look at just the maximum observed value. If you have ever watched a performance monitor while you start up a program you have seen the processor utilization jump to almost 100% during startup. Every machine will hit 100% utilization or come close to it at some point or another. The key is to understand sustained loads. 488 VMware Infrastructure 3: Install and Configure ESX Server Sizing: VM Load Profiles 12 Planning VI Deployment For example say that the mail servers that are monitored run about 7% Processor Utilization represented by the blue line above. These are average numbers across all the mail servers monitored. However, in the morning, they typically run 3 to 4 times higher than the average. The same is true after lunch and at closing time. If we were to reduce the capacity allocated to Exchange to be able to meet the needs of the average utilization, we would have a lot of very unhappy users in the morning, at lunch and at closing time. If peak load is not considered, we might have thought that combining the load of 5 of these mail servers into one ESX Server was reasonable. If Peak load is considered, we would never attempt that type of consolidation. Module 12 Planning VI Deployment: Plan VMware Infrastructure Deployment 489 ESX Server Sizing: VMkernel Resources Sum desired RAM maxima for all VMs, or minima if overcommitment is desired Sum desired disk sizes for all VMs; include space for all other VMs’ files, which includes the VM swap file and VM snapshot files, if used Sum needed CPU cycles for all VMs; allocate more if GigE is to be used Sum needed bandwidth for all VMs When planning for ESX Server resources, calculate the resources that each virtual machine will need in order to run. Each powered-on virtual machine has some memory overhead. The VI Client reports this overhead in the Summary tab of a virtual machine (the Resource Management Guide provides a table of memory overhead values based on the number of CPUs and memory size of the VM). There is also memory allocated to the VMkernel. It is at least 50MB, plus additional memory for device drivers. To determine how much memory the VMkernel is using on a running ESX Server, use the VI Client, select your ESX Server, click its Configuration tab, and click the Memory link. The System value represents how much memory the VMkernel uses. For disk space, figure out how much disk space is needed if this system were a physical machine. This value sizes the virtual disk. In addition, there are other files that make up a virtual machine. Most of these files are relatively small, such as the virtual machine's configuration files. However, you must also account for the size of the VMkernel swap file allocated to each virtual machine when it is powered on. The size of the VMkernel swap file is determined by the difference between the VM’s available memory and its memory reservation. If you take snapshots of your VMs, then account for the disk space used to hold the snapshot files. Snapshots are used for testing software, such as patches, for developing software, and for VM backups, if you are using VCB. A snapshot consists of a delta disk file which contains the changes made to the VM. This file could potentially grow to the size of the VM’s virtual disk. A snapshot also consists of a 490 VMware Infrastructure 3: Install and Configure memory state file, if you choose to snapshot the VM’s memory. The size of this file is the size of the VM’s available memory. To calculate the amount of network bandwidth for each VM, find out the average amount of bandwidth needed for each VM and sum the totals. 12 Likewise for CPU, find out the average amount of CPU cycles needed for each VM and sum the totals. Also note that the service console, VMkernel and Gigabit Ethernet adapters require some amount of CPU cycles, so conservatively speaking, add an extra CPU for system overhead and also future growth. Here are a few other sizing considerations: • If a VM's application is extremely sensitive to a resource, dedicate a resource to that VM: • Consider dedicating a disk LUN to a database application • Consider dedicating a CPU and a NIC to applications with low-latency requirements Planning VI Deployment Module 12 Planning VI Deployment: Plan VMware Infrastructure Deployment 491 ESX Server Sizing: Service Console Resources Up to 800 MB (272 MB by default) Service console filesystems One CPU One NIC The service console requires some amount of resources too. It needs 272 MB of memory, which is the default and also the recommended size. It needs disk space for its partitions, which we covered earlier. One NIC is sufficient for the service console, which connects it to the management network. Finally, the service console is a single-CPU operating system and always runs on the first hardware execution context. 492 VMware Infrastructure 3: Install and Configure Booting ESX from a Fibre Channel or iSCSI SAN LUN 12 Planning VI Deployment SAN • The ESX Server’s BIOS must designate the HBA as the boot controller • The HBA’s BIOS must be enabled to locate the target boot LUN ESX Server supports booting from a Fibre Channel SAN LUN or an iSCSI SAN LUN (using a hardware initiator only). Before you consider how to set up your system for boot from SAN, decide whether it makes sense for your environment. Use boot from SAN: • If you do not want to handle maintenance of local storage. • If you need easy cloning of service consoles (ESX Server 3 only). • In diskless hardware configurations, such as on some blade systems. Do not use boot from SAN if I/O contention might occur between the service console and VMkernel (ESX Server 3 only). For example, there could be I/O contention between the service console and virtual machines if they are all using the same disk array. If the decision to boot from SAN has been determined, there are a few extra necessary steps. After shutting down the ESX Server and before it completely boots up, the configuration on the HBA's BIOS must be enabled to boot and the ESX Server's BIOS must be configured to identify the Fibre Channel card as the first boot device. For details on configuring the ESX Server to boot from an iSCSI SAN LUN, consult the iSCSI SAN Configuration Guide, available on the VMware Web site. Module 12 Planning VI Deployment: Plan VMware Infrastructure Deployment 493 For details on configuring the ESX Server to boot from a Fibre Channel SAN LUN, consult the Fibre Channel SAN Configuration Guide, available on the VMware Web site. 494 VMware Infrastructure 3: Install and Configure Example: Booting ESX Server from a Fibre Channel SAN LUN 12 • Configure BIOS so that Fibre Channel adapter is the boot device, and desired LUN is the boot volume Planning VI Deployment • Disable built-in IDE controller if present The example above shows a sample BIOS configuration and Fibre Channel configuration typical for supporting boot from SAN. The BIOS configuration is from an HP Proliant server. Notice that the server's BIOS first boot device is the Fibre Channel controller. In the second screen, the QLogic adapter's BIOS is enabled and the first LUN is targeted as the boot LUN. This configuration identifies the boot LUN by the worldwide number (WWN) and the LUN number in hexadecimal format. The ESX Server can boot from SAN using any LUN that the server can access. For example, if the ESX Server were assigned LUNs 7, 8 and 9, the ESX Server could boot from LUN 7, 8 or 9. In some cases, the IDE controller on the ESX Server must be disabled. For example, if you are running an IBM eServer BladeCenter and use boot from SAN, you must disable IDE drives on the blades. Module 12 Planning VI Deployment: Plan VMware Infrastructure Deployment 495 Your VirtualCenter Deployment Support for up to 200 hosts and 2000 virtual machines ActiveDirectory Domain Managed Hosts VMware Infrastructure Clients Use SQL Server or Oracle in production As of this writing, for VirtualCenter 2.5, there are no new numbers for the recommended number of CPUs and amount of RAM. With VirtualCenter 2.0.x, increasing the hardware requirement to dual CPUs and 3 GB RAM can scale the VirtualCenter Server to support up to 50 concurrent client connections, 100 managed hosts and 1500 virtual machines. Please monitor the VMTN forums for new information on this topic. VirtualCenter Database A single VirtualCenter Server with minimum hardware requirements is recommended for supporting up to 20 concurrent client connections, 50 managed hosts and 1000 virtual machines. VirtualCenter Server can support a maximum of 200 managed hosts and 2000 virtual machines. VMware recommends against using SQL Server 2005 Express as the VirtualCenter database except for demos and proof-of-concepts. VMware recommends either a SQL Server or an Oracle database for your production environments. In planning for the VirtualCenter database size consider the number of ESX Servers and virtual machines the ESX Server will manage. Also consider the statistics collection level setting in VirtualCenter. The higher the setting the more data that will need to be stored in the database. For example a VirtualCenter installation managing 100 hosts and 1500 virtual machines could range between 5 gigabytes for Statistics Collection Level 1 to 162 gigabytes of disk space needed for Statistics Collection Level 4. Consider using VirtualCenter’s built-in database sizing calculator for planning the database size needed for VirtualCenter. 496 VMware Infrastructure 3: Install and Configure VirtualCenter Inventory Guidelines 12 Planning VI Deployment The datacenter is your primary organizational structure. Managed objects such as hosts, virtual machines, networks and datastores, belong to a single datacenter. Tasks such as cloning virtual machines, deploying virtual machines from templates or migrating virtual machines can only be performed with objects in the same datacenter. Use the following guidelines for planning your VirtualCenter inventory hierarchies: • Group hosts in a datacenter that are under a single administrative control • Group hosts in a datacenter that meet VMotion requirements • Group hosts in a cluster to form a single pool of resources • Group VMs into folders, e.g. by business unit or function Module 12 Planning VI Deployment: Plan VMware Infrastructure Deployment 497 Lesson Summary • Always check the ESX 3 compatibility guides before selecting your ESX Server hardware • The datacenter is VirtualCenter’s primary organizational structure • The ESX Server can be installed to boot from a local LUN or a remote, iSCSI or Fibre Channel LUN 498 VMware Infrastructure 3: Install and Configure 12 Planning VI Deployment Lesson 2 Storage Considerations Lesson Topics • Storage comparisons • Storage considerations : Module 12 Planning VI Deployment: Storage Considerations 499 Storage Comparison-Fibre Channel, NAS, iSCSI Technology Fibre Channel iSCSI Protocols FC/SCSI IP/SCSI Transfers Block access of data/LUN Block access of data/LUN File (no direct LUN access) Interface FC HBA iSCSI HBA or NIC NIC NAS IP/NFS The table above compares the features of the storage technologies available to the ESX Server. 500 VMware Infrastructure 3: Install and Configure ESX Server Feature Comparison by Storage Type 12 Type Fibre Channel iSCSI NAS Local Storage Boot VM Yes Yes Yes Yes Boot ESX Server Yes Yes No Yes VMotion VMFS RDM VM Cluster Yes No No No VMware HA/ DRS Yes Yes Yes No Planning VI Deployment VCB Yes Yes Yes No Yes Yes No Yes Yes Yes No Yes Yes Yes No No The table above compares the ESX Server features supported by the different storage types. Module 12 Planning VI Deployment: Storage Considerations 501 Storage Considerations (1 of 2) Component VMFS Considerations One VMFS volume per LUN; Use more than one VMFS to maintain separate test and production environments Use RDMs with VMs for 1) physical-to-virtual clusters or cluster-across-boxes and 2) use of hardware snapshotting functions of the disk array Each boot LUN should be seen only by the ESX Server booting from that LUN LUNs holding the VM’s virtual disks must be visible from both source and destination ESX Servers Each server has access to same shared storage; RDM Boot-from-SAN VMotion VMware HA All LUNs use by clustered VMs must be seen by all ESX Servers In general, it is best to use a LUN for one purpose at a time, whether it be used for a VMFS datastore, a mapped SAN LUN, or a boot LUN for an ESX Server. When a LUN is used for shared storage, for example, when it is used for VMotion migrations or for VMware HA, ensure that both source and destination ESX Servers have visibility to the same LUN. 502 VMware Infrastructure 3: Install and Configure Storage Considerations (2 of 2) 12 Component iSCSI Considerations For best performance and security, put iSCSI on a separate and isolated IP network For best performance and security, put NAS on a separate and isolated IP network; Planning VI Deployment NAS/NFS ESX Server needs full access to NFS datastores to create directories, set permissions (Use no_root_squash) 8 NFS mounts per ESX Server allowed, by default; Avoid VM swapping to NFS volumes If accessing both iSCSI and NAS storage from an ESX Server, put each storage device type on a separate, isolated network for best performance and security. NFS considerations: • Use no_root_squash: By default, the root user (whose UID is 0) is given the least amount of access to an NFS volume. This option turns off this behavior because the VMkernel needs to access the NFS volume using UID 0. • 8 NFS mounts per ESX Server allowed, by default. This number can be increased to 32. To increase this number, select host from inventory, click its Configuration tab, then select the Advanced Settings link. Click NFS in the left pane, then adjust "NFS.MaxVolumes" to the appropriate value. A reboot of the ESX Server is required in order for this change to take effect. • Avoid VM swapping to NFS volumes: This is for performance reasons. Therefore, have the VM swap to a VMFS volume instead. To do this, edit the VM's configuration file and add the following line: sched.swap.dir = "/vmfs/volumes/volume_name/ directory_name" Module 12 Planning VI Deployment: Storage Considerations 503 General SAN Considerations • Each LUN should have the right RAID level and storage characteristics for applications in VMs that will use it • Spread I/O loads over available paths to storage On Active/Active arrays use preferred paths to set up your ESX Server so that various LUNs are accessed over various paths: for example, one path should use one Fibre Channel adapter, and the other path should use the other. It is a common practice to create RAID volumes with seven disks or less. In RAID volumes consisting of more than seven disks, the overhead of parity calculation can overwhelm any performance benefit. Remember that physical resources are finite: both bandwidth to the disk array and I/O capacity to each LUN. 504 VMware Infrastructure 3: Install and Configure Two Schemes for Locating Virtual Disks 12 Planning VI Deployment One approach to storage management involves building LUNs with a variety of storage characteristics and then placing VMFS volumes in each, labeled to reflect those characteristics: "RAID5", "RAID0", etc. Now place virtual disks for each application into VMFS volumes appropriate for that application. If keeping the number of LUNs low (and thus easy to manage) is more important than optimizing each VM's I/O performance, simply create large LUNs and use them broadly; but carefully watch for virtual machines whose performance is unacceptable. Don't forget that system images (C: drives, for example) often have different I/O characteristics from application data. This is another reason why it is wise to build separate virtual disks for system and data. Module 12 Planning VI Deployment: Storage Considerations 505 Lesson Summary • In general, it is best to use a LUN for one purpose at a time, whether it be used for a VMFS datastore, a mapped SAN LUN, or a boot LUN for an ESX Server • If accessing both iSCSI and NAS storage from an ESX Server, put each storage device type on a separate, isolated network for best performance and security • Use preferred paths to set up your ESX Server so that various LUNs are accessed over various paths 506 VMware Infrastructure 3: Install and Configure Module Summary • When planning for ESX Server resources, calculate the resources that each virtual machine will need in order to run • In planning for the Virtual Center database size consider the number of ESX hosts and virtual machines the ESX host will manage • ESX Server supports Fibre Channel, iSCSI, NAS and local storage 12 Planning VI Deployment Module 12 Planning VI Deployment: Storage Considerations 507 Questions? Questions? 508 VMware Infrastructure 3: Install and Configure