Table of Contents
A powerful rack server can be used as a host for installing lots of virtual machines, and it can be used as a data storage as well. This article will describe how to use such a server, installed with CentOS, as a host for virtual machines.
The server is a SuperMicro X9DRL-3F/IF with these parameters:
|HDD||2x320GB + 2x3TB|
|Network||2 Gb interfaces + 1 KVM/IMPI interface|
The first two disks (2x320GB) are used in RAID1 configuration to keep the host and the virtual servers, and the 2x3TB diska are used for the data.
1 Installation of CentOS
During boot-up, we press Ctrl+I and configure a RAID1 device with the first two disks (320Gb each).
Installation was done with CentOS-6.2-x86_64-minimal.iso (standard installation, where the installer automatically partitions the first disk drive (the raid one). The standard installation of CentOS is very easy (with a GUI interface), it use RAID automatically, partitions the disk automatically, and uses LVM for the partitions.
2 Disk partitioning and formating
We have two disks of 3TB each that we need to partition and manage, however the partition tables of type msdos (most commonly used) cannot manage more than 2TB of disk space. The solution is to use partion tables of type GPT. Fortunately, the partition editor (parted) of Linux, supports them quite well.
The GPT partitions can also have more than 4 primary partitions, so there is no need for extended partitions and tricks like this. I split both of the disks in 6 primary partitions of 500GB each, formating them with ext4. Later, I am going to manage these partitions with LVM.
The steps below show roughly how I did the partitioning:
yum install parted parted /dev/sdc parted /dev/sdd
(parted) print (parted) mklabel gpt (parted) mkpart primary 0.0GB 500.0GB (parted) mkpart primary 0.0GB 500.0GB (parted) mkpart primary 500.0GB 1.0TB (parted) mkpart primary 1.0TB 1.5TB (parted) mkpart primary 1.5TB 2.0TB (parted) mkpart primary 2.0TB 2.5TB (parted) mkpart primary 2.5TB 3.0TB (parted) print Model: ATA ST3000DM001-9YN1 (scsi) Disk /dev/sdd: 3001GB Sector size (logical/physical): 512B/4096B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 500GB 500GB primary 2 500GB 1000GB 500GB primary 3 1000GB 1500GB 500GB primary 4 1500GB 2000GB 500GB primary 5 2000GB 2500GB 500GB primary 6 2500GB 3001GB 501GB primary (parted)
And this is how I formated them with ext4:
mkfs.ext4 /dev/sdc1 mkfs.ext4 /dev/sdc2 mkfs.ext4 /dev/sdc3 mkfs.ext4 /dev/sdc4 mkfs.ext4 /dev/sdc5 mkfs.ext4 /dev/sdc6 for i in 1 2 3 4 5 6; do mkfs.ext4 /dev/sdd$i ; done
3 Managing partitions with LVM
One of the best advantages of Logical Volume Management (LVM) is the flexibility. LVM disks and partitions can be resized easily, when needed. Actually, in the terminology of LVM, logical disks are called Volume Groups (VG), and logical partitions are called Logical Volumes (LV). We can create several VGs, and inside each of them we can create LVs. The sizes of VGs and LVs are flexible, we can extend them later, if needed.
Let's create a volume group named vg_data by including some Physical Volumes (PV, physical disk partitions) to it:
vgdisplay vgcreate vg_data /dev/sdc1 /dev/sdc2 /dev/sdd1 /dev/sdd2 vgdisplay
Then we can extend it by adding some more PVs (partitions) to it:
vgdisplay vgextend vg_data /dev/sdc3 /dev/sdc4 /dev/sdd3 /dev/sdd4 vgdisplay
Now, inside the VG named vg_data, let's create an LV (logical partition) named /dev/vg_data/lv_mirror, of size 1TB:
lvdisplay lvcreate vg_data -L 1T -n /dev/vg_data/lv_mirror lvdisplay
We can create an ext4 filesystem on it like this:
mkfs.ext4 -L mirror /dev/vg_data/lv_mirror
Another LV can be created like this:
lvcreate vg_data -L 500G -n /dev/vg_data/lv_cache lvdisplay
4 Creating bridged interfaces on CentOS
We want the virtual machines to be connected directly to the network, and for this reason we should create bridged interfaces on the host system. We create a bridged interface for each of the network interfaces of the server. The steps below show how it can be done on CentOS.
DEVICE="eth0" HWADDR="00:25:90:76:91:9A" ONBOOT="yes" BRIDGE="br0"
DEVICE="eth1" HWADDR="00:25:90:76:91:9B" ONBOOT="yes" BRIDGE="br1"
DEVICE="br0" TYPE="Bridge" BOOTPROTO="static" ONBOOT="yes" IPADDR="192.168.10.254" NETMASK="255.255.255.0" DELAY="0"
DEVICE="br1" TYPE="Bridge" BOOTPROTO="static" ONBOOT="yes" IPADDR="192.168.1.254" NETMASK="255.255.255.0" DELAY="0" GATEWAY="192.168.1.1"
- Restart the network:
service network restart
5 Installing KVM and libvirt
- First check if the CPU supports hardware virtualization:
egrep '(vmx|svm)' --color=always /proc/cpuinfo
- Install kvm and libvirt:
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY* yum install kvm libvirt python-virtinst qemu-kvm
mdns_adv = 0. Then restart libvirtd and check it with virsh:
service libvirtd restart virsh -c qemu:///system list
- Add a user that can manage the virtual machines:
useradd virtadmin passwd virtadmin usermod -a -G kvm virtadmin
We would like to be able to manage the virtual machines remotely (for example with virt-manager), and it is not a good idea to use the root account for doing it. So we create another account, virtadmin, that has permissions to manage the virtual machines. These permissions are assigned to it simply by adding it to the group kvm.
/etc/selinux/configand then reboot:
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted
- For easy backup, we keep all the configurations and images on a
separate directory, called
/systems(which can also be on a separate partition). Move all configurations and images to
mkdir /systems mv /etc/libvirt /systems/etc ln -s /systems/etc /etc/libvirt mv /var/lib/libvirt/ /systems/var ln -s /systems/var/ /var/lib/libvirt mkdir /systems/images/
<!- - <path>/var/lib/libvirt/images</path> --> <path>/systems/images</path>