• Installation. Customized package selection, everything deselected except base utilities.
• Log in for the first time
• Create swap file because I like the flexibility (512 Mb):

 # dd if=/dev/zero of=/swapfile1 bs=1024 count=$[512*1024]
 # mkswap /swapfile1
 # swapon /swapfile1
 # chmod 640 /swapfile1

• Add following line to /etc/fstab:

 /swapfile1 swap swap defaults 0 0

• Install kernel-xen, install group Virtualization.
• Install sudo, xauth.
• Turn off selinux in /etc/selinux/config
• Set a static IP dom dom0, since the xen kernel doesn't like to bring up eth0 through dhcp
• Create image for domU's through virt-manager; this hangs after answering two questions; copied tree locally and set up local httpd. Doesn't work and neither does ftp work. New approach is kickstart:

 system-config-kickstart  noarch     2.6.19.1-1.el5   base              985 k
 Installing for dependencies:
 comps-extras            noarch     11.1-1.1.el5.centos  base               46 k
 notify-python           x86_64     0.1.0-3.fc6      base               13 k
 pirut                   noarch     1.2.10-1.el5.centos.1  base              246 k
 pykickstart             noarch     0.43-1.el5       base              127 k
 system-config-language  noarch     1.1.18-1.el5     base               68 k

• Created minimal config, succeeded in creating image through ftp
• Run yum update, and add exclude=kernel* to /etc/yum.conf
• On domU www2, networking doesn't work. Form of interfaces on dom0: vif<domid>.<vifid> where the domid can be checked with xm list. Reconfigured network to static IP, restarted domU, works.
• Test booting with either disk decoupled. Works after installing GRUB on 2nd disk. Have to press key to continue, though. This is a bit dumb for a PowerEdge server.
• Test performance on
  

1. Plain partition
   
2. Raid 1 partition
   
3. Raid 1 and LVM partition

Results plain 4G partition, formatted as ext3:

 $ bonnie++ -d /mnt/tmp/blah/ -s 2048
 Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 dv1              2G 45833  95 72654  29 31361   3 44004  77 75138   1 187.4   0

Results RAID-1 4G partition, formatted as ext3:

 $ bonnie++ -d /mnt/tmp/blah/ -s 2048
 Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 dv1              2G 43924  93 66645  29 31763   3 49976  93 71902   1 329.0   0

Results RAID-1 4G partition, LVM on top of it (one PV, one VG, one LV), formatted as ext3:

 $ bonnie++ -d /mnt/tmp/blah/ -s 2048

 Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 dv1              2G 45236  95 62809  29 34264   4 48406  91 71589   2 347.9   0

As previously, but now running in Xen, where root is an ext3 image in a logical volume:

• Made complete disk LVM'ed.
• Tried to move disk image (from file) to an LVM partition -- straight to the device with cp. This works but following up with resize2fs doesn't work. Straight copy of an LVM device works.
• After rebooting, RAID /dev/md2 gone. Had to assemble it:

 # mdadm -A /dev/md2 /dev/sda3 /dev/sdb3

• This takes ages. Added an extra configuration line to /etc/mdadm.conf, hoping that this helps.
• Testing performance outside and inside a Xen VM:

Outside:

 Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 dv1              1G 45744  95 73869  30 30175   3 45599  84 71956   2 309.4   0

Inside:

 Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 localhost.locald 1G 45288  94 66781  25 25768   2 41047  39 58338   1 397.4   0

Inside, now in a volume group consisting of two partitions:

 Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
 Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
 parttest         1G 44894  93 71689  27 26850   2 43208  41 63040   1 353.1   0
 

• What's actually pretty funny is that RedHat virt-manager does not allow you to install new virtual machines in different partitions. Well, they do allow when the installation is running, but these partitions are then installed inside the one block device that you pointed to when starting installation from VirtManager. Problem now: no good solution.
• Solution is to install into an image. Mount image to directory image. Format and mount logical volumes to directories root and var or whatever, and use cp -ax to copy appropriate directories from image to mounted logical volumes. Then boot with kernel /boot/vmlinuz-2.6.18-8.el5xen, which was copied from image to dom0.
• This isn't so good either. Lots of things (not the kernel, but system utilities) assume that the root resides on a disk which is partitioned. So what virt-manager does, is actually OK.

Todo:

• Boot with laptop and serial cable
• Configure and test eth1
• Install OpenIPMI tools
• Make Nagios plugin, including mdadm, xen and smart plugins
• Xen working again
• Check out logwatch
• daily security update
• Set up tripwire
• How important is the UUID and the MAC address settings in Xen config files?
• Installing SVN on www1
• Finish software
• After reboot, VMs should come up as well.
• How does VNC function with the virt-manager utility?
• Lock down sshd
• Remove server keys from image, set IP address, change MAC address, shut down and remove superfluous services
• Set hostname and static IP on www2.
• Install s/w on www2.
• Lock down dom0
• Copy www2 to ns1.
• See how kernel can be adjusted and recompiled
• See whether the Dell management software works, see http://lists.us.dell.com/faq
• Can we enable access to SMS?
• Check the TLS stuff
• See how CentOS works security-wise

Mounting

If you do a straight install of CentOS to an image with virt-manager, you'll get partitions in there. To mount the first partition:

 # lomount -t ext3 -partition 1 -diskimage /var/lib/xen/images/testimage /mnt/image

See also Fedora Wiki

Overview

• lv0: fresh installation with one partition inside it
• lv1: for www1