SNMP on the VM

Setting up SNMP on a machine so it can be monitored by Zenoss seems to mess me up every time. This time the problem was the -i option of snmpconf. It's advertised to put the configuration file where the SNMP programs will find it, but it doesn't put it at the front of the list of paths where the programs look, at least not on Ubuntu 6.06.

The solution: don't use snmpconf -i. Run snmpconf to set the access. Make sure it matches what you've set up in Zenoss, particularly the version of SNMP and therefore the access model. When you're done, do sudo mv snmpd.conf /etc/snmp/.

SNMP

The basic VM needs to have SNMP running on it, because there's no point having a server if you're not monitoring it. I had Zenoss set up a year ago monitoring some of my computers, but I was getting "bad oid" messages on the new VM template I was setting up.

The solution: Zenoss had a default SNMP version of 1 for Linux systems. I had set up SNMP on the new VM for version 2c. In Zenoss 2.0 I navigated to /Devices/Server/Linux page and selected the zProperties tab, then scrolled down to zSnmpVer and set it to v2c.

Basic Tiny VM Part 1

The basic tiny VM needs:

• Ubuntu 6.06.1 Server (the basic install, not LAMP)
• VMTools
• SNMP so you can monitor it (I'm using Zenoss)
  
• ssh so you can administer it
  
• ntp as a client so it keeps time. For now I'll sync to my existing ntp server
  
• basic firewall rules that allow the above

Build an ISO library in /usr/local/vmware/ISOs. Put in the Ubuntu CD and type:

mount /dev/cdrom
sudo dd if=/dev/cdrom0 of=/usr/local/vmware/ISOs/Ubuntu-6.06.1.iso

The VMTools ISOs are in the /tmp/vmware-server-distrib/lib/isoimages:

sudo cp /tmp/vmware-server-distrib/lib/isoimages/*.iso /usr/local/vmware/ISOs

Install VMTools. Here are some good instructions.

sudo apt-get install ssh ntp-simple snmpd snmp

(snmp is the package that contains snmpconf, which you need to set up snmp, and snmpwalk, which is useful for debugging.)

Configure the ntp server. I've set up an ntp server in the DNS, so I set the "server" line in /etc/ntp.conf to the following:

server ntp

And then restart ntp:

/etc/init.d/ntp-server restart

Run snmpconf to set up SNMP. That's probably a whole post in itself.

I'll do the firewall later. I've ignored my family for too long tonight.

Can't Connect to Console of VMs

I had everything built and running VMWare Server. Good. So I copied all the VMs I'd built when I was running VMWare on my desktop over to the new server. I started a few, and they were running fine. I could connect to the Zenoss console on one of them, and could ping both. However, all I got was a black screen when I tried to look at the console of the VM using VMWare Console.

The VMWare documentation recommended using the version of VMWare Console program specific to the server you're running. I grumbled a bit and re-installed (which was actually quite easy), then tried viewing the console of my VMs again. I still got a black screen, but I also got an error message saying that the .vmx file had to have execute permission for the user running the VMWare Console. I checked the .vmx files and sure enough, because of the way I copied them everything had 0644 permissions.

So I cd'd to the directory where all the VM directories were and typed:

find . -name "*.vmx" -exec chmod u+x \{} \;

That worked because the user connecting with VMWare Console is the same one that owned all the files. You'll have to do something slightly different if that's not the case.

Now they work fine.

VMWare Server on Ubuntu 6.06.1

The install went smoothly. I created a user "vmware" and added it to the admin group. Then I had to:

sudo apt-get install xinetd
sudo apt-get install libx11-6 libx11-dev libxtst6 xlibs-dev

The last line was thanks to this post. Without it, it wouldn't validate my serial number (and I'm sure I would have run into other problems).

The only default I changed was to put my virtual machines in /usr/local/vmware/Virtual Machines, because /usr/local is the big partition I made for VMs.

Virtualization So Far

As should be obvious from my recent posts, I've been trying to set up a host for virtual machines. I need to be able to try things out easily, and virtual machines are great for that. I'd also like to get rid of my old boxes that are running core network infrastructure. It's not so much that I want to get rid of them, but the risk of continuing to use them is a problem. I have an 11-year-old Macintosh Performa that's my DHCP and DNS server for my whole network. If it breaks, I'm scrambling to replace it unless I get something new built. Obviously if it runs on a computer with a 1 GB hard drive and 32 MB of memory, I should be able to run it on a VM.

Anyway, being cheap I wasn't sure I wanted to pay for VMWare. They have a free version of course, but XenSource's free version is a license-key upgrade, whereas VMWare Server to Virtual Infrastructure (AKA ESX) is a complete software upgrade. So I thought I'd try XenSource, especially since they seemed to be saying that they could run any OS if you bought a CPU with virtualization support.

So I carefully researched the chips I was looking for and bought a Dell SC440 with an Intel Xeon 3050. A low-price server but with the right parts, or so I thought.

The install of XenSource was easy, as was the install of XenCenter, the control program on Windows. Unfortunately, there was a problem with the shortcut to install XenCenter. I posted a question in the Xen community boards and got no help. I found the solution myself a few days later, but not after noticing that there was very, very little activity on the community boards. I wonder if anyone is using Xen, or at least is anyone using it without paying Citrix for support?

Also, it turns out you can't run anything you want as a VM. I tried to run Ubuntu Server 6.06.1 and it gets disk errors. This is a known problem, apparently. Okay, I know it's hard to support every Linux distro, but Ubuntu should be one you support. Look at the numbers.

Anyway, worse than not supporting Ubuntu is that the answer from Citrix seemed to be, "use one of our supported distros." They'll always be niche if that's their approach. The market for virtualization is the world of heterogenous data centres that need to shrink their power and A/C footprint. You're not going to get into that market unless you can run anything that an off-the-shelf PC can run. So, I decided to try VMWare.

Installing a 60-day evaluation copy of ESX 3i didn't work. Neither did installing an evaluation copy of ESX 3.5, but at least it told me that the network card wasn't supported. So I tried Ubuntu 6.06.1, and the network card wasn't supported there, either. Broadcom, what are you doing releasing a NIC that doesn't work with older drivers? I found how to get Ubuntu installed, and so I'll continue with installing the free version of VMWare Server. This is not what I wanted to be doing.

I guess the lesson is you really have to check the hardware compatibility list, but I didn't even know I was going to go this path. I'm interested in how many other problems I'm going to have.

Even though I'm not up with VMWare Server, I have to say that it's the preferable approach. At least you have an underlying OS you can work with, and my experience with VMWare elsewhere says it's going to run whatever I try to put on it. Too bad the thinner versions (ESX) don't work on my hardware.

Ubuntu 6.06.1 on Dell SC440

The Dell SC440 has a Broadcom BCM5754 NIC, which isn't supported by the Ubuntu 6.06.1 server CD. You have to build the server without a network interface, then copy the new driver source onto it using a USB drive and build and install the driver.

I'm building Ubuntu on this server to run VMWare Server, so I also was particular about the disk partitioning. I created a 4 GB partition for root, then let it partition the rest itself. It made partition 5 the swap with 6.1 GB (I have 2 GB of RAM), and the rest of the disk on partition 2 (143.7 GB), which I put on /usr/local. I changed the usage of the file system to "largefile4" to give one inode for every 4 MB. I don't really know what that's going to do to performance, but it seems to make sense given that I'm going to be creating VMs there.

Next I followed the instructions here to build and install the driver. The instructions worked perfectly (with the usual 50 percent "forget to sudo" rate).

The I edited /etc/dhcp3/dhclient.conf to send the hostname:

send host-name "vmhost01";

And restarted the network:

ifdown eth0; ifup eth0

Then next time my DNS refreshed it got the server name.

Mount a USB Drive on Ubuntu 6.06.1

If this is the first time, make yourself a mount point for USB devices, like:

sudo mkdir /mnt/usb

Thereafter:

sudo mount /dev/sdb1 /mnt/usb

If mount can't figure out the filesystem type, you'll have to figure it out yourself and specify the type to mount with the -t option.

Installing the Xen Guest Agent on an Ubuntu VM

To install the guest agent on an Ubuntu VM, I tried:

1. Start the Ubuntu VM if it isn't already running
2. In XenCenter, select the VM in the left panel, and select the Console tab in the right panel
3. Select "xs-tools.iso" from the drop down list just above the console window
4. Click on the console window, log in if necessary, and type sudo mount /dev/hdd /mnt

So in other words, where the Xen documentation says /dev/xvdd, use /dev/hdd on Ubuntu.

If you just type sudo /mnt/Linux/Install.sh, it tells you you're running an unsupported distribution. I thought I'd be clever and try the force it to use the Debian 3 tools:

1. Type: sudo /mnt/Linux/install.sh -d debian -m 3
2. Reboot: sudo shutdown -r now
   

That left me with an unbootable kernel. I booted from the original Ubuntu kernel, then edited /boot/grub/menu.lst to change the default kernel to 2 so it would boot the good kernel.

Installing Ubuntu on Xen

I was able to install Ubuntu 6.06.1 on Xen by putting the Ubuntu install CD in the CD drive of the Xen server. I got lost for a while because I didn't create the storage when initially setting up the VM. Once I actually made a disk, the install went okay. I've found a post saying that Ubuntu will start having errors and switch the disk to read-only, so I'll watch for that.

I still need to install the Xen tools...

Ubuntu on Xen

I wanted to continue to use Ubuntu Server in my VMs. I like their approach to long-term support, the server install, and the way they provide a pre-configured LAMP server.

As I read about how to build one, I stumbled across a post from someone else who was doing what I wanted to do, and had problems. That scared me off just blindly stumbling ahead, so now I have more research to do.

I'm pretty sure it should be relatively easy to have a kernel that works on top of Xen, and on which I can run an otherwise standard Ubuntu 6.06 install, but it's going to take some digging... This is harder than I wanted it to be.

First Guest on Xen

I created a Debian 3.1 VM from the templates using XenCenter. I just followed the XenCenter wizard. It was trivial, although before I created it I spent almost 1 1/2 hours reading documentation and the Internet.

The only thing I had to do was set the time zone. Being old fashioned, I used the command line console and tzconfig(1).

If IT Isn't Broken...

If NT provides what you need, who cares if the platform is a bit worn and some of the paint doesn't look so good?

XenServer Install

I bought a Dell SC440 for Jade Systems (that's me). I'm setting it up as a virtual host so I can create virtual computers at will. I've decided to try XenServer, I think mostly because the free version is just a license-key upgrade to the full version. Besides, I already have some experience with VMWare, albeit second hand.

I had to F2 into the BIOS on initial boot of the SC440 to turn on virtualization in the CPU. It was relatively easy to find on the menus, but not right at the top level (My apologies for the vagueness here. The servers are in another room, so I wasn't blogging while I was installing).

The install of XenServer 4.0.1 went exactly according to the instructions. I read through the four pages of documentation first, and so I had all the answers I needed -- the usual network set-up, which time servers to use, etc.

I just finished installing the security patch. First challenge: actually getting the file to the XenServer host. I put a DVD in the DVD drive, but there's no entry for CD or DVD in /etc/fstab. The installation instructions recommend using a USB key. I flailed on that for a while, Googling for Linux-for-dummies level help on USB, before I said to myself, "Right. I guess I still have to be a Linux sysadmin," and found the appropriate dev to mount the dvd. From there it was maybe a minute to install the patch.

Total time: About half an hour to install XenServer the other night, mostly unattended after the usual initial Linux questions, plus 45 minutes now, mostly Googling.

Why Open Source is Better for Us All

I just read a very interesting way of looking at how open source creates a fundamental shift in the factors that drive software acquisition. Given that much of what's wrong in IT is about the relationship that customers and software developers are driven into by the economics of developing software, I really like the ideas in the post.