Sunday, 4 April 2010

Looking for IP Addresses in Files

I've moved a couple of data centres. And I've virtualized a lot of servers. In all cases, the subnets in which the servers were installed changed. If anything depends on hard-coded IP addresses, it's going to break when the server moves.

The next data centre I move, I'm going to search all the servers for files that contain hard-coded IP addresses. The simplest thing to do for Linux and Unix is this:
egrep -R "\b([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}\b" root_of_code
The regular expression matches one to three digits followed by a "." exactly three times, then matches one to three digits, with word boundaries at either end.

That's not the most exact match of an IP address, because valid IP addresses won't have anything higher than 255 in each component. This is more correct:
egrep -R "\b((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b" /!(tmp|proc|dev|lib|sys) >/tmp/ips.out
It yields about two percent fewer lines when scanning a Linux server (no GUI installed). (Thanks to this awesome site for the regular expression.)

When I run the above egrep command from "/", I have problems. There are a few directories I had to exclude: /tmp, /proc, /dev, /lib and /sys. I used this file pattern match to get all the files in root except those directories:
/!(tmp|proc|dev|lib|sys)
The reason I wanted to exclude /tmp is that I wanted to put the output somewhere. /tmp is a good place, and by excluding it I didn't end up writing to the file while reading it. /sys on a Linux server has recursive directories in it. /proc and /dev have special files in them that cause egrep to simply wait forever. /lib also caused egrep to stop, but I'm not sure why (apparently certain combinations of regular expressions and files cause egrep to take a very long time -- perhaps that's what happened in /lib.)

I'll write about how to do this for Windows in another post. I'll also write about how to do it across a large number of server.

Friday, 2 April 2010

The Cost of Storage

Over the years I've seen SAN storage cost between C$10 and C$20 per GB (C$ is approximately equal to US$ right now). This is the cost of the frame, a mix of disks, redundant director-class fibre channel switches with a number of 48 port cards in each switch, management computer, and a variety of management and replication software. The cost doesn't include the HBAs in the servers, or cabling.

The price above is for a very raw GB, before you apply the loss for whatever classes of RAID you apply.

The management and replication software in all the above cases was the basic stuff you need to manage a SAN and replicate it. There was no fancy de-duplication or information lifecycle management going on.

The costs above also didn't include the cost of training dedicated storage staff to set up and manage a SAN, or the higher salary you'll have to pay to keep them after you train them.

Compare that to direct attached storage: Right now I can get a 1TB drive for less than C$300, or about 30 cents per GB. If I put that in a RAID 1 configuration with a RAID controller (less than $400 easily) you would still be paying less than $1 per GB.

I get RAID 1 storage for an order of magnitude cheaper than raw storage on a SAN. No need for special management software. You've got rsync for replication. You can use standard tools that everyone knows how to use.

No wonder Google uses direct-attached storage in commodity servers for their index. It's just way more cost-effective. What's your business case for SAN-attached storage?

Sunday, 31 January 2010

Ubuntu Netbook Remix Desktop Disappears

I have a netbook running Ubuntu Netbook Remix (UNR) 9.04. I switched it to the regular Ubuntu desktop just to try. Before I switched back, I rebooted (the battery ran all the way down). Apparently, this is known to be a bad thing. When you restart, all you get is a blank desktop -- no panels at the top and bottom to allow you to get at any commands.

The fix is described in Launchpad here, but I'm going to summarize it because it's a little spread out in the comments to the bug.
  1. Right click on the desktop and select "Create Folder..."
  2. Double click on the folder you just created
  3. Navigate to "/usr/bin/desktop-switcher" and run it
  4. Switch back to the UNR desktop
  5. Now navigate to your home directory
  6. Show hidden files (View-> Show Hidden Files, or Ctrl-h)
  7. Delete the .gconf, .gconfd, and .config folders
  8. Log out and log back in
This should fix the problem. Now, with respect to the classical desktop, don't do that :-)

Monday, 25 January 2010

Ubuntu Support Saturday in Vancouver

The ever-fantastic Ubuntu Vancouver Local Committee is organizing a Support Saturday. Come on down and learn about the world's most popular free operating system. If you already use Ubuntu, get some help to make your experience even better.

The details:

Saturday January 30th, 2010 11am - 2pm
Vancouver Community College
1155 East Broadway (Broadway Campus)
Building B, Room G219

Click here for the poster.

Here's the best door to use:


View Larger Map

Friday, 18 December 2009

Healthcare IT Doesn't Reduce Costs

A study in the Harvard Medical Journal shows that healthcare IT doesn't reduce costs, and perhaps provides a very marginal increase in quality of care. The authors speculate that one of the reasons may be that the cost of obtaining and running the system outweighs the benefit. No kidding. That fits what I've seen completely.

It's a short article, and worth reading. Thanks to Andrew for pointing me to it.

Saturday, 28 November 2009

Ubuntu Local Committee Install and Tweak

The ever-fantastic Ubuntu Vancouver Local Committee is organizing a Support Saturday. Come on down and learn about the world's most popular free operating system. If you already use Ubuntu, get some help to make your experience even better.

The details:

Saturday Dec 5th, 2009 11am - 2pm
Vancouver Community College
1155 East Broadway (Broadway Campus)
Building B, Room 219G

The poster is at: http://is.gd/560zM (a pdf).

Sunday, 20 September 2009

Ubuntu Breaks Thinkpad Yet Again

Sometime in the last couple of weeks the brightness controls on my Thinkpad x300 stopped working.

I fixed it by creating /etc/modprobe.d/thinkpad_acpi.conf and putting this line in it:
options thinkpad_acpi hotkey=enable,0xfdffff
Then you have to remove and reinstall the module to read the new options:
sudo rmmod thinkpad_acpi
sudo modprobe thinkpad_acpi
I'm not sure I'm getting all the brightness I should, but at least the control works to put the brightness up to make the screen readable.

Whine: This isn't the first time some part of the interface between Ubuntu and my laptop has broken due to an update. I can sympathize with the developers and the difficulty of testing all laptops out there, but I can sympathize more with the user who isn't going to put up with an O/S that mysteriously starts and stops working as updates are installed.