Ubuntu VMs and Time

Ubuntu 8.04 in VMs under VMWare Server 2.0.1 need the kernel parameter "clocksource=acpi_pm" when they boot. Edit /boot/grub/menu.lst and add "clocksource=acpi_pm" to the end of the line that starts "# kopt=". Don't remove the "#". In this context it's not a comment. After you save the file, run "sudo update-grub" and then reboot.

If the guest doesn't have this kernel parameter specified, time runs backwards on the guest O/S, or time hangs or gets stuck. This was showing up especially on my backup server running bacula, and the backup clients that had large amounts of data. I suspect that high loads exacerbate the problem.

xsane Problems

This morning I scanned a small French reader that Marc particularly liked. At noon I had to scan two pages for Angela, because she'd left them at home. When I went to scan I got this: "Failed to start scanner: Invalid argument".

After much flailing, including re-installing hplip and xsane, and doing sudo

ln -s /usr/lib/libhpmud.so.0 libhpmud.so

I finally got brave and simply moved ~/.sane/xsane to ~/.sane/xsane.old, then started xsane and it worked.

After the fact, I figured out that the problem started when I clicked on the preview pane in xsane and set the size of the area I was scanning to 0. The message wasn't very helpful. (I figured it out by diff'ing the xsane.conf that worked with the one in xsane.old.)

In addition, hp-check reports there's no plug-in installed for my printer, even though everything seems to be working. Go figure.

All this under Ubuntu 9.04 using an HP CM1312nfi MFP.

Getting Identity From Active Directory

I needed a test environment where I could test mounting and accessing Windows shares on a Linux machine, using identities and permissions obtained from Active Directory (AD). After the initial setup, I wanted to run a processes periodically in the background, without user intervention. Therefore, having the user enter the password each time wasn't an option. Also, the background process would be run periodically forever in the future. I didn't want to store passwords because the processes would fail after the user changed their password (and it's not a good idea to store passwords anyway).

The Kerberos authentication scheme in Windows and Linux uses tickets, which can be used to prove that a process is acting on behalf of a user. A user gets a ticket by requesting one and providing their password. Until that ticket expires, processes that support Kerberos can be run with the permissions of that user.

So let's say we want to access a Windows share as user "testa", which is a Windows user known to the AD server. The Linux machine asks for a ticket for testa, using testa's password. The AD server validates the password and gives the Linux machine a ticket. The Linux machine can then mount the Windows share using Kerberos authentication. Accesses to the files and directories on the share will then be allowed or denied based on testa's permissions.

I built an AD server on Windows 2003 Server SP2. The client machine was Ubuntu Desktop Edition 9.04.

Here's how I went about it:

1. Build an Active Directory server accepting the defaults. This included allowing it to set up its own DNS server. I already have DNS servers in my network, but I'm not a DNS expert. I've had bad luck changing my DNS setup in the past, so for this test I just let AD do its thing.
2. Install required packages on the Linux machine:
   

sudo apt-get install krb5-user keyutils


3. Replace the installed /etc/krb5.conf with the following. You have to replace "my.domain.tld" with your own domain, of course. Be careful to copy uppercase and lowercase:

[libdefaults]
default_realm = MY.DOMAIN.TLD
default_checksum = rsa-md5


[realms]
MY.DOMAIN.TLD = {
kdc = ADServer.my.domain.tld
}

[domain_realm]
.my.domain.tld = MY.DOMAIN.TLD
my.domain.tld = MY.DOMAIN.TLD


4. Add the following line to /etc/request-key.conf. The order of the lines is important. I put it last and nothing changed. I put it first and everything worked:

create cifs.spnego * * /usr/sbin/cifs.upcall %k %d


5. Get a key with kinit. Run kinit with sudo. The ticket you get is for the AD user testa whether you run as sudo or not, but the place that kinit stores the ticket depends on the Linux user who runs kinit. Since the mount command runs as root, you have to get a ticket for root or mount won't find the ticket
   

sudo kinit -f testa


6. Mount the share, replacing "FileServer", "Share", and "/tmp/mnt" with appropriate values for your systems:

sudo mount -t cifs -o sec=krb5i //FileServer/Share /tmp/mnt

For a while I was getting "mount error(2): No such file or directory" when I tried to mount. It was because I hadn't installed the keyutils package.

I've tested this up to and including the mount. I haven't finished testing the background process I originally wanted to build. I may modify this post based on my testing experience, so check back later.

Whither the CBC

The CBC like many media outlets has been in crisis recently. On top of repeated cuts in government funding over the last twenty years, they're suffering from the collapse of advertising revenue to traditional media, due to the television services' dependency on ad revenue.

(U.S. readers: The CBC is the government-funded public television and radio broadcaster in Canada, like the BBC in the UK or the ABC in Australia.)

To me, the role of the CBC is to provide channels for Canadian cultural expression. Market forces will inevitably regurgitate what is produced by the massively-productive American media machine, so we need someone to provide the infrastructure for cultural dissemination. Nothing in that says that the CBC has to be a radio or television network in the traditional sense.

In the post-network, post-newspaper world, I think there are a lot of exciting opportunities for public broadcasters to facilitate even greater promotion of Canadian (or British, or Australian, or South African or New Zealand or ...) culture. So here's my road map for CBC over the next decade. It consists of two themes, distribution and production:

It should be obvious that people don't think of network television or radio as their primary way of getting information or entertainment anymore. Music comes on a iPod. Video comes from YouTube. I want to listen to my local CBC radio morning show on my iPhone, because that's what I take with me when I walk my dog. And I want to view CBC TV on all those devices, too.

CBC should stream all its content, in local timezones, and in open source formats. This means CBC should support open source projects that develop and/or package codecs and whatever other software is needed to ensure that users of all platforms have access to CBC audio and video content: Mac, Linux, Windows, iPhone, Blackberry, Android and other phone platforms. A lot of this technology is available already, but much of it is only accessible to us hard core geeks. It needs to be really easy for anyone to use.

Even more exciting is the possibility for production: Enable the public to produce their own high-quality podcasts and video. This means, again, funding open source projects to develop and package audio and video production tools. Having done a bit of screencasting, I know it's not easy for the general public to do today, but we're not a long way from it being easier.

The CBC, or Canadian universities, could also produce a series of screencasts or videos to show people how to produce content: A sort of on-line journalism and documentary production school. And there's no reason to limit this to news and documentaries. The CBC, NFB and universities

This is technically feasible. I believe it's economically feasible within a budget that CBC has or could get. Maybe they can cut real costs (rather than cutting services) from the traditional parts of the broadcast system by moving programs around on the Internet, instead of via satellite or other dedicated pipes. Maybe they can increase their budget, by showing the government they're more relevant than they've been in years (not likely with Harper as PM, but he'll be gone someday).

The real challenge is the mindset change that would have to take place within the CBC bureaucracy. Therefore, this is a call to action to those who know people in the CBC or that can influence the CBC's direction. Start spreading this idea. The important thing is that we start discussing how we're going to create and distribute Canadian content in the Internet age, and then start acting on it.

Showing Lists of Events in Drupal 6

I wanted to have a page in Drupal that showed upcoming events for the Parent Advisory Council of my son's school, with a tab making it easy to show past events. The idea was to have one place to go to find out about upcoming events, or to find previous events so you could get materials from the event (e.g. handouts provided by speakers).

Drupal has a pretty good module for showing calendars, but it shows them on day, week or month views in a calendar. My users wanted to see a list of events. Our events are more spread out, so a list or table view is actually easier to work with.

So to summarize, we want:

• A menu in Drupal's primary menu for "Events"
• The menu takes the user to a page that shows upcoming (in the future) events
• There is a tab on the page for "All Events". Clicking the tab shows all events, paged with 20 events per page
• There is also a tab for "Upcoming Events". Clicking it takes you back to just the future events
• Events have to have a start and end time, and may have an additional description and attachments (e.g. agenda, minutes, hand-outs)
  

The high level overview of what I did is:

1. Install and enable some contributed modules
2. Create a "content type" for events. This defines what information you're going to store for each event
3. Define a "view" and two "displays" for the view. The view and its displays are partly a query of the information you want to appear, and partly the definition of the menus that get you to the information. The format of the information is defined by your theme, which I won't talk about in this post
4. Put the menu in the right place
5. Create some events and test it out. Testing is really important. It's almost impossible to get this right the first time

A small warning before you get started: The smallest error in anything can lead to very bizarre behaviour, or simply to some part of the page not showing up, with no apparent connection to the error. It took me days to get this working (with a lot of side-tracks into Drupal esoterica). Don't expect to get it right the first time, even with detailed instructions.

Here's what I did. I don't guarantee that I've covered every single thing, so please let me know by commenting on this story if I missed anything. (I assume you know how to do Drupal basics like install modules.)

1. Install the Content Creation Kit (CCK), Views, Date, and Advanced Help modules. Date module version 6.x.2.1 won't work for the use case we're trying to implement. The filter relative to a date (e.g. "now") is broken, which means you won't be able to show upcoming events. Use date-6.x-2.x-dev (see http://drupal.org/node/426990) until the fix gets rolled into the release version
   
2. Enable the "Content" part of CCK
3. Enable all parts of Date except "Date Locale" and "Date Tools", unless you want them for other reasons. Also, if you're running on PHP5 you don't need "Date PHP4"
4. Enable the "Views" and "Views UI" parts of Views

Now, create the content type for events:

1. Administration-> Content Management-> Content Types
2. Click on the "Add content type" link at the top of the "Content types" area
3. Enter the name "Event" and type "event". There are many options to set, and the help text is pretty self-explanatory. Under "Workflow settings", "Attachments", make sure they're enabled (since we said that was a goal of what we're doing). Most of the other options depend on your needs, although under "Workflow settings" it's likely you'll want to turn off "Promoted to front page" under "Default options"
4. Click the "Save content type" button at the bottom of the page
5. Click on "Manage fields" beside the Event content type
6. Under "Add", "Existing field", select "Datetime: field_event_datetime..." You may have to scroll down to see the existing fields. If you don't see an existing field called "field_event_datetime", then create one under "Add", "New field"
   
7. Click "Save"
8. You will be presented with another page allowing you to set a bunch of options. Set them according to your needs
9. Click "Save"

Now create the view. You really should read the Drupal Views documentation, which you have available because you installed the Advanced Help module. Read it now. Click on Administer-> Site building-> Views and you'll be given a link to the documentation near the top of the "Views" area

Now, here's my summary of what's important in the Views documentation:

• A view can have several displays. The display determines where the view is seen, and can also override some parts of the view's default settings. In the rest of this post, I use "view" and "display" in these specific meanings
  
• Set up the common aspects of the view first in the Default display. In our case, that means the output format (a table), the fields to be viewed, and some of the filters
• Be very careful when editing non-default displays to click "Override" before editing anything that you want to be unique to the display. If you don't explicitly click "Override", you'll change the default which will affect other displays that use the default value
  

So what we have to do is:

1. Create the view
2. Set up the default for the view the way we want it
3. Create a "Page" display for the "Upcoming Events" tab
4. Move the menu to the right place
5. Enter some test events
6. Check that the "Upcoming Events" tab works
   
7. Create another "Page" display for the "All Events" tab
8. Check that the "All Events" tab works
   

Here are the steps:

1. Administer-> Site building-> Views
2. Click the "Add" link at the top of the "Views" area
3. Put "event_list" in the "View name" field, and "Event List" in the "View tag"
4. Make sure the "View type" is node and click "Next"
5. This brings you to the Views UI page. I found this page to be extremely unintuitive. You have to be patient and always remember to scroll down after clicking on something, because many links simply change what's displayed below what's likely visible when you click on the link

Set the defaults for the view:

1. Under "Basic Settings", click "Unformatted" under "Style". Scroll down. You'll see some radio buttons. Click "Table" and click "Update". Ignore the error message that appears about no fields defined. We'll get to that later
   
2. Beside ""Use pager", click "No". Scroll down. Select the "Mini pager" radio button and click "Update"
3. Beside "More link", click "No". Scroll down. Select the "Create more link" check box and click "Update"
4. Under "Fields", click on the "+" beside "Fields"
5. Scroll down. Click on the check boxes for "Content: Event Dat and Time... From date". Don't check the box for the corresponding "To Date" field. Both are included in the "From date". Click on the check boxes for "Node: Body", "Node: Title", and "Upload: Attached files". Click "Add"
6. You'll now be presented with a number of pages to allow you to set options for each field. For "Event Date...", accept the defaults by clicking "Update"
7. For "Node: Body", accept the defaults by clicking "Update"
8. For "Node: Title", click the "Link this field to its node" checkbox and click "Update"
9. For "Upload: Attached file", click the "Link this field to download the file" checkbox and click "Update"
10. Now, put the fields in the order you want: Click the up and down arrows by "Fields". Scroll down. Drag "Title" up to the top and click "Update"
11. Set the sort order: By "Sort criteria", click the "+". Scroll down. Select "Content: Event date and time" and click "Add"
    
12. In the next page, select "Ascending", so the next event will appear at the top of the list. Click "Update"
13. Choose the nodes you want to display, specifically the published event nodes. Click the "+" beside "Filters". Scroll down. Select the checkbox for "Node: Published" and "Node: Type" and click "Add"
14. On the page for "Node: Published", click the "Yes" radio button and click "Update"
15. On the "Node: Type" page, click the "Event" checkbox under "Node type" and click "Update"
16. Click "Save". We have our default view set up.

Create the "Upcoming Events" page:

1. Click the "Add Display" button (the drop down box above it should read "Page" already)
2. Under "Basic settings", click on "Page" beside "Name". Scroll down. Type in "Upcoming Events" and click "Update"
3. Click on "None" beside Title. Scroll down. Click "Override". Enter "Upcoming Events" and click "Update"
4. Under "Page settings", click on "None" beside "Path". Scroll down. Enter "events/upcoming_events" and click "Update"
5. Click on "No menu" beside "Menu". Scroll down. Select the "Default menu tab" radio button. Enter "Upcoming Events" under "Title". Click "Update"
6. On the next page, click on the "Normal menu item" radio button, and enter "Events" under "Title". Click "Update"
7. Click on the work "Filters". Scroll down. Click on "Override" and then "Update". This overrides the filter selection but makes a copy of the two filters that we set as default
8. Click on the "+" beside "Filters". Scroll down. Click on the checkbox beside "Date: Date (node)". Make sure you get the right one. There are three fields that start with "Date:". Click "Add"
9. Scroll down until you see a heading that says "Date field(s)". Select the checkbox beside "Content: Event Date and Time... From date". Click "Update"
10. On the next page that appears, under "Operator", pick "Is greater than or equal to". Under "Date default", enter "now". Click "Update"
    
11. Click "Save". Note that after clicking "Save" you're again editing the default settings, not the upcoming events display. Be careful to select the display you mean to edit. I got that wrong lots of times

Set up the menus correctly:

1. The above puts the menu under the site navigation menu. We want it under the Primary Links menu. Click on Administer-> Site building-> Menus
2. Click on "edit" beside "Events"
3. Under the "Parent item" heading, select "" from the drop-down list
4. Click "Save". The "Events" menu will show up at the end of your primary menu. Drag it to the appropriate place in the list and click "Save configuration"

Create an event to test what you have so far:

1. Go to "Create Content"
2. Click on "Event"
3. Enter an event on the your current day or sometime in the future. It should be self-evident how. Watch that you end date is after the start date. The user interface doesn't automatically do it like some other programs do. Click "Save"
4. Enter another event with a date in the past. Again, make sure the end date is the same as the start date
   
5. Click on the "Events" menu on your primary menu. You should see the one future event you just entered, but not the past event
   

If the event doesn't show up, see the troubleshooting tips at the end of this post.

Create the "All Events" page:

1. Click on Administer-> Site building-> Views
2. Select "edit" for the "Event List"
3. Click the "Add Display" button (the drop down box above it should read "Page" already)
4. Under "Basic settings", click on "Page" beside "Name". Scroll down. Type in "All Events" and click "Update"
5. Click on "None" beside Title. Scroll down. Click "Override". Enter "All Events" and click "Update"
6. Under "Page settings", click on "None" beside "Path". Scroll down. Enter "events/all_events" and click "Update"
7. Click on "No menu" beside "Menu". Scroll down. Select the "Menu tab" radio button. Enter "All Events" under "Title". Click "Update"
8. Click "Save". Note that after clicking "Save" you're again editing the default settings, not the upcoming events display

Test again:

1. Click on the "Events" menu
2. You should see the table with the one future event. You should also see two tabs for "All Events" and "Upcoming Events"
3. Click on "All Events". You should see both events in the table
4. Click on "Upcoming Events". You should see just the future event again

Here are some troubleshooting tips:

• If the events aren't sorting the way you think they should according to the display's "Sort Criteria", click on the little gear beside "Style" under "Basic Settings". Scroll down. If the radio button under "Default Sort" is anything but "None", the sort here will override the sort in "Sort Criteria". Click on the radio button beside "None"
• With the Date module circa March-April 2009, you have to install a patch or you'll get error messages when creating events. Get the patch here and how to install it is here
  
• If the menu doesn't appear on the primary menu, or the tabs don't appear, make sure the paths for each display are below the same name as I showed above. In other words, the display paths should be of the form "x/y" and "x/z", even though the user will never navigate to "x"
• Recheck all your settings. This is tedious but essential. In my experience, any one being wrong can cause a mysterious and seemingly unrelated failure
• Be sure that you didn't accidentally change the defaults instead of overriding them, or didn't accidentally override something that should be taken from defaults. Values taken from the default display are in italics when looking at other displays

I hope this helps someone.

Building a Website for a Grassroots Group

As I've been blogging about recently, I've been building a Drupal-based website for my son's school's Parent Advisory Council (PAC). As a geek, the technical nature of putting together a content management system like Drupal for a site is interesting, and I learned a lot. However, what really interested me is the social aspect of it.

You can't have a business it seems without having a website. That must mean there's value for a business to have a website. But is there value for a grassroots group to have one? If technology truly is neutral, then grassroots groups must be able to benefit from websites. (I'm not sure technology is inherently neutral, but that's another topic.)

When we started, the PAC has a static HTML website updated infrequently by a parent who knew some HTML and how to FTP files to his server. It was out-of-date, since the parent's children had moved on to high school and he was obviously not so active in the PAC. Members of the PAC did want some place to post meeting dates and minutes, and to publicize their activities, so I offered to help out.

We set up a committee with four members. I built a test site under my company's site (jadesystems.ca) and we started to work.

Right from the start a few people really took to it and started to use it in ways that I couldn't keep up with. This included translation into Chinese, especially after I enabled the translation modules in Drupal.

I also set myself up to do screencasts, and posted a number of them. It's still early, but I've got some feedback from people that they find them very useful. For grassroots groups, where you don't site face to face with people every day, I think screencasts are going to prove to be an excellent tool.

I think the main lesson I've learned so far is this: I like Drupal and it's allowed me to do a lot. However, I can't help feeling that I'm writing in assembly language for content management. What I mean is that I've had to learn too much about how the system works to get it to do what I want. We're still a long way from grassroots groups being able to set up a website without someone with technical skills.

My dream would be something analagous to blogging. You simply go to a website and create your own community site. The task is inherently more complicated than setting up a blog, so it will never be as simple, but the skill set should be relatively similar. You should be able to pick and choose based on functions the user wants (e.g. a list of upcoming events), not functions based on how the programmer delivers them (e.g. a Views module with an obtuse and fragile configuration screen that lets you cobble together a list of upcoming events if you bang your head on it long enough).

More on this later.

Parent Advisory Council Website

The Parent Advisory Council (PAC) website is live at http://hastingspac.ca/ and is in a relatively stable form. Much work remains on the translations, but the bones of the site are there and it's up to everyone else to put up the content (muscle).

I think we've got a pretty good site for a volunteer group. Our goal is to increase participation in the PAC to benefit the education of our children.

I'll be posting more about the technology behind the site (Drupal and content management systems) this morning.

I'd love to hear comments and ideas you might have about the site.