The MikeCam

For the longest time, I’ve had a problem where this stupid eth1394 module would be loaded and become eth0 instead of my network card (it would take eth1). Today, I was working on getting a DV camera hooked up through firewire and I finally found a solution. Insert this line into /etc/modprobe.conf (for 2.6 kernels):
install eth1394 /bin/true

Yay!

http://chaosengine.net/p5020d says that “for a 2.4 you add the following to /etc/modutils/local then run update-modules: alias eth1394 off”

One of the problems I’ve had with all the removable devices that I use (iPod, SD cards, and external hard drive) is that while Mandrake 10.1 (Mandriva) is generally pretty good about hotplugging them, if I have more than one plugged in, their device name (and Mandriva’s auto-generated mount point) differs depending on the order they are plugged in. Previously, I solved that by always plugging the devices in in a certain order, but that’s less than ideal and quite annoying. Today I ran across this how-to for a USB memory key under RH9 that introduced me to devlabel, which will (hopefully!) allow me to assign permanent device names to my removable devices. Unfortunately, there don’t seem to be any Mandrake RPM’s available nor any previous user who’s set it up. I’m downloading the tarball to see if I can compile it (fingers crossed!). Interestingly, it seems to be developed by Dell (and more friendly promo page).

ADD THE DISK TO DEVLABEL

devlabel attempts to give the same device name to removable media,
wherever they appear on the bus. We want our memory disk to always
appear as /dev/usbdisk1 no matter what USB bus it is on or what other
USB disk might be present.

# devlabel printid -d /dev/sda1

S80:XXXXXXXXXXXXXXXXXXXX

# devlabel add -s /dev/usbdisk1 –partnum 1 -u ‘S80:XXXXXXXXXXXXXXXXXXXX’

Created symlink /dev/usbdisk1 -> /dev/sda1
Added /dev/usbdisk1 to /etc/sysconfig/devlabel

Ran across this funny, imaginary breakdown of how KDE developers work and live as compared to GNOME developers.

First Mandrake acquires Conectiva and becomes Mandriva, and now it’s just bought some of Lycoris. It’s an acquisitions monster! Actually, I’m happy. I hope all these new purchases translate into some kickass new Mandrake- er, Mandriva releases next year.

This is not only my new favorite Linux app, but also by far the best music player I’ve used, period. Amarok has terrific smart playlists, support for autodetecting track information based on musicbrainz fingerprinting, automatic downloading of album cover art from Amazon, an OSD feature to briefly overlay the info of a track as it begins playing, crossfade, song suggestions, automatic lyric download, and iPod support (although I haven’t tested this yet). It’s almost the perfect player.

Some small criticisms: The UI is very good, but a little cluttered in some places due to the number of features and amount of smart information available. It isn’t that bad, and I do have a ton of music, but it would be nice if the program could be sped up a little bit, as it lags a tiny bit sometimes. Could just be my computer and my collection, though. Inline tag editing (clicking a field in the playlist) like JuK’s would be nice, as would adding an item to the right-click menu to auto-detect track information, instead of having to open up the meta tag editor window. Playing music with the aRTs engine was a little bit troublesome at first, but aRTs needs serious help. Remembering what random songs were played, like in iTunes and Rhythmbox, would be great (it works going backwards, but not forwards). It doesn’t appear to have a graphic equalizer, but that’s about the biggest thing I’m missing.
Apparently there is an equalizer, under the Tools menu, that I missed. It is not available for the aRTs engine, however, and is rather limited. Inline tag editing is also in place, but is rather unintuitive – you have to select the song, then shift click on the field you want to change. More usability annoyances/bugs: If you change the album title of a song that is playing and try to download the cover art, it will not work because it is using the old album name. You can’t download cover art in the meta tag editor, only in the cover manager or while the song is playing. I am unable to edit the meta tags of a certain set of my songs, I think it may be due to their complex file names, but I have not confirmed yet.

As you can see, I don’t have any real problems or huge wishes thus far. I’d tried it in the past but it kept crashing on me, first on trying to load my music collection and then on trying to play music using aRTs. This time (after I got all the dependencies installed, grr), it worked perfectly and is playing very well – I highly recommend it.

I learned something today:

$ fortune -m LikeThis    
(knghtbrd)
%
<Overfiend> Thunder-: when you get { MessagesLikeThisFromYourHardDrive }
<Overfiend> Thunder-: it either means { TheDriverIsScrewy }
<Overfiend> or
<Overfiend> { YourDriveIsFlakingOut BackUpYourDataBeforeIt’sTooLate
            PrayToGod }
%

I’m posting from that a Knoppix 3.3 LiveCD on that old ThinkPad 600 I saved from the trash bin. Spent yesterday struggling to get the wireless card – a Netgear MA521 – to get an IP address from the DHCP server. Using Ndiswrapper with the driver they suggest worked great for detecting the card and to get signal and the correct network (according to Kwifimanager), but it just would not acquire the IP for some reason.

On the heels of that, this morning I started getting errors like this one:

end_request: I/O error, dev 03:05 (hda), sector 4432611
hda: read_intr: status=0x5b { DriveReady SeekComplete DataRequest Index Error }
hda: read_intr: error=0×40 { UncorrectableError }, LBAsect=4956834, sector=4432611

What I’ve gathered from newsgroup posts and reiserfsck output is that the hard drive is on its way out, just as the fortune says. I’m running badblocks as suggested by the “Bad block handling in ReiserFS” article in a last-ditch attempt to salvage the drive for a little while longer, but the list of bad blocks is enormous, so I’m fairly certain it’s shot. Bummer. Maybe I can still get it sold on eBay for $200…

On a side note, nothing but nothing is more annoying to me than accidently clicking a button or hitting some keyboard shortcut in the middle of typing a long post and then hitting back only to find that my entire text has been erased because it expired from the cache. Brilliant. I mean, could they not provide some sort of contingency for that, save form data little while longer? Cripes, it’s a royal pain in my butt.

Wrote another script today, this time to help me in my weekly backups. I’ve been running rsync as a cronjob to mirror all of my files weekly so I that I have a relatively recent backup in case of system failure or infection. I don’t use RAID because I don’t have the hardware and software RAID would suck resources, but primarily because RAID is meant to protect against hardware failure and to keep the system running, where I am more concerned with viruses or – more likely – mucking up my own system. With RAID the errors would be on all the drives and then I’d be screwed.

Anyway, I used to be backing up an 80gb internal IDE drive to a second one, but now I have a 160gb drive in an external enclosure, so now I’m backing up both internals to the external via Firewire. The problem that can arise with an external hard drive (as I discovered the first time I tried this) is that if for some reason it is not mounted (power is off, cable is disconnected, firewire module is throwing a fit), just running rsync will copy the entire drive onto itself in a folder where the mount should be. At least until it’s completely full and the system dies. Which is what happened to me. So I wrote a Bash script to check if the drive is mounted, attempt to mount it if it’s not, and then either run rsync to backup or exit with an error.

#!/bin/bash
BACKUPHD='/external'
EXCLUDE='/home/michael/rsync.exclude'

backup() {
echo -e "Running rsync to backup to $BACKUPHD\n"
rsync -ac --delete --exclude-from=$EXCLUDE / $BACKUPHD
}

if grep $BACKUPHD /etc/mtab
then
backup
exit 0
else
echo "Backup drive not mounted! Attempting to mount... "
if mount $BACKUPHD
then
echo -e "Success!\n"
backup
exit 0
else
echo -e "Failed!\n"
exit 1
fi
fi
exit 0

And the crontab for every Sunday at 3am:
# Backup
0 3 * * 0 /home/michael/sysutils/backup.sh

Well I’m doing a great job of avoiding my homework by writing these posts… Anyway, I wrote a couple Bash scripts this weekend to automate some common tasks involved with my site. The first one was to automate processing and updating of my chat logs. I run Xchat2 as an IRC client on my channel #the_end on irc.cyberarmy.net and keep logs. Every day, I run the IRC stats analyzer pisg and then FTP the resultant HTML file to my chat stats page. With the help of a cronjob and a few Bash commands, however, I no longer have to remember to do that little annoyance myself. Here it is:

#!/bin/bash
FTPSERVER='ftp.themikecam.com'
FTPUSER='username'
FTPPASS='password'
SOURCE='/data/mikecam/public_html/newmikecam/chatstats/index.html'
DEST='/public_html/newmikecam/chatstats/index.html'

echo -e "\nRunning pisg to analyze stats...\n"
/data/mikecam/utils/chatstats/pisg;

echo -e "Running ftp to upload stats page...\n"
ftp -n $FTPSERVER <<EOF
user $FTPUSER $FTPPASS
put $SOURCE $DEST
quit
EOF
echo "Done!"
exit 0

The only difficult/annoying part of this was figuring out how to pass commands to ftp once it was running. I couldn’t use echo because that would wait until FTP finished running to execute. I couldn’t manage to get the right search terms to find a solution on Google, but luckily Kyle came to my rescue again, making a suggestion that led me to the magic of Here Documents, which let me stream commands to FTP while it was running. Now I just have to figure out either how to make pisg only process the past week or two or write something to lop old days off of the current xchat log.

Woops, forgot the crontab! Here ’tis, every night at midnight:
0 0 * * * /data/mikecam/utils/chatstats.sh

My second script is to grab a webcam image and upload it. I’ve been using Camorama, which is a very nice little Gnome program to view a webcam and automate capture and FTP upload. Unfortunately, it suffers from a few deficiencies, including a lack of flexibility on image size (only choices are “small,” “medium,” and “large”) and on capture interval (it is in minutes rather than seconds). I also had come up with the idea of using my ASU web space to store my cam so I wouldn’t have to worry about using up transfer here on the MikeCam and could decrease the interval between shots. ASU, however, only permits secure FTP connections to manage files, so I had to write a script to use one of the already available capture scripts and the sftp program. Ta-da:

#!/bin/bash
export FTPUSER='username'
export FTPSERVER='general.asu.edu'
export FTPDIR='www'
export FTPPASS='password'
export CAMFILE='webcam/webcam.jpg'

while [ 1 ]; do
vidcat -d /dev/video0 -f jpeg -o $CAMFILE -q 60 -s 320x240
expect <<'END'
set timeout -1
match_max 1000000
spawn sftp $env(FTPUSER)@$env(FTPSERVER):$env(FTPDIR)/
expect "password: "
send -- "$env(FTPPASS)\r"
expect "sftp> "
send -- "put $env(CAMFILE)\r"
expect "sftp> "
send -- "quit\r"
END
sleep 5.1;
done

exit 0

A couple challenges here: First, finding a capture script that I could use to just capture and let me handle the transferring manually. I went with w3cam for now, but hopefully I can find something, either a replacement or an additional program, that will allow me to easily overlay text on the image so I can add “themikecam.com” and a timestamp. The second challenge was how to deal with SFTP. At first I thought I could do the same as FTP, but the SFTP program doesn’t allow you to pass it a password through an argument, nor does it allow you to use run it and use the “open” and “user” commands, so I had to figure out how to give it the password when it prompted for it. I searched on Google a bit, found someone who had posted about a similar problem, and someone suggested the “expect” program, which lets you wait until you get a certain output before continuing. As you can see, I used it to wait until it prompts for the password, then wait until it authenticates, then wait until the file is transferred. There’s also a loop to capture and upload the cam image every 5.1 seconds. I should probably rewrite the loop so that it doesn’t disconnect and reconnect every time.

Okay, so the first of a series of Linux tips tonight, or rather solutions to problems I’ve had:
I recently bought a ByteCC 3.5″ aluminum combo USB2.0/Firewire external enclosure and a 160gb Samsung hard drive both from NewEgg, for the purpose of having an external backup and/or mobile disk. I was able to format the disk to ReiserFS fine, but I’ve been having a problem whereby after transferring data steadily for a while, it would suddenly stop and spit errors like this:
ieee1394: sbp2: aborting sbp2 command
Test Unit Ready 00 00 00 00 00
ieee1394: sbp2: reset requested
ieee1394: sbp2: Generating sbp2 fetch agent reset
ieee1394: sbp2: aborting sbp2 command
ieee1394: sbp2: aborting sbp2 command
The result would be that I could not umount the drive nor rmmod the sbp2 module nor reboot because it would hang waiting for the module and I’d be forced to do a hard reset. Now, as to the solution to this, the linux1394 FAQ suggests trying the serialize_io option and a user on the mailing list said that this did in fact solve his problem. So, I opened up /etc/modprobe.conf and add the line “options sbp2 serialize_io=1″. I can’t say definitely yet whether that IS the solution, but cp has copied 8gb so far without so much as a chirp, so I’m hopeful that it will last.

Some miscellaneous tips I learned along the way:
When resetting after the failures, if I leave the fstab entry for the partition (/dev/sda1 in my case), Mandrake (10.1) will want to check it on boot because it wasn’t cleanly unmounted. Since the Firewire driver isn’t loaded at that point, however, it can’t access it and winds up constantly checking my second hard drive, /dev/hdb1. It would get mostly through checking it, then get stuck, and even if I went to single-user mode and ran reiserfsck to clear it, it would still check. The solution to this, it turns out, is just to comment out the fstab entry. I also now take the precaution of doing “init 1″ before doing my hard reset, so I can softly shut down everything possible.

Update: Just finished transferring 40gigs of data and no problems yet. I’ll be running rsync all night to backup my entire drive, so hopefully my luck will hold
Update 3 Mar. 2005: I’ve now been using my Firewire drive without a single problem for a week and half, so turning on serialize_io has definitely cured it.