Tyler J. Wagner (ابو پاسكال) ([info]giantlaser) wrote,
@ 2006-06-07 02:13:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
Current music:Run-D.M.C. - Walk This Way

Dapper, I Say!
Ubuntu 6.06 "Dapper Drake" went live on 1 June. (Geek translation: shiny new Linux!) I lasted one day before giving in to the upgrade madness.

My laptop is named "charcoal". For the last 8 months, I have not been kind to charcoal's dpkg database. I have cross-installed packages from Debian unstable, Marillat, the Penguin Liberation Front, and several random people who packaged stuff I didn't care to compile myself. Where packages weren't available I did compile software, including kernel modules. I also run both Crossover Office and VMWare, which are commercial packages with custom installers. My laptop runs the same software as the servers I manage, which means it's also the testbed for anything I plan to do to them. It's not uncommon to see it running Apache2, BIND, MySQL, various PHP-driven dynamic websites, and really bizarre firewall and routing configurations. In short, charcoal's filesystem on 31 May was a mess.

Dapper is a big upgrade from the last release, "Breezy Badger". So I opted to archive my files and reinstall from scratch. This was a really wise move.

I prefer Kubuntu (KDE) over Ubuntu (Gnome). Kubuntu Dapper is excellent. Out of the box, it's shiny and fast and functional. Nearly everything worked without configuration; not just on charcoal but on Jayme's cranky consumer-grade laptop too. We bought her laptop in the Baghdad markets 2.5 years ago. It had really irritating IRQ issues under Breezy and the wireless network card required ndiswrapper to work properly. Now it just works.

I am extremely pleased with the entire user experience. I spent a day playing with the new toys and migrating my old configuration files to the new system. Take my advice: move only what you need. The default Kubuntu UI configuration is nearly perfect.

For the three people on my friends list who run Linux, here are some recommendations:

1. Install KNetworkManager immediately (package "knetworkmanager"). This is what KWifiManager and the default Control Panel network settings manager should have been. If you are a laptop user, you want this.

2. Use EasyUbuntu to set up all the video/audio support and various non-free stuff, including the PLF repository you will need.

3. Install the following additional packages. You might have installed some of these with EasyUbuntu. For the casual users:

easytag firefox firefox-gnome-support flashplugin-nonfree gimp gimp-print gimp-svg grip inkscape kchmviewer xchm libk3b2-mp3 sun-java5-jre sun-java5-plugin scribus scribus-template

Privacy nuts want:

kgpg gnupg-agent pinentry-qt gaim gaim-otr gaim-extendedprefs

Network/security engineers and various geeks might want:

agrep apt-file deborphan dnsutils ethereal tethereal flip gip gtkterm iptraf iftop nmap screen sipcalc tree vim-gtk

All of those package lists are on one line so you can run "sudo apt-get install listofpackages". Try it.

I really don't care to repeat my mistakes from Breezy. No more testing server builds directly on charcoal. Instead, I will install a server build in a VMWare virtual machine, then break the VM over and over again. [info]ttmooney has been doing this for about a month now using Parallels, and it seems to be the way forward.




(Post a new comment)


[info]rightsock
2006-06-07 01:48 am UTC (link)
With a recommendation like that, how can I not give it a try? I'll add it to my rsync mirror and use kubuntu as my next upgrade path. I've been annoyed with FC for some time now, but laziness has kept me from changing. though I still will have to re-invent-the-wheel with "do this when specific usb key XYZ is inserted" as there are kernel bugs that I have to work around, and NOBODY has thought to re-implement this functionality since the pcmcia subsystem days.

(Reply to this) (Thread)


[info]giantlaser
2006-06-07 08:29 am UTC (link)
Yes, I would also like to specify actions based on the specific USB/IEEE1394 device I connect. That didn't work under Breezy, but I really haven't tested Dapper. You can specify what actions to take based on the type of device, but 90% of my devices classify as "Unmounted Removable Disk". Amarok also seems to have some kind of hooks in place for automatically managing inserted iPods. I haven't tested that either.

(Reply to this) (Parent)(Thread)


[info]rightsock
2006-06-07 04:57 pm UTC (link)
Here's what it looks like for FC, just to give you an idea of how bad it can be:

this is the easy one: launch a cmd-line palm backup (which coincidently doesn't work any more) when it is inserted and the button is pushed.

# usb automation
# first, install custom version of pilot-link
cd ~/Ftp/pilot-link-0.11.8/src/
make install
#
cd /etc/hotplug/
cat >usb/palm.usbmap <<'_EOF_'
# Sample entry (replace 0xVVVV and 0xPPPP with vendor ID and product ID respectively) :
# libusbscanner      0x0003      0xVVVV   0xPPPP    0x0000       0x0000       0x00         0x00            0x00            0x00            0x00               0x00               0x00000000
#
# usb module         match_flags idVendor idProduct bcdDevice_lo bcdDevice_hi bDeviceClass bDeviceSubClass bDeviceProtocol bInterfaceClass bInterfaceSubClass bInterfaceProtocol driver_info
# LABEL = USB product 830/60/100 vendor=830, prod=60
#  1   2     3    4 5 6 7 8 9 10 11 12 13
palm 0x3 0x830 0x60 0 0 0 0 0  0  0  0  0
_EOF_
cat >usb/palm <<'_EOF_'
#!/bin/sh

sleep 3

# yuck. I have to grep /var/log/messages??
PORT=`tail -100 /var/log/messages | grep "converter now attached to tty" | fmt -1 | tail -1`
PORT=`echo $PORT`
if [ "x$PORT" = 'x' ] ; then
        logger "error, can't find HotSync port description in syslog. guessing ttyUSB0"
        PORT=ttyUSB0
        fi

logger "launching backup on port $PORT in 3 seconds..."

( sleep 3 ; env PILOTRATE=115200 /home/kjw/Ftp/pilot-link-0.11.8/src/pilot-xfer -p /dev/$PORT -U --sync /home/kjw/Src/pilot-autobackup/ ) &
# the --sync is my personal addition. it does a backup and stores it in the users' name as a sub directory. this would allow a palm dock to be put in the foyer and let multiple people to back up their palm to the same computer.

_EOF_
chmod +x usb/palm

(Reply to this) (Parent)(Thread)


[info]grumpy_sysadmin
2006-06-08 01:15 am UTC (link)
Oi. Did you consider <tt>? Or maybe a link to plain text? I'm going to have to fire up lynx to read this whole post because modern web browsers have no concept of "sane columns to display"...

(Reply to this) (Parent)(Thread)


[info]rightsock
2006-06-08 03:42 am UTC (link)
http://rightsock.com/~kjw/tmp/block.agent.txt

for now. it'll disappear eventually.

note that this is based on FC3

(Reply to this) (Parent)


[info]rightsock
2006-06-07 05:06 pm UTC (link)
I don't have thorough instructions for usb hotplug, but due to kernel bugs it's far far worse. Basically what happens is that on insert, this script is called with arguments of sda. then later it gets called with arguments of sda1, sda2, sda3 for as many partitions as there are on the device. Problem is that if you "mount /dev/sda1 /mnt/usb" it triggers the "re-insertion" of /dev/sda, and the script is called again. When that happens, the automatically created /dev/sda1 devices disappear! (Note that if you manually create your own mount devices, eg /dev/my_sda1, they stick around). So part of the solution is to specify the filesystem time (because trying to auto-probe involves talking to /dev/sda which contains the partition table, which triggers the re-insertaion), and the bigger part is to just ignore re-insertions for 60 seconds and sleep and wait until the devices I want appear. Yuck.

I'll document it more thoroughly next time I do this, and fwd to you. But for now, here's my HEAVILY modified /etc/hotplug/block.agent

$ expand -4 block.agent
#!/bin/sh
#
# BLOCK specific hotplug policy agent
#
# userspace plugins for scripts to run, etc
#
# things user scripts can expect to have.
#
# tested against pcmcia flash cards and the like.
#
# First call for the main drive device (hdc):
# HOTPLUG_TIME='Sat Aug 13 18:37:11 PDT 2005'
# PHYSDEVPATH=/devices/ide1/1.0
# SUBSYSTEM=block
# HOTPLUG_ARGS=
# DEVPATH=/block/hdc
# MINOR=0
# HOTPLUG_TYPE=block
# ACTION=add
# MAJOR=22
# PHYSDEVDRIVER=ide-disk
# PHYSDEVBUS=ide
# SEQNUM=1154
#
# Second through N calls for each individual partitions (hdc1, etc)
# HOTPLUG_TIME='Sat Aug 13 18:37:11 PDT 2005'
# PHYSDEVPATH=/devices/ide1/1.0
# SUBSYSTEM=block
# HOTPLUG_ARGS=
# DEVPATH=/block/hdc/hdc1
# MINOR=1
# HOTPLUG_TYPE=block
# ACTION=add
# MAJOR=22
# PHYSDEVDRIVER=ide-disk
# PHYSDEVBUS=ide
# SEQNUM=1155
#

exec >>/tmp/block.agent 2>&1
DEBUG=yes
export DEBUG

cd /etc/hotplug
. ./hotplug.functions

log_to_stdout >>/var/log/hotplug/events

USER_MAP_DIR="$HOTPLUG_DIR/block/"

devpath_to_dev()
{
    set $(echo $1 | tr / ' ')
    if [ "$1" != 'block' ] ; then
        mesg "impossible: $1 != 'block'?"
        fi
    while [ -n "$2" ] ; do
        shift
        done
    echo "$1"
}


(Reply to this) (Parent)(Thread)

part 2
[info]rightsock
2006-06-07 05:07 pm UTC (link)
block_map_modules()
{
# Input: stdin should be the /etc/hotplug/block/*.usermap files, one at a time.
# Output: appends to global $DRIVERS, as matches are found
    while read LINE ; do
        if [ -n "`echo \"$LINE\" | egrep '^[    ]*#'`" ] ; then
            # skip comments
            continue
            fi
        # parse usermap line
        set $LINE
        DRIVER=$1
        SERIAL=$2
        PARTITION=$3
        if [ "$DRIVER" = '#' ] ; then
            continue
            fi
        #
        DEVICE=$(devpath_to_dev $DEVPATH)
        debug_mesg "device=$DEVICE"
        case $DEVICE in
        hd*1|hd*2|hd*3|hd*4|hd*5|hd*6|hd*7|hd*8|hd*9|hd*0)
            debug_mesg "found partition. checking for device /dev/$DEVICE "
            # disallow multiple init's to run within 60 seconds
            if ! lockfile -1 -r 1 -l 60 "/var/run/$DEVICE" >/dev/null ; then
                debug_mesg "abort. someone else has the $DEVICE lock (/var/run/$DEVICE)"
                return
                fi
            debug_mesg "I have the $DEVICE lock"
            BASEOK=false
            for N in 5 4 3 2 1 ; do
                ls -dal /dev/$DEVICE 2>&1 | mesg
                if [ ! -b /dev/$DEVICE ] ; then
                    debug_mesg "stall #$N to init partition $DEVICE"
                    sleep 2
                    continue
                    fi
                #else
                BASEOK=true
                break
                done
            if ! $BASEOK ; then
                debug_mesg "timed out waiting for part $DEVICE to init"
                fi
            BASE="`echo \"$DEVICE\" | sed -e 's/\(hd[a-z]\)\([0-9][0-9]*\)/\1/'`"
            if [ "$BASE" = "$DEVICE" -o -z "$BASE" ] ; then
                debug_mesg "Programmer error; DEVICE=$DEVICE -> BASE=$BASE"
                fi
            # wait until base device is properly initialized
            BASEOK=false
            for N in 5 4 3 2 1 ; do
                debug_mesg "stall #$N to init $DEVICE"
                sleep 2
                if [ ! -f /var/run/$BASE ] ; then continue ; fi
                DATA=`tail /var/run/$BASE | egrep "^$BASE " | tail -1`
                debug_mesg "found $DATA in /var/run/$BASE"
                set $DATA
                DEVICE_=$1
                SERIAL=$2
                TIMESTAMP=$3
                NOW=`date +%s`
                if [ `expr $NOW - $TIMESTAMP` -gt 3 ] ; then
                    debug_mesg "base ok. continuing init of $DEVICE"
                    BASEOK=true
                    break
                    fi
                # else continue
                done
            if ! $BASEOK ; then
                debug_mesg "timed out waiting for $BASE to init"
                continue
                fi
            ;;
        hd*)
            debug_mesg "found drive. checking for device /dev/$DEVICE "
            ls -dal /dev/$DEVICE 2>&1 | mesg
            # serial number only found on device (hdc), not partition (hdc1)
            if [ "$(echo $DEVICE | sed -e 's/^.*[0-9]\$/0/')" != '0' ] ; then
                eval $(hdparm -i /dev/$DEVICE | sed -n -e "
                    /=/{
                    s/^ //g
                    s/, /\n/g
                    s/=/=\"/g
                    s/\n/\"\n/g
                    s/$/\"/
                    p
                    b }"
                    )
                fi
            if [ "$SERIAL" != '*' -a "$SerialNo" != "$SERIAL" ] ; then
                debug_mesg "Fail: serial $SerialNo != $SERIAL"
                continue
                fi
            debug_mesg "SUCESS: serial $SerialNo == $SERIAL"
            echo "$DEVICE $SERIAL `date +%s`" >>/var/run/$DEVICE
            ;;
        esac
        debug_mesg "success: found driver $DRIVER"
        DRIVERS="$DRIVERS $DRIVER"
        done
}

(Reply to this) (Parent)(Thread)

part 3
[info]rightsock
2006-06-07 05:07 pm UTC (link)

#
# What to do with this BLOCK hotplug event?
#

case $ACTION in
add)
    debug_mesg "launched $0"
    for MAP in $USER_MAP_DIR/*.usermap ; do
        if [ ! -f "$MAP" ] ; then
            mesg "Error: No such usermap $MAP"
            continue
            fi
        debug_mesg "Checking usermap $MAP"
        block_map_modules <$MAP
        for DRIVER in $DRIVERS ; do
            mesg "Matched driver $DRIVER"
            if [ -x "$USER_MAP_DIR/$DRIVER" ] ; then
                mesg "+ $USER_MAP_DIR/$DRIVER"
                $USER_MAP_DIR/$DRIVER
            else
                mesg "Error: Driver script $USER_MAP_DIR/$DRIVER is not executable"
                fi
            done
        done
    ;;

remove)
    debug_mesg "BLOCK $ACTION event not supported"
    exit 1
    ;;

*)
    debug_mesg "BLOCK $ACTION event not supported"
    exit 1
    ;;

esac

(Reply to this) (Parent)


[info]giantlaser
2006-06-07 10:15 pm UTC (link)
Wow. Hotplug doesn't even exist any longer in Dapper. I can see a loadable kernel module, but the rest of it is gone. Can't say I miss it.

(Reply to this) (Parent)(Thread)


[info]rightsock
2006-06-08 03:40 am UTC (link)
well, goodbye and good riddance! Its architecture never made any sense to me. It was hugely complicated, and only got in the way. didn't actually help me DO anything useful.

(Reply to this) (Parent)


[info]grumpy_sysadmin
2006-06-08 01:19 am UTC (link)
You're probably already doing something similar if not more, but (my personal friend's) quick hack for this solved at least my usage cases.

(Reply to this) (Parent)(Thread)


[info]grumpy_sysadmin
2006-06-08 01:22 am UTC (link)
Right... and absent needing to work around hotplug, that's probably meaningless, and it makes a heap of assumptions about the device inserted, rather than doing things based on it. SO NEVER MIND.

(Reply to this) (Parent)


[info]rightsock
2006-06-08 03:35 am UTC (link)
Oh, I've done that before too. except more painfully since I was using xauth. this script assumes that you have disabled xauth. It's messy, in any case.

(Reply to this) (Parent)


[info]dataghost13
2006-06-07 02:00 am UTC (link)
Thanx Tyler! Easy Ubuntu was nicer to use than Automatix!! I'm also suggesting it for use to my Ubuntu friends!

(Reply to this)


[info]phuff
2006-06-07 04:37 am UTC (link)
vim-gtk

What! No Emacs? ;)

(Reply to this) (Thread)


[info]grumpy_sysadmin
2006-06-07 06:53 am UTC (link)
Never mind vim-gtk being blasphemy in and of itself, of at least two varities.

First variety: it's not lemacs + X + vi keyboard config.

Second variety: it's not ed(1) on a serial console + 9V battery + two paperclips.

(Reply to this) (Parent)(Thread)


[info]not_secure
2006-06-07 03:46 pm UTC (link)
vi keyboard config?? MORE blasphemy!

what's wrong with Emacs as an OS?

(Reply to this) (Parent)(Thread)


[info]grumpy_sysadmin
2006-06-08 01:22 am UTC (link)
s,wrong,not &,

(Reply to this) (Parent)(Thread)


[info]not_secure
2006-06-08 01:55 am UTC (link)
it is on the LispM and I have two 3645s at home still. I had to give up Teco to learn it. and I wrote my BAL Emulator project in Teco 8^)

(Reply to this) (Parent)


[info]grumpy_sysadmin
2006-06-07 06:49 am UTC (link)
I do hope you're not counting me as one of the three. I've long since given up on OSS giving me a base GUI that actually works, even though some apps (GIMP, OpenOffice) are fine. In case that didn't make it clear, I'm (long since) back on the Mac OS X, and loving it like a controlled substance.

(Reply to this) (Thread)

PS
[info]grumpy_sysadmin
2006-06-07 06:58 am UTC (link)
I haven't forgotten your yet-more backup question(s), I just haven't been drunk enough early enough to make a new post, which the question deserves, if only by volume of response.

Additionally, I want you to say something righ' heah regarding how close Bacula comes to the admirable and appropriate goals, in no small part because this weeks project is getting the first 1.5 TB of what will probably be 3 TB at home before November online. This is data I can but would really prefer not to have to rebuild, see.

(Reply to this) (Parent)(Thread)

Re: PS
[info]giantlaser
2006-06-07 08:24 am UTC (link)
I'm editing that response for clarity. Ignore the first, which I have deleted.

We've been using Bacula for a few months now. I have tested restores, but two days ago we had our first real test. The machine was fine, but we needed to restore a data tree which had been deleted on the previous Friday. The restore process was:

1. SSH to the backup director host (because I don't allow bacula admin via remove bconsole session).
2. Type "restore" and choose "Select backup for a client before a specified time", then choose the host and date.
3. Bacula builds a virtual filesystem tree from the various full/differential/incremental backups, then dumps you into a shell. You basically have shell navigation, ls, and the ability to mark and unmark files and directories recursively for restore.
4. Exit the shell, confirm some stuff, and optionally send the restore to a temp directory on that host instead of overwriting the existing directory trees.
5. Wait for the data to copy. You'll get an email and a message in bconsole when it is done.

I still haven't done a bare-metal restore, but the worst case scenario is as you said: install OS, install bacula file daemon, restore.

(Reply to this) (Parent)(Thread)

Re: PS
[info]grumpy_sysadmin
2006-06-08 01:30 am UTC (link)
The "builds a virtual file system" is a neat trick... especially if you mean "and mounts it, but gives I/O errors on file reads", since you can build directories just fine with only TOC info. It sounds, though, like what you're describing is essentially what ufsrestore (or vxfsrestore, or whatever) had done forever. Not that that's a bad thing; it's a completely appropriate (CLI) interface that's way better than anything NetBackup provides. (Even in the NBU GUI, restoring the file you wanted, in full, from the date-of-change you wanted is... excruciating. On the command line, it's a lot of bplist blahblahblah (the command requires something like 5 arguments to do anything at all).

Calling "install base OS, possibly from customized boot CD, install backup client, restore data" the "worst case" is a disservice: it's often the best bet (shortest time, least effort) for BMR. It's important that the process be clearly documented and the (custom) boot CD(s) kept current, but a process that has three logical steps rather than just "push a button" is a far better plan... 'cause what the fuck do you do when "push a button" fails? You don't even know where or why it failed, generally...

(Reply to this) (Parent)

Re: PS
[info]giantlaser
2006-06-07 08:26 am UTC (link)
Postscriptum: The only down side is that the "ls" command doesn't give file stats like "ls -l" might give. So you can't check file sizes or dates or what have you. It might also be nice to have a GUI navigator, which I do think exists. But since I don't allow remote Bacula mangement sessions, SSH + bconsole is the way I do it. I'll look into that if you are actually interested.

(Reply to this) (Parent)(Thread)

Re: PS
[info]grumpy_sysadmin
2006-06-08 01:31 am UTC (link)
Eh, don't go out of your way. I know enough at this point that I'm willing to try it, and likely to fix anything I find that's broken once I'm that far invested.

Sure, most of what I'll be putting on tape came off of recordable disks... but it takes a Long Time to load all those recordable disks again, dammit.

(Reply to this) (Parent)


[info]o_foruse_o
2006-06-07 10:56 am UTC (link)
I'm (long since) back on the Mac OS X, and loving it like a controlled substance.

Ohhh, I heart that line...

(Reply to this) (Parent)


[info]not_secure
2006-06-07 01:26 pm UTC (link)
I'm probably not on your radar as one of the three but I've got several Linux boxen running, from my main router/firewall using RH8 to my PVR running KnoppMyth (Knoppix/Debian unstable) to a hobby business with custom hardware running on RH7 to old Macs running various PPC flavors and a 4 way KVM hosting my firewall, my OSX iBook, my Win2k work laptop and an old Win98SE machine for some custom hardware apps that only run there.

My general, every day usage machine is the iBook. I lust after one of the new intel powerbooks as the ability to dual boot into windows for work will reduce my carry on baggage weight and clutter

2 people in the house, 10+ computers constantly running on the home net 8^)

(Reply to this) (Thread)

MythTV
[info]not_secure
2006-06-07 03:34 pm UTC (link)
a terabyte of tv programs I recorded because I didn't have enough time to watch when they aired and STILL don't.

(Reply to this) (Parent)

Not entirely smooth
[info]lhoriman
2006-06-13 09:06 am UTC (link)
I'm catching up on LJ, noticed your post.

I didn't even last a day. I upgraded my colo box on the day-of. Boy was that almost a big mistake...

Dapper's PCMCIA software has problems starting on the Breezey SMP kernel, and horks the install to the point of locking up the machine. Fortunately it happened in two stages, the first of which left me enough time to scour the internet for other people's experiences. Changing my GRUB default to the non-SMP kernel (and the remote power switch) saved me a trip to the colo. The box doesn't even have a PCMCIA device, ugh.

Seems to work great now though. Well enough that I'm migrating all of kink.com's production facility over to it, and deploying our apps as .DEB files.

(Reply to this) (Thread)

Re: Not entirely smooth
[info]giantlaser
2006-06-13 09:43 am UTC (link)
I haven't moved any of our servers to Dapper yet. We've knee deep in other problems to deal with a migration.

Are you running the server-compiled kernel or the default dapper kernel on that unit?

(Reply to this) (Parent)(Thread)

Re: Not entirely smooth
[info]lhoriman
2006-06-13 10:07 am UTC (link)
# uname -a
Linux wasabi 2.6.15-23-686 #1 SMP PREEMPT Tue May 23 14:03:07 UTC 2006 i686 GNU/Linux

Wasabi was Breezey Server. I ran the upgrade-dist and this is what I got.

We're not exactly /moving/ our existing systems at kink; since we're cycling out most of the systems over the next six months, we're just replacing everything with Dapper as we go. Lazy-man's migration :-) I'm really looking forward to seeing how deploying our own apps as .deb files will go.

(Reply to this) (Parent)


Create an Account
Forgot your login?
Login w/ OpenID
English • Español • Deutsch • Русский…