Reverse engineering the Palm Address book and Todo Lists

I installed the latest version of the Palm Desktop software on a new computer and finally got around to (at least partially) deciphering the new format, which allows for more phone numbers and several addresses, as well as birthdays (I haven't decoded these yet). The new code is in a .c (C++) file which you can download here. compile with g++ -o dialer.exe newdialer.c

Since I never sync with the palm pilot, records are never deleted from the file, just marked as deleted (they'd be moved to the archive at next sync I believe) so I now only print out records if they don't have the "deleted" flag set.

After writing my address book dump code, someone pointed me to this page that describes the format: Palm Desktop Software ADDRESS.DAT and ADDRESS.ABA File Structure

Since I wrote this, someone has pointed me to a web page that describes the palm file format. You could search for file formats on the web, eg. myfileformats or wotsit.

Also, there's now an SDK for the Palm desktop 4, so one can write plug-ins that don't even need to know the file format. I wrote some simple import/export routines using that, but still use my dialer code for command line decoding of the file. (I suspect thta my plugin code made the desktop unstable).

I originally used my Palm V (IBM work pad c5) under Linux (with Gnome tools) and wrote some simple scripts to read the text "gcard" address book file and grep it etc. as well as to convert my old CSV format address book into gcard so that I could import it to the palm.

However, when I had to start working with Windows2000 (now XP), I found that the address book is stored in a binary format and I had no command line access to the address book. I spent a few evenings reverse engineering the format (mostly using "od" in cygwin) and came up with the following C program to dump addresses in text from the address book.

dialerxml.c source code. (newer version Feb 2007 - it was pointed out that before my html version lost some important escape characters, this is the original c.)

It's far from a complete deciphering of the palm format. I don't make any attempt to understand the binary fields except to identify some of the phone number/email address types and find the field and record separators. I didn't find any thing else on the web that tried to do the same thing or explained the format.

Here are some bash functions to use the program to access the address book from the command line. I mainly use dial <name> to search for a particular person's address/phone number.

Note also that I store email addresses followed by a space and an alias, and I use that alias to generate my .mailrc file using "gcardtomailrc" which will also cause xemacs to reload the new .mailrc file. I keep the invariable parts of my mailrc (alias lists and variables) in a ~/.mailrc_fixed file

  addressfile=/cygdrive/c/WorkPad/<USERNAME>/address/address.dat

function dial () 
{ 
    if [ -z "$1" ]; then
        ~/bin/source/dialer $addressfile | less;
    else
        ~/bin/source/dialer $addressfile | grep -i "$1";
    fi
}

## Reconstruct the .mailrc file from palm db. 
function palmtomailrc () 
{ 
    mv ~/.mailrc ~/.mailrc.bak && cp ~/.mailrc_fixed ~/.mailrc && ~/bin/source/dialer $addressfile | awk -F , '{for (i=5; i<=9; i++) if (match($i, "\\[Email\\]")) { email=$i; gsub("^ *\\[Email\\]: *","", email); gsub("\\\\","", email); if (match(email, "@")) print email;};}' | awk 'NF==2 {print "alias "$2" "$1}' | tr -d '\r' >>~/.mailrc;
    echo "< New / > old";
    sort ~/.mailrc >/tmp/m1 && sort ~/.mailrc.bak >/tmp/m2 && diff -b /tmp/m[12] && rm /tmp/m[12];
    /usr/local/bin/i686-pc-cygwin/gnuclient.exe -batch -eval '(rebuild-mail-aliases "'$HOME/.mailrc'")' && echo "Loaded to emacs!";
    echo "DUPES: ";
    awk '/alias/ {print $2}' ~/.mailrc | sort | uniq -d
}

function dial () { cat ~/Pilot/Address.gcrd | tr -d "\r" | awk -v search="$1" -f ~/bin/gcardtoplain.awk  | more ; } 
function dialf () { cat ~/Pilot/Address.gcrd | tr -d "\r" | awk -v search="$1" -v longformat=1 -f ~/bin/gcardtoplain.awk  | more ; } 
function addaddress () { echo "Type entries,one per line ^D to end. !!alias,email%%note/@hours"; 
	awk -F , -f ~/bin/addresstogcard.awk > /tmp/address$$;
	more /tmp/address$$;
	echo "If this is OK, press return, else ^C";
	read 
	cat  /tmp/address$$ >> $HOME/Pilot/Address.gcrd;
	}

function addresstomailrc () { awk '/!!/ {gsub("^[^!]*!!","!!"); gsub("!!","\n"); print $0}' ~/various/address.csv | awk -F , '/@/ {print "alias "$1" "$2;}' ; } 

# Make a stack of the aliases for each person 
# Then print out the aliases with the email addresses. 
function gcardtomailrc () { mv ~/.mailrc ~/.mailrc.bak; cp ~/.mailrc_fixed ~/.mailrc; cat ~/Pilot/Address.gcrd | tr -d '\r' | awk '
/^BEGIN:VCARD/ { delete alias; delete email; an=0; en=0} 
/^EMAIL.*ALIAS:/ { a=$1; gsub("^[^:]*:","",a); alias[an]=a; an++} 
/^EMAIL;INTERNET:/ { e=$1; gsub("^[^:]*:","",e); email[en]=e; en++; } 
  /^END:VCARD/ { for(i=0; i<en && i<an; i++) print "alias "alias[i]" "email[i]; } ' >> ~/.mailrc; 
# Now upload them to emacs! if gnuserv is running. 
gnuclient -batch -eval '(rebuild-mail-aliases "'$HOME/.mailrc'")' && echo "Loaded to emacs!"; 
cp ~/.mailrc //obo/home/aws
} 
function gcardtoplain () { 
	cat ~/Pilot/Address.gcrd  | tr '\r' '\n' | awk -f ~/bin/gcardtoplain.awk ;
	}

function palmtohtml () 
{
 dial | sed -e 's/^[, ]*//g; s/,[, ]*/, /g' | sort -f | awk -f ~/bin/awk/addresstohtml.awk  > ~/www/phones.html; 
}