Palm::SMS - parse SMS database files

Palm-SMS documentation Contained in the Palm-SMS distribution.

Index

Code Index:

NAME

Top

Palm::SMS - parse SMS database files

SYNOPSIS

Top

    use Palm::PDB;
    use Palm::SMS;

    my $pdb = new Palm::PDB;
    $pdb->Load("sms.pdb");

    my $record = $pdb->{records}[0];
    print "$record->{text}\n";

DESCRIPTION

Top

The SMS PDB handler is a helper class for the Palm::PDB module. It is intended for reading, manipulating, and writing the .pdb files used by Handspring SMS application on PalmOS devices such as Handspring Treo 270.

Palm::SMS module is the result of a reverse engineering attempt of trasforming a Handspring Treo 270 SMS PDB file into a plain text file. The PDB test file was produced by Handspring's application SMS v. 3.5H.

Due to lack of knowledge about how PDB files work and what format SMS database files conform to, at present this module is not suitable for from-scratch SMS generation. Conversely, you may find it extremely useful if you intend to extract SMS messages from merged PDB files and convert them to a human readable form.

Fields

    $record = $pdb->{records}[N];

    $record->{name}
    $record->{firstName}
    $record->{phone}
    $record->{folder}
    $record->{timestamp}
    $record->{text}

    $record->{smsh}
    $record->{unknown1}
    $record->{unknown2}
    $record->{unknown3}

The fields provided for each record are the following:

name

A string containing the name of the person who wrote the message.

firstname

A string containing the first name of the person who wrote the message.

phone

A string containing the phone number.

timestamp

An integer which represents the number of seconds elapsed from Unix epoch to message creation time.

It is worth noticing that there is no way of retrieving neither the TZ nor the DST out of data stored in the PDB file. This timestamp always expresses the time of your handheld's clock at message creation time. Hence, I suggest passing the value GMT as third argument to Date::Format's time2str() to get the right timestamp rappresentation:

  use Date::Format;
  ...
  $timestamp = time2str("%T %%Z",   $record->{timestamp}, "GMT");
  $timestamp = time2str($timestamp, $record->{timestamp}       );
folder

An integer which represents in which folder the message was stored. English folder names (such as Inbox or Sent) are available as

  $Palm::SMS::folders[$record->{folder}];
text

A string containing the message body.

The module provides additional fields which will probably be less commonly used.

smsh

This string of four bytes ("SMSh") is present at the start of each record.

unknown1
unknown2
unknown3

These fields contain a chunk of bytes whose function is not yet known. Please, refer to the ParseRecord() method.

Fields for the Treo 680

The Treo 680 uses different software, and Palm have not documented its message format. Consequently only some information is available, and some records that are extracted are extracted incorrectly. The following fields are available, and have been reverse-engineered from a single sample database. Consequently, you should treat their values with suspicion.

Treo 680 databases are read-only.

If the Palm::Treo680MessagesDB module is available, then that will be used instead. Over time, that module is intended to do a better job, as and when I figure out new bits of the puzzle.

device

This will always be "Treo 680"

direction

The direction of the SMS relative to your phone. This will be either 'inbound' or 'outbound'

number

The other party's phone number (same as for any other device)

name

The other party's name (same as for any other device)

text

The text of the message (same as for any other device)

type

A number representing the type of message. If this is 'unknown' then none of the above fields will be populated.

rawdata

The raw binary data of the record

METHODS

Top

new

  $pdb = new Palm::SMS;

Creates a new PDB, initialized with the various Palm::SMS fields and an empty record list.

Use this method if you're creating a SMS PDB from scratch.

new_Record

  $record = $pdb->new_Record;

  $record->{phone}  = "1234567890";
  $record->{folder} = 1;
  ...

  $phone  = $record->{phone};
  $folder = $Palm::SMS::folders[$record->{folder}];
  ...

Creates a new SMS record, with blank values for all of the fields.

new_Record does not add the new record to $pdb. For that, you want $pdb->append_Record.

Default field values are:

  name     : undef
  firstName: undef
  phone    : undef
  timestamp: localtime()
  folder   : 1
  text     : undef

  smsh     : "SMSh"
  unknown1 : undef
  unknown2 : undef
  unknown3 : undef

ParseRecord

ParseRecord() returns a parsed representation of the record, typically as a reference to a record object or anonymous hash. It is automatically called from within Palm::PDB and, as such, is not intented to be used directly from applications.

The record structure which an SMS posses is:

    smsh     :  4-byte ASCII string
    unknown1 :  2-byte data whose function is unknown
    timestamp:  32-bit, big-endian, unsigned integer rappresenting
                the number of seconds since 1904
    unknown2 :  26-byte data whose function is unknown
    phone    :  Null terminated string
    name     :  Null terminated string
    firstname:  Null terminated string
    unknown3 :  16-byte data whose function is unknown
    text     :  Null terminated (sent messages only) string

folder field value is copied from category field which is computed by Palm::PDB and then delted since there is no application info block (see Palm::StdAppInfo) in the PDB file.

unknown3 is empty for messages belonging to category 1 (folder Sent).

It is worth noticing that length, offset, and even availability of unknown data are not preserved between module version when their meaning becomes clear.

PackRecord

This is the converse of ParseRecord(). PackRecord() takes a record as returned by ParseRecord() and returns a string of raw data that can be written to the database file. As ParseRecord(), this function is not intended to be used directly from applications.

Because there are chunk of record data whose function is unknown (see ParseRecord()), this method may produce an invalid result, expecially when passed record was created from scratch via new_Record().

This method is granted to work if the record being packed has been unpacked from an existing PDB and no information has been added.

BUGS

Top

Not all data chunks have a known function. Hence, the module is suitable only for data extraction and simple database manipulations.

I heard rumors that SMS format changes with GSM network providers. Please, contact me if this module cannot correctly handle your messages, hopefully attaching a patch which corrects the shortcoming.

SEE ALSO

Top

Palm::PDB by Andrew Arensburger <arensb@ooblick.com>

Palm::Treo680MessagesDB

smssync v 1.0 by Janne Mäntyharju <janne.mantyharju@iki.fi>

AUTHOR and MAINTAINER

Top

Lorenzo Cappelletti <lorenzo.cappelletti@email.it> with some contributions from David Cantrell <david@cantrell.org.uk>.

David Cantrell is now the primary maintainer.

COPYRIGHT AND DISCLAIMER

Top

This program is Copyright 2005-2009 by Lorenzo Cappelletti and David Cantrell. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA.

Palm-SMS documentation Contained in the Palm-SMS distribution. 
# SMS.pm
#
# Perl module for reading, manipulating, and writing the .pdb files
# used by Handspring SMS application on PalmOS devices such as
# Handspring Treo 270.
#
# This code is provided "as is" with no warranty.  The exact terms
# under which you may use and (re)distribute it are detailed
# in the GNU General Public License, in the file COPYING.
#
# Copyright (C) 2005 Free Software Foundation, Inc.
#
# Written by Lorenzo Cappelletti <lorenzo.cappelletti@email.it>
#
#
# $Id: SMS.pm,v 1.1 2009/01/10 16:17:59 drhyde Exp $

package Palm::SMS;

use strict;
use warnings;
use Palm::Raw();
use vars qw($VERSION @ISA @folders);

$VERSION = 0.03;

@ISA = qw(Palm::Raw);


@folders = (                            # SMS folder names
  "Inbox",
  "Sent",
  "Pending",  # guessed
);

my $EPOCH_1904 = 2082844800;            # Difference between Palm's
                                        # epoch (Jan. 1, 1904) and
                                        # Unix's epoch (Jan. 1, 1970),
                                        # in seconds.

sub import {
  &Palm::PDB::RegisterPDBHandlers(__PACKAGE__,
                                  [ "SMS!", "DATA" ],
                                 );
  &Palm::PDB::RegisterPDBHandlers(__PACKAGE__,
                                  [ "HsCh", "SMct" ],
                                 );
  eval "use Palm::Treo680MessagesDB" ||
      &Palm::PDB::RegisterPDBHandlers(__PACKAGE__,    # GSM Treo 680
                                  [ "MsSt", "MsDb" ], # Messaging app v 2.6.1
                                 );
}


#'
sub new {
  my $classname = shift;
  my $self = $classname->SUPER::new(@_); # no need to rebless
  $self->{name} = "SMS Messages"; # default
  $self->{creator} = "SMS!";
  $self->{type} = "DATA";
  $self->{attributes}{resource} = 0; # not a resource db


  $self->{sort} = undef; # empty sort block

  $self->{records} = []; # empty list of records

  return $self;
}


sub new_Record {
  my $classname = shift;
  my $retval = $classname->SUPER::new_Record(@_);

  $retval->{name}       = undef;
  $retval->{firstName}  = undef;
  $retval->{phone}      = undef;
  $retval->{timestamp}  = localtime;
  $retval->{folder}     = 1;
  $retval->{text}       = undef;

  $retval->{smsh}       = "SMSh";
  $retval->{unknown1}   = undef;
  $retval->{unknown2}   = undef;
  $retval->{unknown2}   = undef;

  return $retval;
}


sub ParseRecord {
  my $self = shift;
  my %record = @_;
  my @unpack;

  my $smsh;       # each record starts with "SMSh": SMS handler?
                  # not on Treo 680
  my $unknown1;
  my $timestamp;
  my $unknown2;
  my $name;
  my $firstName;
  my $unknown3;
  my $phone;
  my $folder;
  my $text;

  if ($self->{creator} eq "HsCh") {
    ($smsh,
     $unknown1,
     $timestamp,
     $unknown2,
     $text,
    ) = unpack("a2 A4 N a24 Z* a*", $record{data});
    if ($timestamp eq "") {$timestamp=$EPOCH_1904;}
    else {$timestamp -= 14400;}
    if ($smsh eq "\0\0" ) { $phone="Target"; }
    else { $phone="Me"; }
  } elsif($self->{creator} eq 'MsSt') { # Treo 680
      my $buf = $record{data};
      my $type = 256 * ord(substr($buf, 10, 1)) + ord(substr($buf, 11, 1));
      my($dir, $num, $name, $msg) = ('', '', '', '');
      if($type == 0x400C || $type == 0x4009) { # 4009 not used by 680?
	  $dir = ($type == 0x400C) ? 'inbound' : 'outbound';
	  ($num, $name, $msg) = (split(/\00+/, substr($buf, 34)))[0, 1, 3];
	  $msg = substr($msg, 1);
      } elsif($type == 0) {
	  $dir = 'outbound';
	  ($num, $name, $msg) = split(/\00+/, substr($buf, 0x4C), 3);
	  $msg =~ s/^.{9}//s;
	  $msg =~ s/\00.*$//s;
      } elsif($type == 0x0002) {
	  $dir = 'outbound';
	  ($num, $name, $msg) = split(/\00+/, substr($buf, 0x46), 3);
	  $msg =~ s/^.Trsm....//s;
	  $msg =~ s/\00.*$//s;
      } else {
          $type = 'unknown';
      }
      @record{qw(device type direction phone name text rawdata)} =
          ("Treo 680", $type, $dir, $num, $name, $msg, $buf);
  } elsif ($record{category} == 0) {
    ### Inbox folder ###
    my $nameFlag;       # whether name and firstName are available
    my $extra;          # temporary string

    ($smsh,
     $unknown1,
     $timestamp,
     $unknown2,
     $phone,
     $extra,
    ) = unpack("A4 a2 N a26 Z* a*", $record{data});

    # unknown2 tells whether name and firstName are available
    $nameFlag = unpack("x7 H", $unknown2);
    if ($nameFlag eq "4") {
      ($name,
       $firstName,
       $extra,
      ) = unpack("Z* Z* a*", $extra);
    }

    # $extra's head contains unknown3 followed by "\d\0"
    ($unknown3, $text) = $extra =~ m/(.*?\d\0)([^\0]+)$/;

  } elsif ($record{category} == 1) {
    ### Sent folder ###
    my $unpack;

    ($smsh,
     $unknown1,
     $timestamp,
     $unknown2,
     $phone,
     $name,
     $firstName,
     $text,
    ) = unpack("A4 a2 N a26 Z* Z* Z* Z*", $record{data});
    $unknown3 = "";

  } elsif ($record{category} == 2) {
    ### Pending folder ###
    die "Never tried to parse a message from Pending folder";

  } else {
    die "Unknown category";

  }

  # Work out common extracted values
  $timestamp -= $EPOCH_1904;

  # Assign extracted values to record
  if($self->{creator} ne 'MsSt') {
      $record{name}      = $name;
      $record{firstName} = $firstName;
      $record{phone}     = $phone;
      $record{timestamp} = $timestamp;
      $record{folder}    = $record{category};
      $record{text}      = $text;

      $record{smsh}      = $smsh;
      $record{unknown1}  = $unknown1;
      $record{unknown2}  = $unknown2;
      $record{unknown3}  = $unknown3;
  }

  delete $record{data};

  return \%record;
}


sub PackRecord {
  my $self = shift;
  my $record = shift;
  my $retval;
  my $pack;

  $pack = "A4 a2 N a26 Z*";

  if ($record->{folder} == 0) {
    ### Inbox folder ###
    if (not ($record->{name} or $record->{firstName})) {
      $pack .= " a* A*";
      $retval = pack($pack,
                     $record->{smsh},
                     $record->{unknown1},
                     $record->{timestamp} + $EPOCH_1904,
                     $record->{unknown2},
                     $record->{phone},
                     $record->{unknown3},
                     $record->{text});

    } else {
      $pack .= " Z* Z* a* A*";
      $retval = pack($pack,
                     $record->{smsh},
                     $record->{unknown1},
                     $record->{timestamp} + $EPOCH_1904,
                     $record->{unknown2},
                     $record->{phone},
                     $record->{name},
                     $record->{firstName},
                     $record->{unknown3},
                     $record->{text});

    }

  } elsif ($record->{folder} ==1) {
    ### Sent folder ###
    $pack .= " Z* Z* Z*";
      $retval = pack($pack,
                     $record->{smsh},
                     $record->{unknown1},
                     $record->{timestamp} + $EPOCH_1904,
                     $record->{unknown2},
                     $record->{phone},
                     $record->{name},
                     $record->{firstName},
                     $record->{text});

  } elsif ($record->{folder} == 2) {
    ### Pending folder ###
    die "Never tried to pack a message to Pending folder";

  } else {
    die "Unknown category";

  }

  return $retval;
}

1;
__END__