Blackberry Date Formats

Timestamps within Blackberry IPD files follow 3 different date formats depending on which database they are stored in. At first it seems pretty odd, but then it’s probably the work of independent programmers/teams working on different databases and associated software for the device.

1. Call Log, Phone History and SMS Date

The date used in these databases is a Java date 8 bytes in size. Java dates are similar to unix timestamps, with only a multiplier and larger size (8 bytes instead of 4 for unix) for better precision.

Javadate = 1000 * UnixTimestamp

Hence to convert back, simply divide the 8 byte timestamp by 1000 and you get a unix timestamp.

2. Calendar Date

This date’s precision is only to the number of minutes. It is the number of minutes since 1 Jan 1900 0:0:0. Converting this date to standard unix timestamp (which starts at 1 Jan 1970) means we have to subtract the difference equivalent to the number of minutes between 1900 and 1970.

Taking into account the 17 leap years also between 1900 and 1970, we calculate the difference.

Difference = (Number of years * Minutes per year) + (Number of Leap Years * Minutes per day)
= (70 * 365 * 24 * 60) + (17 * 24 * 60)
=36816480 minutes

Our simple formula for conversion will be

UnixTimestamp = (CalendarDate – 36816480) * 60

3. Email Date

The dates used in email metadata (sent and received dates) are stored as 2 byte date value and 2 byte time value.

2 byte Date

The date is stored with upper 7 bits representing the year since 1996, ie, 1 = 1997, 2 = 1998 and so on. The next 4 bits represent the month (1-12) and the remaining 5 bites represent the day (0-31).

+----------------+----------------+

|      BYTE 1    |      BYTE 0    |

| 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 |

  \___________/ \_____/ \_______/

       year      month     day

2 byte Time

Time is stored with a precision of 2 seconds. That means that 14:05 will be represented as 14:04 and 14:06 as 14:06. The creators did not think that this level of fine precision would ever be required for an email’s metadata date. One reason for this is that you really need 17 bits to represent all possible values when hours, minutes and seconds are stored separately, 5 bits for hours (0-23), 6 bits for minutes (0-59) and 6 bits for seconds (0-59). But only 16 bits are available in 2 bytes, so the BB developers have dropped a bit from the seconds field, hence only 5 bits are used as shown below.

+----------------+----------------+

|      BYTE 1    |     BYTE 0     |

| 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 |

  \_______/ \_________/ \________/

     hour      minute       sec

Enscript Tutorial 1 - Parse XP System Restore Logs

This post is about enscript programming using a real world example, parsing of the windows XP restore point logs to extract original filename and file path information of files collected by System Restore. Link to complete code file is at the bottom of this post.

System Restore
In windows XP, the system restore functionality creates backup copies of files under the "<DRIVE>\System Volume Information" folder. These files are identical copies of the original in data and metadata except for two details, the filename and path. The filename consisting of “name. extension” has the correct extension but the name is now "Axxxxxxx.ext" where x denotes a number. So commonly files like A0000001.ini, A0000002.txt and A0000005.lnk are seen here.

For analysis, we need to know the filename and full path of the Axxxxxxx files. This information can be parsed from the change.log or change.log.x files found in the same folder. We shall not discuss the format of the change.log file as it is documented elsewhere on the web in detail. Instead we shall focus on how to write a program to accomplish this task using enscript.

How to parse file name and path from change.log?

We shall use a shortcut instead of reading the entire structure of the file. After studying the format, I have understood that the path is present in a structure that begins with the signature (hex string) "12EFCDAB" and is 64 bytes from the signature. We shall therefore search for the hex string "12EFCDAB" and then skip 64 bytes to get to the path string. 8 bytes beyond the path is the filename. This will be our strategy.

Searching in enscript

The method to search is not a one liner and takes some setup code. It is roughly 4 steps:

1. Create SearchClass object
2. Add keyword(s)
3. Open a file and use the search object to find hits
4. Iterate through the hits array to get details of every hit

Program logic and concepts

We have broken down our program into a set of small manageable tasks for ease of understanding.

1. Recurse folders to find change.log.x files in System Volume Information
   
   The change.log files will always have a full path resembling
   "\System Volume Information\_restore{GUID}\RPxx\change.log.x"
   We can use this information to craft logic as follows:
   
   
   forall (EntryClass e in c.EntryRoot()) {
     if (e.Name().Find("change.log") == 0 &&
         e.Parent().Name().Find("RP") == 0 &&
         e.Parent().Parent().Name().Find("_restore{") == 0)
     {
     // Found change.log, now process it here
     }
   }


2. Search file for our artifact signature (12EFCDAB)
   
   SearchClass search();  // create SearchClass object
   search.AddKeyword("\\x12\\xEF\\xCD\\xAB....\\x15\\x00\\x00\\x00....", KeywordClass::GREP);   // add 16 byte GREP keyword
   search.Create();       // initialize search object
   EntryFileClass file(); // create file object
   file.Open(entry);      // open entry as file
   search.Find(file);     // execute the search on the file
   foreach (SearchClass::HitClass hit in search.GetHits()) {
     // Process search hits here
   }
   


3. Parse artifacts and print to console
   
   Once we have the hit object, we can seek into the file at the appropriate offsets and read out the data.
   
   file.Seek(hit.Offset() + 64); // seek to 64 from hit offset
   String fullpath;
   String newfilename;
   file.SetCodePage(CodePageClass::UNICODE); // codepage set to unicode
   file.ReadString(fullpath);    // Read path
   file.Skip(8);                 // Skip 8 bytes
   file.ReadString(newfilename); // Read name, ie, Axxxxxxx
   Console.WriteLine(newfilename + "      " + fullpath); // Print information to console

Sample script output

Click here to download the complete enscript code file.