Writing Center

In honor of #Marchintosh, I threatened in an earlier post to discuss The Writing Center, one of the many writing programs marketed by the Learning Company for the Mac. This one was developed by Datapak Software, Inc and I think they wanted to watch the world burn.

This format was different enough from the Student Writing Center and the “Ultimate Writing & Creativity Center” to need its own post. Moreover, I am pretty sure the developers of this software were actively trying to frustrate anyone trying to document the format. Let me explain.

In the early Macintosh world, very rarely were extensions used. Current systems use extensions to link the file to an application which can open the file. On the Mac, the system would use special attributes called Type / Creator codes. These codes were registered with Apple so they would be unique to a specific software and type of file. The codes used the FourCC system and unfortunately Apple never released a full list of codes used. Some folks over the years have tried to document as many as they can. Many used simple understandable codes, for example, A Microsoft Word document has a Type / Creator of W6BN / MSWD. The creator code of MSWD is very readable, and the type code W6BN is unique to a document from version 6 of Microsoft Word.

This Sample Report file from The Writing Center, when investigated with the ResEdit tool show interesting Type / Creator codes. If we look at the hexadecimals values for the codes. The first four bytes are the Type code and the second set of 4 bytes are the Creator code.

xattr -p com.apple.FinderInfo "Sample Report" 
0000   0A 57 50 31 0A 1A 57 50 01 00 00 00 00 00 00 00    .WP1..WP........

getfileinfo "Sample Report" 
file: "Sample Report"
type: "\nWP1"
creator: "\n\^ZWP"
attributes: avbstclInmedz
created: 10/13/1990 00:10:54
modified: 07/25/1991 11:58:20

The first thing to know is the encoding for all Type / Creator codes is MacRoman, so if we look up the hexadecimal code for “0A” we learn it is the character for a new Line Feed, why in the world would you use the line feed character? The developers must have had a sense of humor, or are psychopaths, and I’m leaning toward the latter. Trying to put this character into any sort of spreadsheet or text based document with other codes throws everything off! When I try and use a spreadsheet with a group of codes and then use a script to look them up on the command line I get crazy formatting. Not to mentioned the second character in the creator code is “1A” which is a substitute character.

This is just one example of crazy characters being used in Type / Creator codes. Stay tuned for more on these in future discussions.

Even though the Type / Creator codes are very useful in identification of this format, often times the Finder attribute is lost. This can happen if the file is moved off an HFS disk, usually a network or through the internet. Then all we have is the binary data fork and a file with no extension. So finding a signature to identify this format is useful.

hexdump -C "Sample Report" | head
00000000  00 12 cf fc 00 00 05 78  00 00 00 00 01 18 01 eb  |.......x........|
00000010  ff ff ff c4 ff ff ff c4  00 00 02 82 00 00 02 28  |...............(|
00000020  00 00 00 00 00 00 00 00  00 00 05 76 00 00 00 30  |...........v...0|
00000030  00 00 02 70 00 aa 00 00  05 76 00 00 00 30 00 00  |...p.....v...0..|
00000040  02 70 00 aa 00 00 00 00  00 00 00 00 00 00 00 00  |.p..............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 12  |................|
00000070  d1 2c 00 00 05 3f 00 00  00 00 01 00 06 47 65 6e  |.,...?.......Gen|
00000080  65 76 61 00 00 00 00 00  00 00 00 00 00 00 00 00  |eva.............|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|

hexdump -C WC-s01 | head        
00000000  03 df cd 9c 00 00 00 09  00 00 00 00 02 c3 02 64  |...............d|
00000010  00 00 00 00 00 00 00 00  00 00 00 59 00 00 02 64  |...........Y...d|
00000020  00 00 00 00 00 00 00 00  00 00 00 07 00 00 00 00  |................|
00000030  00 00 00 00 00 79 00 00  00 07 00 00 00 00 00 00  |.....y..........|
00000040  00 00 00 79 00 00 00 00  00 00 00 00 00 00 00 00  |...y............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 03 df  |................|
00000070  cd 78 00 00 00 00 00 00  00 00 01 00 06 47 65 6e  |.x...........Gen|
00000080  65 76 61 00 00 00 00 00  00 00 00 00 00 00 00 00  |eva.............|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|

Looking at the hexadecimal values of the header of a couple samples doesn’t initially look promising, the first few bytes are very different meaning there is no magic bytes at the beginning of the file. In fact the only thing the same is the mention of the Geneva font used in the document. Looking further into the files.

hexdump -C "Sample Report"       
00000000  00 12 cf fc 00 00 05 78  00 00 00 00 01 18 01 eb  |.......x........|
...
000000b0  00 00 00 00 00 00 00 02  84 28 ff ff 00 00 00 00  |.........(......|
000000c0  00 17 4e 26 00 12 d2 fc  00 00 00 00 00 12 d0 88  |..N&............|

hexdump -C WC-s01        
00000000  03 df cd 9c 00 00 00 09  00 00 00 00 02 c3 02 64  |...............d|
...
000000b0  00 00 00 00 00 00 00 02  84 28 ff ff 00 00 00 00  |.........(......|
000000c0  03 e3 a5 70 03 df cd 8c  00 00 00 00 03 df cd 64  |...p...........d|

hexdump -C Stationery 
00000000  00 12 d2 e8 00 00 00 02  00 00 00 00 01 17 01 ec  |................|
...
000000b0  00 00 00 00 00 00 00 02  84 20 ff ff 00 00 00 00  |......... ......|
000000c0  00 17 56 f8 00 12 cd f8  00 00 00 00 00 12 ce 40  |..V............@|

The only bytes I could find near the beginning that seemed semi consistent is the highlighted bytes above. I did however notice some consistent bytes at the end of each of the files.

hexdump -C "Sample Report" | tail                                                      
00007250  e5 00 02 e5 00 02 e5 00  02 e5 00 02 e5 00 02 e5  |................|
00007260  00 02 e5 00 02 e5 00 02  e5 00 02 e5 00 ff 00 07  |................|
00007270  00 00 00 05 04 31 2e 30  30 00 09 00 00 00 05 04  |.....1.00.......|
00007280  31 2e 30 30 00 08 00 00  00 05 04 31 2e 30 30 00  |1.00.......1.00.|
00007290  0a 00 00 00 05 04 31 2e  30 30 00 0b 00 00 00 02  |......1.00......|
000072a0  00 00 00 0c 00 00 00 10  00 00 00 00 00 00 00 00  |................|
000072b0  00 00 00 01 00 00 00 01  00 11 00 00 00 08 00 2b  |...............+|
000072c0  00 03 01 52 01 fd 00 13  00 00 00 02 00 00 7f ff  |...R............|
000072d0  00 00 00 00 00 00 72 dc  7f ff ff ff              |......r.....|

hexdump -C WC-s01 | tail                                                              
000003c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000003d0  01 00 00 80 0c 00 08 00  05 00 00 00 00 01 d2 03  |................|
000003e0  ee dc 3e 00 00 00 00 00  07 00 00 00 01 00 00 09  |..>.............|
000003f0  00 00 00 01 00 00 08 00  00 00 01 00 00 0a 00 00  |................|
00000400  00 01 00 00 0b 00 00 00  02 00 00 00 0c 00 00 00  |................|
00000410  10 00 00 00 00 00 00 00  00 00 00 00 01 00 00 00  |................|
00000420  01 00 11 00 00 00 08 00  2b 00 c7 02 fd 03 3a 00  |........+.....:.|
00000430  13 00 00 00 02 00 00 7f  ff 00 00 00 00 00 00 04  |................|
00000440  45 7f ff ff ff                                    |E....|

hexdump -C Stationery | tail
000039a0  00 02 e3 00 02 e3 00 02  e3 00 02 e3 00 02 e3 00  |................|
000039b0  02 e3 00 02 e3 00 02 e3  00 02 e3 00 02 e3 00 ff  |................|
000039c0  00 07 00 00 00 05 04 31  2e 30 30 00 09 00 00 00  |.......1.00.....|
000039d0  05 04 31 2e 30 30 00 08  00 00 00 05 04 31 2e 30  |..1.00.......1.0|
000039e0  30 00 0a 00 00 00 05 04  31 2e 30 30 00 0b 00 00  |0.......1.00....|
000039f0  00 02 00 00 00 0c 00 00  00 10 00 00 00 00 00 00  |................|
00003a00  00 00 00 00 00 01 00 00  00 01 00 11 00 00 00 08  |................|
00003a10  00 2b 00 03 01 51 01 fe  00 13 00 00 00 02 00 00  |.+...Q..........|
00003a20  7f ff 00 00 00 00 00 00  3a 2e 7f ff ff ff        |........:.....|

The four bytes at the end of each file by themselves would not be a good signature as there are many formats which end with a few “FF” sequences. But maybe combined with bytes near the beginning, a signature might be found. I added a couple samples to my Github page if you would like to take a look. In order to retain the extended attributes, I encoded the files as MacBinary.

lsar -L "Sample Report.bin"
Sample Report.bin: MacBinary
Sample Report: 
  Name:                    Sample Report
  Size:                    29.4 KB (29,404 bytes)
  Compressed size:         29.4 KB (29,440 bytes)
  Last modified:           Thursday, July 25, 1991 at 12:58:20 PM
  Created:                 Saturday, October 13, 1990 at 1:10:54 AM
  Mac OS type code:        ?WP1 (0x0a575031)
  Mac OS creator code:     ??WP (0x0a1a5750)
  Mac OS Finder flags:     0x0100
  Index in file:           0
  Length of embedded data: 29404
  Start of embedded data:  128
  Original archive entry:  Is an embedded MacBinary file: Yes

Apple Package Format

Let’s talk about Apple’s iWork software. Apple’s Office Suite of applications was first released in 2005 and provided a WordProcessor (Pages), Presentations (Keynote), and a little later, Spreadsheet (Numbers). They are exclusive to the Macintosh and iOS devices.

iWork was released in a few different versions. They get a little confusing as each application has its own version which all seemed to unify and stabilize in 2020. Here is a matrix of major versions.

VersionPackage or ZIP
iWork ’05Package
iWork ’06Package
iWork ’08Package
iWork ’09ZIP
iWork 2013Package
iWork 2014ZIP
iWork 2019ZIP
iWork 2020ZIP

You may already be aware but MacOS can sometimes be weird. I use the term weird in a loving, sometimes proud way, but I admit, there was some “odd” choices made in regards to how applications and documents are used and stored files on a Mac.

On early Macintosh computers Apple used an interesting method of storing resources for applications and some file formats. The Resource Fork for an application contained all the “resources” needed to run in the operating system. It would contain all the icons, warning screens, graphics, sounds, etc. This help true until Mac OS X came along and then Apple started using a bundle or package format. Still in use today, what appears to be a single file or application is actually a folder of all the resources needed to run the application.

Show Package Contents

By right clicking or control clicking on the icon you can open the folder and see all the contents which make up the Application.

Directory listing of Pages.app on MacOS

Nifty right? The MacOS which knows which extensions to treat as a package. If you were to move the application over to another system it would be a folder with the extension “.app”.

For an application I can see how this makes sense as it will only execute in the MacOS environment. The problem comes in when you use the same package method for the documents the application creates.

Contents of Pages version 1 sample file.

So instead of a single “file” with a bytestream, you get a folder of files which make up the file format. Here is Apple’s description:

Document Packages

If your document file formats are getting too complex to manage because of several disparate types of data, you might consider adopting a package format for your documents. Document packages give the illusion of a single document to users but provide you with flexibility in how you store the document data internally. Especially if you use several different types of standard data formats, such as JPEG, GIF, or XML, document packages make accessing and managing that data much easier.

Apple actually defines two similar methods:

Although bundles and packages are sometimes referred to interchangeably, they actually represent very distinct concepts:

  • package is any directory that the Finder presents to the user as if it were a single file.
  • bundle is a directory with a standardized hierarchical structure that holds executable code and the resources used by that code.

A couple years ago a processed digital collection made its way down to me. It had been processed by a new digital archivist and when I went to prepare the collection for preservation, I found a folder with the extension .pages and inside the folder a whole directory of files. Many of which they had renamed and arranged. Needless to say, I had to track down the original disk so I could properly preserve the file.

So looking back at the earlier table, iWork switched back and forth between the package format and a ZIP container. For preservation purposes, the ZIP container is easier to maintain outside the MacOS. Lets look a little closer at each. If you would like to follow along I have copied a few samples onto a hybrid ISO.

iWork ’05 through iWork ’08 used the same package format and structure. Because they are a package format, they are difficult to preserve as original files. I suppose you could zip them up, but probably the best option is to open with a current version of Pages and save to the newer ZIP container format.

tree iWork08/Keynote-06.key 
├── Contents
│   └── PkgInfo
├── QuickLook
│   └── Thumbnail.jpg
├── index.apxl.gz
└── theme-files
    ├── Blue 2.jpg
    ├── Blue 2.tif
    ├── Cool Gray-2.jpg
    ├── Cool Gray.tif
    ├── Green-8.jpg
    ├── Green.tif
    ├── Headlines_bullet.pdf
    ├── Headlines_star.pdf
    ├── Orange 2.tif
    ├── Orange_2.jpg
    ├── Purple-6.jpg
    ├── Purple.tif
    ├── Red.jpg
    ├── Red.tif
    ├── endpoints.pdf
    └── headlines_hi-res.jpg

iWork ’09 changed this practice. The documents saved from Pages, Keynote, and Numbers were contained in a ZIP file and can be identified using the PRONOM registry container signatures.

filename : 'iWork 2013/Pages2013-Sample09.pages'
filesize : 105900
modified : 2019-11-21T20:36:00-07:00
matches  :
  - ns      : 'pronom'
    id      : 'fmt/1439'
    format  : 'Apple iWork Pages'
    version : '09'
    class   : 'Word Processor'
    basis   : 'extension match pages; container name index.xml with byte match at 195, 76' 
Sample09.pages
Type = zip
WARNINGS:
Headers Error
Physical Size = 105900

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-11-21 20:36:00 .....       364773        22413  index.xml
2019-11-21 20:36:00 .....         7007         7007  Hardcover_bullet_black.png
2019-11-21 20:36:00 .....        69176        69176  Simple_Noise_2x.jpg
2019-11-21 20:36:00 .....          232          232  buildVersionHistory.plist
2019-11-21 20:36:00 .....         6406         6406  QuickLook/Thumbnail.png
------------------- ----- ------------ ------------  ------------------------
2019-11-21 20:36:00             447594       105234  5 files

Then Apple went back to a Package format with iWork 2013. For reasons unknown. But the content and structure changed. Its a package format with a Index.zip instead of index.xml

Pages2013-Sample.pages
├── Data
│   └── Hardcover_bullet_black-13.png
├── Index.zip
├── Metadata
│   ├── BuildVersionHistory.plist
│   ├── DocumentIdentifier
│   └── Properties.plist
├── preview-micro.jpg
├── preview-web.jpg
└── preview.jpg

3 directories, 8 files

The ZIP within the package contains a new Apple format. IWA

Pages2013-Sample.pages/Index.zip
Type = zip
Physical Size = 39361

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-11-21 20:47:14 .....         3860         3860  Index/Document.iwa
2019-11-21 20:47:14 .....           26           26  Index/Tables/DataList.iwa
2019-11-21 20:47:14 .....          336          336  Index/ViewState.iwa
2019-11-21 20:47:14 .....          160          160  Index/CalculationEngine.iwa
2019-11-21 20:47:14 .....          121          121  Index/DocumentStylesheet.iwa
2019-11-21 20:47:14 .....        31931        31931  Index/ThemeStylesheet.iwa
2019-11-21 20:47:14 .....           22           22  Index/AnnotationAuthorStorage.iwa
2019-11-21 20:47:14 .....         1889         1889  Index/Metadata.iwa
------------------- ----- ------------ ------------  ------------------------
2019-11-21 20:47:14              38345        38345  8 files

Luckily Apple came to their senses and went back to the ZIP container format for iWork 2014 and later. The container signature looks for the IWA file Apple started using with iWork 2013.

filename : 'iWork 2014/Pages2014-Sample.pages'
filesize : 66256
modified : 2019-11-22T00:03:56-07:00
errors   : 
matches  :
  - ns      : 'pronom'
    id      : 'fmt/1441'
    format  : 'Apple iWork Document'
    version : '14'
    class   : 'Presentation, Spreadsheet, Word Processor'
    basis   : 'extension match pages; container name Index/Document.iwa with byte match at 16, 6; name Metadata/Properties.plist with name only'
Path = iWork 2014/Pages2014-Sample.pages
Type = zip
Physical Size = 66256

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-11-22 00:03:54 .....         3930         3930  Index/Document.iwa
2019-11-22 00:03:54 .....          364          364  Index/ViewState.iwa
2019-11-22 00:03:54 .....          206          206  Index/CalculationEngine.iwa
2019-11-22 00:03:54 .....        33573        33573  Index/DocumentStylesheet.iwa
2019-11-22 00:03:54 .....           22           22  Index/AnnotationAuthorStorage.iwa
2019-11-22 00:03:54 .....           23           23  Index/DocumentMetadata.iwa
2019-11-22 00:03:54 .....         8761         8761  Index/Metadata.iwa
2019-11-22 00:03:54 .....          322          322  Metadata/Properties.plist
2019-11-22 00:03:54 .....           36           36  Metadata/DocumentIdentifier
2019-11-22 00:03:54 .....          273          273  Metadata/BuildVersionHistory.plist
2019-11-22 00:03:54 .....        14611        14611  preview.jpg
2019-11-22 00:03:54 .....          838          838  preview-micro.jpg
2019-11-22 00:03:54 .....         1571         1571  preview-web.jpg
------------------- ----- ------------ ------------  ------------------------
2019-11-22 00:03:54              64530        64530  13 files

Now iWork was not the only Apple software to use the Package/Bundle format for their documents. Be advised the following software may save to the package format.

I remember a few years ago, Trent Reznor (NIN) decided to release a few of his tracks in the Garageband format. A little harder to find these days, but the good old wayback machine kept a copy for us! Grab them here. Be warned, they may be in the package format. Thanks Apple!