SDIF

Posted on by Thor Leave a comment

I have used and have researched a lot of audio editing software. Some are very simple and straightforward, others are feature rich and take some time to learn. While looking in a format, I came across some Audio software which nothing like I have used before. At first I was confused, I figured it would be simple to open a certain file format and play the audio. Not so fast.

Max is software which proudly says it is an, “infinitely flexible space to create your own interactive software”. Created by Cycling ’74 software, Max has been around for awhile, being developed in the mid 1980’s. It allows the user to make “patches” stringing around components and effects to accomplish an infinite amount of options and outcomes.

The software produces simple project files and patch files, but hey are just JSON data, at least in the latest version. But when working with audio files the software can save to a number of formats.

One of the options is a format called “SDIF”, which stands for “Sound Description Interchange Format“. SDIF was jointly developed by IRCAM and CNMAT, with proposals starting back in the mid-1990’s. Originally written as a Spectral Description, it was later changed to refer to a Sound Description.

The Specification states the general idea was to “store information related to signal processing and specifically of sound, in files, according to a common format to all data types. Thus, it is possible to store results or parameters of analyses, syntheses…” So not exactly the same as a simple WAVE file you can open and edit, this format was meant to store signal data for analysis.

Each SDIF file consists of a header and then an overall a succession of frames, not unlike chunks in the IFF/AIFF/RIFF formats, ordered in time. Each frame matrix declares a “Type” which can be a combination of many options. Lets take a look at a SDIF file:

hexdump -C test.sdif | head
00000000  53 44 49 46 00 00 00 08  00 00 00 03 00 00 00 01  |SDIF............|
00000010  31 54 52 43 00 00 00 20  00 00 00 00 00 00 00 00  |1TRC... ........|
00000020  00 00 00 01 00 00 00 01  31 54 52 43 00 00 00 04  |........1TRC....|
00000030  00 00 00 00 00 00 00 04  31 54 52 43 00 00 00 c0  |........1TRC....|
00000040  3f 74 7a e1 40 00 00 00  00 00 00 01 00 00 00 01  |?tz.@...........|
00000050  31 54 52 43 00 00 00 04  00 00 00 0a 00 00 00 04  |1TRC............|
00000060  3f 80 00 00 45 95 35 c3  00 00 00 00 00 00 00 00  |?...E.5.........|
00000070  40 00 00 00 46 06 e2 14  00 00 00 00 00 00 00 00  |@...F...........|
00000080  40 40 00 00 45 3b 42 3d  00 00 00 00 00 00 00 00  |@@..E;B=........|
00000090  40 80 00 00 43 5d 94 7b  00 00 00 00 00 00 00 00  |@...C].{........|

This test file has the opening frame “SDIF“, to identify it as an SDIF, then a reference to the type “1TRC“. I would try and explain a Matrix 1TRC Sinusoidal Track, but I have no idea what it means. Something, something sine wave, etc. Someone much smarter than me can make use of this format. Here are a couple examples of SDIF with other frame types.

hexdump -C angry_cat.part.sdif| head
00000000  53 44 49 46 00 00 00 08  00 00 00 03 00 00 00 01  |SDIF............|
00000010  31 4e 56 54 00 00 00 88  ff ef ff ff ff ff ff ff  |1NVT............|
00000020  ff ff ff fd 00 00 00 01  31 4e 56 54 00 00 03 01  |........1NVT....|
00000030  00 00 00 61 00 00 00 01  53 74 72 65 61 6d 49 44  |...a....StreamID|
00000040  09 30 0a 44 61 74 65 09  54 68 75 5f 41 75 67 5f  |.0.Date.Thu_Aug_|
00000050  5f 33 5f 32 31 2e 33 32  2e 34 35 5f 32 30 30 30  |_3_21.32.45_2000|
00000060  5f 0a 54 61 62 6c 65 4e  61 6d 65 09 53 69 6e 75  |_.TableName.Sinu|
00000070  73 6f 69 64 61 6c 54 72  61 63 6b 73 0a 57 72 69  |soidalTracks.Wri|
00000080  74 74 65 6e 42 79 09 50  6d 5f 56 65 72 73 69 6f  |ttenBy.Pm_Versio|
00000090  6e 5f 31 2e 32 2e 32 0a  00 00 00 00 00 00 00 00  |n_1.2.2.........|

hexdump -C cymbalum-c4.res.sdif| head 
00000000  53 44 49 46 00 00 00 08  00 00 00 03 00 00 00 01  |SDIF............|
00000010  31 52 45 53 00 00 0d 20  00 00 00 00 00 00 00 00  |1RES... ........|
00000020  00 00 00 04 00 00 00 01  31 52 45 53 00 00 00 04  |........1RES....|
00000030  00 00 00 d0 00 00 00 04  42 49 27 7a 39 59 fc ab  |........BI'z9Y..|
00000040  3d 35 06 c9 00 00 00 00  42 6e 68 68 39 63 99 b1  |=5......Bnhh9c..|
00000050  3e 25 f7 c0 00 00 00 00  42 c6 02 bb 39 8c 31 79  |>%......B...9.1y|
00000060  3f bb 7e 6e 00 00 00 00  43 01 82 96 3a 1d 36 44  |?.~n....C...:.6D|
00000070  3e d9 21 12 00 00 00 00  43 07 35 f0 3a 20 6f 6e  |>.!.....C.5.: on|
00000080  3f 02 32 7f 00 00 00 00  43 30 84 0b 39 97 f9 1b  |?.2.....C0..9...|
00000090  3e c6 43 c7 00 00 00 00  43 4d e4 e4 39 88 14 90  |>.C.....CM..9...|

Unfortunately, the common tools I use to explore AV formats don’t seem to work on this format. MediaInfo, FFProbe, Exiftool, all give me unknown file warnings. So I had to compile the SDIF software in order to get some details.

querysdif angry_cat.part.sdif 
Header info of file angry_cat.part.sdif:

Format version: 3
Types version:  1

Ascii chunks of file angry_cat.part.sdif:

1NVT
{
StreamID	0;
Date	Thu_Aug__3_21.32.45_2000_;
TableName	SinusoidalTracks;
WrittenBy	Pm_Version_1.2.2;
}

Data in file angry_cat.part.sdif (9504872 bytes):
 1933 1TRC frames in stream 0 between time 0.000000 and 5.794875 containing
    1933 1TRC matrices with  45 --400 rows,   4 --  4 columns

An interesting thing is that a SDIF file can be in text form as well.

sdiftotext test.sdif 
SDIF


SDFC

1TRC	1	1	0
  1TRC	0x0004	0	4

1TRC	1	1	0.005
  1TRC	0x0004	10	4
	1	4774.72	0	0
	2	8632.52	0	0
	3	2996.14	0	0
	4	221.58	0	0
	5	1943.02	0	0
	6	123.951	0	0
	7	6705.04	0	0
	8	4304.97	0	0
	9	3554.29	0	0
	10	23.7822	0	0

1TRC	1	1	0.01
  1TRC	0x0004	10	4
	1	4774.72	0.0353114	2.06098
	2	8632.52	0.00442518	0.68795
	3	2996.14	0.0238517	-1.42295
	4	221.58	0.0089712	-2.44141
	5	1943.02	0.00768914	2.64629
	6	123.951	0.0397061	-0.17527
	7	6705.04	0.0245643	-0.168753
	8	4304.97	0.00894803	1.45553
	9	3554.29	0.0265175	2.57231
	10	23.7822	0.0419019	-2.17731

1TRC	1	1	0.2
  1TRC	0x0004	10	4
	1	2284.56	0.02781	2.47054
	2	4222.62	0.0151738	1.55309
	3	31.1554	0.00421461	-0.657285
	4	310.99	0.0122306	1.25794
	5	215.192	0.0174093	1.25468
	6	6253.69	0.000894192	2.21334
	7	8533.32	0.0296167	2.07209
	8	8044.77	0.0423002	2.54088
	9	6087.45	0.0264733	-2.05523
	10	7052.7	0.0287347	0.426339

1TRC	1	1	0.205
  1TRC	0x0004	10	4
	1	2284.56	0	0
	2	4222.62	0	0
	3	31.1554	0	0
	4	310.99	0	0
	5	215.192	0	0
	6	6253.69	0	0
	7	8533.32	0	0
	8	8044.77	0	0
	9	6087.45	0	0
	10	7052.7	0	0

1TRC	1	1	0.21
  1TRC	0x0004	0	4

ENDC
ENDF

An interesting format for sure. But wait, there is more!

My initial interest in this format was when I was given access to a set of MUBU files. I was unclear on how there were created at first and it took me down a long path of learning about SDIF and the Max software from Cycling ’74 and IRCAM. MUBU turns out to be a toolbox for Max which adds more analysis features.

MUBU stands for MUlti-BUffer, which helps overcome some limitations. It is actually a container using the SDIF standard. Lets take a look.

hexdump -C test.mubu | head
00000000  53 44 49 46 00 00 00 08  00 00 00 03 00 00 00 01  |SDIF............|
00000010  31 4e 56 54 00 00 00 78  ff ef ff ff ff ff ff ff  |1NVT...x........|
00000020  ff ff ff fd 00 00 00 01  31 4e 56 54 00 00 03 01  |........1NVT....|
00000030  00 00 00 53 00 00 00 01  4d 75 42 75 2e 43 6f 6e  |...S....MuBu.Con|
00000040  74 61 69 6e 65 72 2e 4e  75 6d 54 72 61 63 6b 73  |tainer.NumTracks|
00000050  09 31 0a 4d 75 42 75 2e  43 6f 6e 74 61 69 6e 65  |.1.MuBu.Containe|
00000060  72 2e 56 65 72 73 69 6f  6e 09 31 2e 35 0a 4d 75  |r.Version.1.5.Mu|
00000070  42 75 2e 43 6f 6e 74 61  69 6e 65 72 2e 4e 75 6d  |Bu.Container.Num|
00000080  42 75 66 66 65 72 73 09  31 0a 00 00 00 00 00 00  |Buffers.1.......|
00000090  31 4e 56 54 00 00 00 38  ff ef ff ff ff ff ff ff  |1NVT...8........|

A MUBU file has the same SDIF frame header, but also include a “1NVT” frame, which is a Name Value Table. This is where the MUBU container is referenced. The MuBu file has its own structure:

If I query the MuBu file like I did the SDIF, I get the following:

querysdif test.mubu
Header info of file test.mubu:

Format version: 3
Types version:  1

Ascii chunks of file test.mubu:

1NVT
{
MuBu.Container.NumTracks	1;
MuBu.Container.Version	1.5;
MuBu.Container.NumBuffers	1;
}
1NVT
{
MuBu.Buffer.Index	0;
}
1NVT
{
MuBu.Track.MxRows	2;
AudioFile	1;
MuBu.Track.NonNumType	0;
MuBu.Track.MaxSize	93515;
meta_ISFT	Lavf60.16.100;
MuBu.Track.Name	mytrack;
MuBu.Track.BufferIndex	0;
MuBu.Track.SampleRate	48000;
FileName	Wilhelm_Scream.wav;
MuBu.Track.MxVarRows	0;
MuBu.Track.MxCols	1;
meta_MetaDataSource	WAV;
MuBu.Track.EndTime	1623.5;
FilePath	/;
MuBu.Track.SampleOffset	0;
MuBu.Track.TimeTags	0;
MuBu.Track.Size	77929;
MuBu.Track.Index	0;
}

1TYP
{
  1MTD	M000	{unnamed}
  1FTD	M000
	{
	  M000	Track-0-MatrixData;
	}
}

Data in file test.mubu (3741392 bytes):
77929 M000 frames in stream 0 between time 0.000000 and 1.623500 containing
   77929 M000 matrices with   2 --  2 rows,   1 --  1 columns

The MuBu file contains one audio track and one buffer. This is a simple test file, but MuBu files can be quite large with multiple tracks.

Working with the Max software or OpenMusic is not something I found to be easy to understand. I am sure if I was more musically inclined and with a little practice I could make some of this work. For the time being, a signature to identify a SDIF and MUBU will have to do. Check out the GitHub for my proposed signature and a couple examples.

PROmotion

Posted on by Thor Leave a comment

The 1990’s was an amazing time for multimedia. Compared to what is possible today, the graphics were more simple but there were many software titles leading the charge in Animation. Macromedia Director, along with Flash, dominated the interactive multimedia market for quite some time. Eventually being picked up by Adobe and discontinued in 2013. Quite a few multimedia disc’s out there were built using Director.

Competing with Director, another company had a strong product. Motion Works International was an early pioneer in the multimedia CD-ROM scene. Rumor has it, Motion Works was started by a 12 year old. Motion Works had been making software for use with the highly successful HyperCard software since 1988. In 1992 they released the successor to their ADDmotion software, a path based animation tool called PROmotion.

PROmotion was used with with many Multimedia titles, some in cooperation with the Corel Home series. In addition to commercial titles PROmotion was a great tool for the creation of animation clips and other marketing material. I came across some stand-alone marketing files for old scriptwriting software called ScriptWare. When I unarchived the HQX file and Installed the Demo, I was presented with a set of files with the .MW extension.

ls -l@
total 10232
-rw-r--r--@ 1 tyler  staff  1392 May  1 23:17 Read me first!
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	 452 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 begin_here.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	158901 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 characters.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	387029 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 cinovation.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	189509 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 cut paste.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	608405 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 formats.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	289698 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 modify formats.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	486730 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 notes.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	319250 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 overview.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	376854 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 scene shuffle.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	359746 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 script elements.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	279052 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 sw_menu.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	421836 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 title page.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	236614 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 transitions.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	471462 
-rw-r--r--@ 1 tyler  staff     0 May  1 23:17 try it.MW
	com.apple.FinderInfo	  32 
	com.apple.ResourceFork	622312 

getfileinfo sw_menu.MW 
file: "sw_menu.MW"
type: "APPL"
creator: "AMvw"

Looking at the files in the directory with their extended attributes I can see all the .MW files have no data fork (0 bytes), only a resource fork. This is common for any Application on the MacOS systems prior to MacOS X. At first the MW extension made me thing of MacWrite, but launching one of these MW files brought up an interactive menu. The type being APPL, which is Application.

What I thought would be a demo of the application Scriptware was actually interactive animations demonstrating the software. By dumping the resource fork of one of the MW files I found some information which helped me know what software created these interactive demos.

derez Scriptware\ Demo\ folder/sw_menu.MW

data 'vers' (1) {
	$"0103 8000 0000 0531 2E30 2E33 2941 4D20"            /* ..?....1.0.3)AM  */
	$"5669 6577 6572 2031 2E30 2E33 0DA9 2031"            /* Viewer 1.0.3.? 1 */
	$"3939 3320 4D6F 7469 6F6E 2057 6F72 6B73"            /* 993 Motion Works */
	$"2049 6E74 6C2E"                                     /*  Intl. */
};

data 'vers' (2) {
	$"0103 8000 0000 0531 2E30 2E33 1E50 6C61"            /* ..?....1.0.3.Pla */
	$"7962 6163 6B20 6279 204D 6F74 696F 6E20"            /* yback by Motion  */
	$"576F 726B 7320 496E 746C 2E"                        /* Works Intl. */
};

data 'STR#' (1250, "ADDmotion HC strings") {
	$"000A 1641 4444 6D6F 7469 6F6E 5F65 7870"            /* ...ADDmotion_exp */
	$"6F72 745F 6672 616D 650E 4144 446D 6F74"            /* ort_frame.ADDmot */
	$"696F 6E5F 696E 666F 1141 4444 6D6F 7469"            /* ion_info.ADDmoti */
	$"6F6E 5F73 7573 7065 6E64 1041 4444 6D6F"            /* on_suspend.ADDmo */
	$"7469 6F6E 5F72 6573 756D 650E 4144 446D"            /* tion_resume.ADDm */
	$"6F74 696F 6E5F 7175 6974 0E41 4444 6D6F"            /* otion_quit.ADDmo */
	$"7469 6F6E 5F70 6C61 790E 4144 446D 6F74"            /* tion_play.ADDmot */
	$"696F 6E5F 7374 6F70 0F41 4444 6D6F 7469"            /* ion_stop.ADDmoti */
	$"6F6E 5F70 6175 7365 0000"                           /* on_pause.. */
};

Makes sense, MW stood for “Motion Works”. ADDmotion was another software title developed by Motion Works, most will remember it as an add-on for Hypercard for adding animation to stacks. These MW files are created using PROmotion and exporting them as a stand-alone animation which includes the “AM Viewer” built in. A regular PROmotion file, however, did not include a viewer and requires the software in order to open and run. 

-rwx------@ 1 tyler  staff      0 Apr 25 15:51 Example Animation
	com.apple.FinderInfo	   32 
	com.apple.ResourceFork	495272

The PROmotion file format also is Resource Fork only, making them difficult to manage outside of a Macintosh. 

getfileinfo Example\ Animation
file: "Example Animation"
type: "ADDm"
creator: "ADDm"

The files do have a Type/Creator code of “ADDm”, but with no data fork, identification through standard means is not possible. They also do not have the “vers” string to help identify them within the Resource Fork. Since standard methods of identification are impossible, I hope in the future there will be more tools available to read the Type/Creator codes while on the Mac, or in a disk image, or within a container and return back the Software which created the file and the file type.

The products from Motion Works where significantly cheaper than animation tools such as Director, but were still pretty powerful for its day. I was surprised when I found the company didn’t last much longer than 1998 before disappearing. There are probably many stories like PROmotion, coming onto the scene with new and exciting features before thought impossible only to die out as other tools dominate the market.

If you are interested in looking at the files yourself, here is a link to some original files, and the same files encoded in MacBinary.

Scheduling EXport

Posted on by Thor Leave a comment

During a recent review of some help files for some older Final Draft software I came across this Q&A.

Needless to say, I was intrigued, but let me give you a pro tip. Googling MovieMagic and “SEX” does not bring back results related to file formats. Also, probably best not to search at work.

Movie Magic refers to software developed by Write Brothers/Screenplay. The main software, Screenwriter is a word processor built specifically for writing screen plays for TV, Movies, theater, etc. The software was first developed in 1983 and quickly replaced typewriters as the favorite for writing the very specific formatting required by screenplays.

Screenwriter version 6 uses the extension MMSW and DEF for templates. Let’s have a look at one under the hood.

hexdump -C Screenwriterv6-01.mmsw | head
00000000  53 63 72 65 65 6e 77 72  69 74 65 72 57 69 6e 56  |ScreenwriterWinV|
00000010  65 72 2e 20 36 2e 30 30  20 00 00 00 00 00 00 00  |er. 6.00 .......|
00000020  00 00 00 0c 4e 4f 4e 41  4d 45 32 2e 4d 4d 53 57  |....NONAME2.MMSW|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000050  00 00 00 00 00 00 00 00  0f 0a 0a 00 00 00 00 0a  |................|
00000060  01 0c 00 00 0f 0a 0a 01  00 00 01 0a 01 0c 00 00  |................|
00000070  19 18 00 00 00 00 04 0a  01 0c 00 00 25 0a 0a 01  |............%...|
00000080  00 00 03 0a 01 0c 00 00  0f 0a 0a 01 00 00 06 0a  |................|
00000090  01 0c 00 00 1e 23 00 00  00 00 05 0a 01 0c 00 00  |.....#..........|

The header is easy to interpret, Version 6 is the latest version of the software. The rest of the file is non-human readable binary so not much to look at. The screenplay website has a chart for figuring out compatibility for all the versions and extensions.

The other major version used the SCW extension. 

hexdump -C ScreenWriter4-s01.scw | head
00000000  53 63 72 65 65 6e 77 72  69 74 65 72 77 69 6e 76  |Screenwriterwinv|
00000010  65 72 2e 20 34 2e 31 31  61 00 00 00 00 00 00 00  |er. 4.11a.......|
00000020  00 00 00 11 53 63 72 65  65 6e 57 72 69 74 65 72  |....ScreenWriter|
00000030  34 2d 73 30 31 00 00 00  00 00 00 00 00 00 00 00  |4-s01...........|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  0f 0a 0a 00 00 00 00 0a  |................|
00000060  00 0c 00 00 0f 0a 0a 01  00 00 01 0a 00 0c 00 00  |................|
00000070  19 18 00 00 00 00 04 0a  00 0c 00 00 25 0a 0a 01  |............%...|
00000080  00 00 03 0a 00 0c 00 00  0f 0a 0a 01 00 00 06 0a  |................|
00000090  00 0c 00 00 1e 23 00 00  00 00 05 0a 00 0c 00 00  |.....#..........|

hexdump -C Screenwriterv6-01.scw | head
00000000  53 63 72 65 65 6e 77 72  69 74 65 72 57 69 6e 56  |ScreenwriterWinV|
00000010  65 72 2e 20 34 2e 39 30  00 00 00 00 00 00 00 00  |er. 4.90........|
00000020  00 00 00 0c 4e 4f 4e 41  4d 45 32 2e 4d 4d 53 57  |....NONAME2.MMSW|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Files from the earlier version appear to be structured the same way. Although files generated on a Macintosh still seem to retain the “win” in the header, but with a capital “Win” and “Ver”. Looking at the older ScriptThing format it is similar.

hexdump -C NONAME1.SCR | head
00000000  53 63 72 69 70 74 54 68  69 6e 67 20 56 65 72 2e  |ScriptThing Ver.|
00000010  20 32 2e 31 39 00 00 00  00 00 07 4e 4f 4e 41 4d  | 2.19......NONAM|
00000020  45 31 2e 53 43 57 00 00  00 00 00 00 00 00 00 00  |E1.SCW..........|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 07 00 01  |................|
00000050  00 0f 49 0b 00 00 0f 49  0b 01 01 19 3b 01 00 04  |..I....I....;...|
00000060  25 49 0b 01 03 0f 49 0b  01 06 1e 30 01 00 05 0f  |%I....I....0....|
00000070  49 0b 01 02 0f 49 0b 01  0a 0f 49 0b 01 0b 0f 49  |I....I....I....I|
00000080  0b 04 14 00 00 02 02 02  02 02 00 00 01 02 00 37  |...............7|
00000090  0b 28 43 4f 4e 54 49 4e  55 45 44 29 00 0a 43 4f  |.(CONTINUED)..CO|

hexdump -C MULTIMEDIA DEMO.SCW | head
00000000  53 63 72 69 70 74 54 68  69 6e 67 20 57 69 6e 56  |ScriptThing WinV|
00000010  65 72 2e 20 31 2e 32 35  64 00 08 44 49 43 54 46  |er. 1.25d..DICTF|
00000020  49 4c 45 1a 4d 75 6c 74  69 6d 65 64 69 61 20 44  |ILE.Multimedia D|
00000030  65 6d 6f 20 53 63 72 69  70 74 2e 53 43 52 00 00  |emo Script.SCR..|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  0a 0e 0a 00 00 00 00 0a  |................|
00000060  00 0c 00 00 0a 0d 0a 01  00 00 01 0a 00 0c 00 05  |................|
00000070  14 1b 00 00 00 00 04 0a  00 0c 00 00 20 0d 0a 01  |............ ...|
00000080  00 00 03 0a 00 0c 00 00  0c 17 0a 01 00 00 06 0a  |................|
00000090  00 0c 00 01 19 26 00 00  00 00 05 0a 00 0c 00 00  |.....&..........|

Screenwriting software is very specialized, the formatting is important as well as keeping track of characters, locations, props, etc. An important part of filming a movie based on a script is to schedule the different scenes and which characters are needed for each scene. This scheduling can be generated by generating a Scheduling EXport. Not sure who decided on this extension, but there it is. 

hexdump -C Screenwriterv6-01.sex | head
00000000  53 53 49 2a 00 23 00 00  00 e2 00 00 00 00 43 61  |SSI*.#........Ca|
00000010  73 74 20 4d 65 6d 62 65  72 73 00 45 78 74 72 61  |st Members.Extra|
00000020  73 00 53 74 75 6e 74 73  00 56 65 68 69 63 6c 65  |s.Stunts.Vehicle|
00000030  73 00 50 72 6f 70 73 00  53 70 65 63 69 61 6c 20  |s.Props.Special |
00000040  45 66 66 65 63 74 73 00  43 6f 73 74 75 6d 65 73  |Effects.Costumes|
00000050  00 4d 61 6b 65 75 70 00  4c 69 76 65 73 74 6f 63  |.Makeup.Livestoc|
00000060  6b 00 41 6e 69 6d 61 6c  20 48 61 6e 64 6c 65 72  |k.Animal Handler|
00000070  00 4d 75 73 69 63 00 53  6f 75 6e 64 00 53 65 74  |.Music.Sound.Set|
00000080  20 44 72 65 73 73 69 6e  67 00 47 72 65 65 6e 65  | Dressing.Greene|
00000090  72 79 00 53 70 65 63 69  61 6c 20 45 71 75 69 70  |ry.Special Equip|

Scheduling Export files begin with “SSI“, which I assume refers to Screenplay Systems, Inc. They begin with listing all the different things which need scheduling in plain text. In Screenwriter 6 there are a couple of export types with the same extension. One called Gorilla Scheduling and another called CompanyMOVE ShowPlanner, but both are identical to the Movie Magic file, so I am not quite sure their purpose. Maybe there would be more to discern from a whole script instead of the simple samples I made for this purpose.

This was a fun format to research, although I had to be careful of the terms I used! You can check out my proposed signatures and samples on my GitHub.

Sibelius

Posted on by Thor Leave a comment

Music notation software is among the earliest software for desktop computers. SCORE in 1987, Finale came around in 1988, Capella in 1992, and Sibelius in 1993. Many others came and went during this time. Music notation software was so much more than the typical word processing or desktop publishing system. Specialized fonts were needed to display the music notation and there are many other variables for different instruments allowing individuals and others the ability to create complicated compositions in an inexpensive way.

Sibelius [SI] + [BAY] + [LEE] + [UHS] was originally developed for the Acorn system in 1986, then released on Windows and Macintosh in 1998-99. The software became very popular and in 2006 was purchased by the software giant AVID. The software was used enough to get a preservation assessment by the British Library in 2017 and draft status format description by the Library of Congress, written by the amazing Ashley!

Both reviews of the format emphasize the proprietary nature of the file format which has been used since the early versions. Aside from the early Acorn release, the Windows and Macintosh versions used a binary format with the SIB extension. They are actually quite easy to identify.

hexdump -C Sibelius-s01.sib | head
00000000  0f 53 49 42 45 4c 49 55  53 00 00 40 00 02 00 a4  |.SIBELIUS..@....|
00000010  f1 ed 00 00 00 30 00 00  00 02 00 00 00 01 00 00  |.....0..........|
00000020  00 2a 00 00 00 00 00 00  00 00 0f 53 49 42 45 4c  |.*.........SIBEL|
00000030  49 55 53 00 00 40 00 02  00 00 00 00 00 00 00 3a  |IUS..@.........:|
00000040  00 00 00 00 38 a1 28 06  b3 d2 2f 66 03 04 16 4e  |....8.(.../f...N|
00000050  5f 5c 8d f3 95 27 3e f1  2a 1b 68 de 08 81 e8 9a  |_\...'>.*.h.....|
00000060  ea 1c bf dd 54 0e 92 8d  4d be e3 34 ed 42 78 36  |....T...M..4.Bx6|
00000070  d2 e1 67 7b 8d f7 98 6a  3a 70 c4 8b 0b 08 7b 26  |..g{...j:p....{&|
00000080  f9 45 00 00 00 00 48 71  7c 4c 98 df 0b 38 7d 9d  |.E....Hq|L...8}.|
00000090  2a 2d 84 9c a4 39 0f 4d  da a2 cc 97 ad 3d b0 55  |*-...9.M.....=.U|

This is exactly how PRONOM and other identification methods determine they are Sibelius files. PRONOM has assigned the format fmt/696 and is looking for the hexadecimal bytes 0F534942454C495553.

The problem with this identification method is that all the Sibelius files are identified as such, regardless of version. As mentioned by Ashley, version of the software used is highly important as new features were added all the time making backwards compatibility difficult. Add in the fact that there were different releases for each version which would limit these features even more and I can see how a musician could get very frustrated. If you created a score in Sibelius 5 and tried to open in Sibelius 5 Student version, you may find your composition lacking in many ways. The only way to avoid compatibility issues is to always open in the latest “Ultimate” version. Sibelius Ultimate can open all versions of the SIB format back to version 2. The software even has an export feature which allows you to export back to a previous version stripping what is necessary to ensure compatibility.

Sibelius export to previous version

For those with a bunch of SIB files in their archives, how would you know which software version created the file? Well lets take a closer look at the bytes and see if we can find some patterns. Let it be known, I am not reverse engineering the format, just looking for patterns which will allow for proper identification!

I am not the first person to ask this question, many others want to know the versions of their SIB files. Thankfully others have found some clues on which bytes hold the version information. It seems we can determine the version based on 4 bytes shortly after the SIBELIUS string. Specifically bytes 10-13. 

hexdump -C Sibelius2-s01.sib | head
00000000  0f 53 49 42 45 4c 49 55  53 00 00 08 00 22 00 47  |.SIBELIUS....".G|
00000010  98 4c 00 00 00 3a 00 00  00 00 4e 81 49 34 41 2c  |.L...:....N.I4A,|
00000020  fa 76 62 f9 71 53 a9 93  0f 54 1e 20 6c 63 61 4d  |.vb.qS...T. lcaM|
00000030  f7 b2 b0 a7 5d bd 82 3a  0d 86 02 8b f2 89 d2 a0  |....]..:........|
00000040  83 1f 8d e0 37 1b ed 1c  6a 8b 82 08 4b 6d 64 60  |....7...j...Kmd`|
00000050  71 59 e8 aa ef b1 3c df  5c 25 0a 9f 66 50 69 de  |qY....<.\%..fPi.|
00000060  2a d3 4e 2a cd 97 88 06  67 5f 50 64 0f 8f 86 2b  |*.N*....g_Pd...+|
00000070  08 0d 3f f7 80 26 e0 63  f6 7d 4e f8 e7 c0 3f fc  |..?..&.c.}N...?.|
00000080  7a 77 ea b3 4a b9 30 59  13 47 6e 09 0a 0b ae 3c  |zw..J.0Y.Gn....<|
00000090  c1 93 85 f6 41 f8 58 22  4b 92 35 3f b2 f5 3f 9d  |....A.X"K.5?..?.|

From what others have gathered and updating it with more recent versions I have come up with a list.

VersionHex 10-13
Sibelius 1.200 00 00 0E
Sibelius 2.x00 08 xx xx 
Sibelius 3.x00 0A xx xx 
Sibelius 4.x00 1B xx xx 
Sibelius 5.000 2D 00 03 
Sibelius 5.100 2D 00 0D 
Sibelius 5.2.x – 5.400 2D 00 10 
Sibelius 6.0.x00 36 00 01 
Sibelius 6.100 36 00 17 
Sibelius 6.200 36 00 1E 
Sibelius 7.000 39 00 0C 
Sibelius 7.0.1 – 7.0.200 39 00 0E 
Sibelius 7.0.300 39 00 13 
Sibelius 7.1.000 39 00 15 
Sibelius 7.1.2 – 7.1.300 39 00 16 
Sibelius 7.5.x00 3D 00 0E 
Sibelius 8.0.0 – 8.0.100 3D 00 10 
Sibelius 8.1.x00 3E 00 00 
Sibelius 8.200 3E 00 01 
Sibelius 8.300 3E 00 02 
Sibelius 8.4.x00 3E 00 06 
Sibelius 8.5.x00 3E 00 07 
Sibelius 8.6.x, 8.7.0, 8.7.100 3F 00 00 
Sibelius 8.7.2, 2018.1, 2018.4.x, 2018.5, 2018.6, 2018.700 3F 00 01 
Sibelius 2018.11, 2018.1200 3F 00 02
Sibelius 2019.100 3F 00 04
Sibelius 2019.4.x, 2019.5, 2019.7, 2019.900 3F 00 06
Sibelius 2019.1200 3F 00 07
Sibelius 8.6-2019.1200 3F 00 0A
Sibelius 2020.100 3F 00 0B
Sibelius 2020.3, 2020.600 40 00 01
Sibelius 2020.900 40 00 02
Sibelius 2022.500 40 00 03
Sibelius 2022.1100 41 00 02
Sibelius 2022.1200 42 00 00
Sibelius 2023.300 42 00 01
Sibelius 2023.800 43 00 07
Sibelius 2024.3.100 44 00 01

That is a lot of versions and I feel there may be some gaps that still need to be identified. It appears that the first two bytes are the major version and the second set of bytes is the minor version. Although it looks like a few major version bytes span across a few software versions. With this chart, one could be very specific in identifying which Sibelius version wrote the file, but for archiving purposes it seems we can group many of these capturing just the major version. The export screenshot above seems to have broken down significant changes and grouped similar formats together, the biggest being 8.6 through 2019.12. A comparison of “student” and “first” formats don’t have any obvious bytes which indicate as such, so for now they are all lumped together.

There is one other similar format which needs to be mentioned. Sibelius Scorch was a product made to share scores online. This has been replaced with Sibelius Cloud Publishing, but for awhile was the best way to share a score with others in a way that protected the original. I have no idea how they were made, but sites like scorestreet.net and sibeliusmusic.com were sites you could upload your score to for sharing. Some SCO files appear to have a PDF embedded within them for proper printing.

hexdump -C smd_h_0000000000097761.sco | head
00000000  0f 43 43 53 43 4f 52 43  48 00 00 36 00 1e 00 c0  |.CCSCORCH..6....|
00000010  d4 55 00 00 00 30 00 00  00 01 00 00 00 01 00 00  |.U...0..........|
00000020  00 22 0f 43 43 53 43 4f  52 43 48 00 00 36 00 1e  |.".CCSCORCH..6..|
00000030  00 00 00 00 00 00 00 3a  00 00 00 00 03 56 11 b9  |.......:.....V..|
00000040  70 dc fe 90 50 48 30 df  eb 39 88 23 8e 88 78 bf  |p...PH0..9.#..x.|
00000050  da ab ab 5b e2 13 98 89  66 eb 94 67 8d 16 00 00  |...[....f..g....|
00000060  00 00 cf 6f 0c 67 85 ec  57 90 e5 c1 ea 8a eb 9f  |...o.g..W.......|
00000070  c8 13 d2 1d 75 bd a5 9f  eb b9 ef 1d 25 79 45 2c  |....u.......%yE,|
00000080  05 bb 74 41 e8 8f 27 6a  01 07 d0 f5 3b 17 ce 87  |..tA..'j....;...|
00000090  7b c2 82 d9 41 6b 82 2f  d8 b8 17 32 fa d3 59 05  |{...Ak./...2..Y.|

I am not sure the best way to handle all the different versions within the PRONOM registry. I went ahead and made a few signatures based on the export dialog of Sibelius 2024. Even with combining a few together, it leaves us with 17 new PUID’s. Maybe further discussion can refine these down a bit more? Regardless, each file can be associated with a specific Sibelius version, making it easier to open and migrate if needed without fear of opening in the wrong version. Take a look at some samples and my signatures on my GitHub page and let me know if there is a better way.

Shorten

Posted on by Thor Leave a comment

I was recently going through some of my old CD-R’s and came across this 11 year old fun memory.

I remember going to this 2003 Toad the Wet Sprocket concert in Salt Lake City with some friends, I had seen this band perform before, but this was the first time I was able to get a recording of the show. Normally having a recording of a concert of a well known band was a little shady, but for some bands, they not only allow recording of their live concerts, but they encourage it. There has been a few bands over the years who have this philosophy, one most have heard of is the Grateful Dead, because of all the tape trading, the band’s numerous concerts will live on forever.

The scene of recording concerts is still alive and well, and if you are into recording and sharing it is expected you share in a lossless audio format. The world of lossless audio is definitely in the minority of all those who listen to music on the daily. Most of us have been placated with the infinite playlists on services like Apple Music, Spotify, and Amazon Music. Most probably don’t care about owning music anymore, but for the few who consider themselves Audiophiles, having a lossless audio file is the only choice.

When it comes to formats, there are a few lossless formats to choose from, they all come with some advantages as well as some downsides. WAV files contain the full PCM audio stream, and while internet bandwidth today can handle full uncompressed audio, it can still be beneficial to use some compression for archiving or sharing over the web.

The most common lossless format today is the Free Lossless Audio Codec or FLAC, but there are also quite a few who like the Apple Lossless Audio Codec. Both offer many advantages, especially with metadata, cuesheets, and can contain cover album art. But many years ago another lossless format was most often used with bootleg recordings and audio sharing.

Shorten was one of the first lossless formats, developed by Tony Robinson in 1993 for SoftSound. It could cut the size in half of a typical 16-bit WAV file. It achieved this by using Huffman coding, kinda the same way a JPEG works, by reducing the frequency of how often patterns occur. Today FLAC and ALAC have replaced this format and offer improved features and support. Many audio players have dropped support for shorten making it difficult to use this old format.

The Shorten format uses the .SHN extension. It is one of the formats listed on the Library of Congress Sustainability of Digital Formats with the ID fdd000199, although a couple links don’t appear to work as it hasn’t been updated since 2011. Support was ended for this format and many of the links found on various websites are for broken, usually referencing the etree wiki. Much of which is archived on the Internet Archive.

Let’s take a look at the what makes up a lossless compressed SHN file. A quick look at a sample header:

hexdump -C test.shn | head
00000000  61 6a 6b 67 02 fb b1 70  09 f9 25 59 52 a4 d1 a8  |ajkg...p..%YR...|
00000010  dd cf 85 5a 01 57 a0 d5  a8 b6 6b 6d d2 41 10 80  |...Z.W....km.A..|
00000020  40 20 10 18 04 0a 01 44  d6 40 20 11 0d 8c 0a 01  |@ .....D.@ .....|
00000030  04 80 44 20 16 4b 0d d2  c3 b8 f8 55 a0 11 80 59  |..D .K.....U...Y|
00000040  98 56 1d b1 79 51 9f 39  f1 12 d2 d3 75 5c cd 08  |.V..yQ.9....u\..|
00000050  06 25 68 6b 52 5e 9f 4c  39 cd c1 32 c4 0d a9 b7  |.%hkR^.L9..2....|
00000060  69 34 56 f0 96 fa 46 89  a2 6e 8c ba d5 d0 58 de  |i4V...F..n....X.|
00000070  f5 44 5b aa 61 82 c7 85  88 37 d6 ee cb ab 4e 44  |.D[.a....7....ND|
00000080  91 19 b7 38 d4 20 ae 98  98 d1 2c 4a 4e 88 dd 3e  |...8. ....,JN..>|
00000090  36 68 1b 59 a8 7d 84 23  76 0a 84 21 a1 cd 80 8e  |6h.Y.}.#v..!....|

The first four bytes seem to be consistent among my samples. It makes me wonder if the ascii values have something to do with the author, Anthony (Tony) J. Robinson. In the source code for the shorten software, the file shorten.h defines the ascii “ajkg” as the magic header for the SHN format. Also found in current ffmpeg code. Although the tools don’t have much to say about them.

mediainfo test.shn 
General
Complete name                            : test.shn
Format                                   : Shorten
Format version                           : 2
File size                                : 3.17 MiB

Audio
Format                                   : Shorten
Compression mode                         : Lossless

ffprobe -i test.shn
Input #0, shn, from 'test.shn':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Audio: shorten, 44100 Hz, 2 channels, s16p

Using the older SHNTOOL, we can get more information. 

shntool info test.shn  
-------------------------------------------------------------------------------
File name:                    test.shn
Handled by:                   shn format module
Length:                       0:32.23
WAVE format:                  0x0001 (Microsoft PCM)
Channels:                     2
Bits/sample:                  16
Samples/sec:                  44100
Average bytes/sec:            176400
Rate (calculated):            176400
Block align:                  4
Header size:                  44 bytes
Data size:                    5697720 bytes
Chunk size:                   5697756 bytes
Total size (chunk size + 8):  5697764 bytes
Actual file size:             3325489
File is compressed:           yes
Compression ratio:            0.5836
CD-quality properties:
  CD quality:                 yes
  Cut on sector boundary:     no
  Sector misalignment:        1176 bytes
  Long enough to be burned:   yes
WAVE properties:
  Non-canonical header:       no
  Extra RIFF chunks:          no
Possible problems:
  File contains ID3v2 tag:    no
  Data chunk block-aligned:   yes
  Inconsistent header:        no
  File probably truncated:    unknown
  Junk appended to file:      unknown
  Odd data size has pad byte: n/a
Extra shn-specific info:
  Seekable:                   yes

Many Shorten Audio Files are found out there in archives and file sharing sites, so even though the format isn’t used to create new files, it will still be around for awhile. My GitHub has my signature proposal and a couple of samples.

Canvas

Posted on by Thor Leave a comment

When it comes to design software there were many options over the years, many being released with a lot of hype and others disappearing not long after they released. There are few which lasted long enough to not be gobbled up by big names such as Adobe. One of those is Canvas by Deneba Systems.

First released in 1987, it is still available over at Canvas GFX. It’s amazing it was never bought by one of the big names, Adobe, Corel, Aldus, etc and remained under Deneba Systems until 2003 when it was bought by ACD Systems, but kept the name Deneba Canvas for a time. The later versions were not popular to all, and Mac support was dropped, but the software continued. Awhile back I was looking through a few of my old ZIP disks and found some software my father used in the mid 1980’s. He had a copy of Canvas version 2 for Macintosh. At that time I was more interested in playing games on our family’s Macintosh 128k than using design software.

Over the years I have come across many Canvas documents. With each version released, changes were made to the file format used to store the drawings and artwork. There were many file format changes as well as the extensions used with each version. Some are easily identifiable and others have some confusing structures. Lets look into it.

VersionPlatformExtensionDescription
Canvas 1-3 & artWORKSMacintoshnoneno strong pattern
Canvas 3.5Mac & WindowsCVSSimilar to v1-3
Canvas 5Mac & WindowsCV5CANVAS5 string
Canvas 6-8Mac & WindowsCNVCANVAS6 string
Canvas 9-XMac & WindowsCVXSimilar to 6-8
Canvas DrawMacCVDDifferent than others
Canvas Image FileCVIDAD5PROX

The first three versions of Canvas were Macintosh only and in those early days there was no extension, just a Type / Creator indicating to the Finder how to open them. Deneba Systems used the Creator codes DAD2, DAD5, through DADX.

The first versions are quite frustrating. I have gathered samples from Version 2, 3, 3.5 and artWORKS version 1. Even with numerous samples, there are no patterns I can discern from them. I even reached out to the current CanvasX technical support for answers. They wanted to be helpful, but their answers didn’t offer much help.

With “CVS” or ‘drw2’ for mac, the header contains ranges inside a structure, and other data like if it was compressed. When we see if it’s a valid file we check the ranges. There is no easy way to determine what hex values would be written because of flipping, Intel vs (PPC or 68K). Unfortunately, the research needed to identify the Hex value will require the original code for version 3.5 which we do not have access to easily. Canvas 3.5 code is 16 bit… this would also be an issue.

Let’s take a look at a couple samples:

hexdump -C Canvas2.1-Sample | head
00000000  00 00 03 06 00 00 3d 9c  00 00 00 2a 00 00 00 0a  |......=....*....|
00000010  00 00 00 76 00 00 00 36  00 00 00 2e 00 00 00 1e  |...v...6........|
00000020  00 00 00 12 00 00 00 42  00 00 00 1a 00 00 00 82  |.......B........|
00000030  00 00 00 3c 00 66 00 01  00 00 3d 9c 00 48 00 00  |...<.f....=..H..|
00000040  40 02 90 00 00 00 00 00  00 00 00 00 00 00 00 00  |@...............|
00000050  00 01 00 00 01 00 00 00  00 20 00 40 00 60 00 80  |......... .@.`..|
00000060  00 c0 01 40 01 80 01 c0  02 40 02 80 00 00 00 00  |...@.....@......|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 05  |................|
00000080  00 00 00 00 00 01 00 10  00 00 00 01 00 03 3f fc  |..............?.|
00000090  80 00 00 00 00 00 00 00  00 07 00 01 00 01 00 0b  |................|

hexdump -C Canvas2-s02 | head
00000000  00 00 03 b2 00 00 07 ec  00 00 00 2a 00 00 00 0a  |...........*....|
00000010  00 00 00 76 00 00 00 36  00 00 00 2e 00 00 00 1e  |...v...6........|
00000020  00 00 00 12 00 00 00 42  00 00 00 1a 00 00 00 82  |.......B........|
00000030  00 00 00 3c 00 66 00 01  00 00 07 ec 00 48 00 00  |...<.f.......H..|
00000040  40 02 90 00 00 00 00 00  00 00 00 00 00 00 00 00  |@...............|
00000050  00 01 01 00 01 00 00 00  00 20 00 40 00 60 00 80  |......... .@.`..|
00000060  00 c0 01 40 01 80 01 c0  02 40 02 80 00 00 00 00  |...@.....@......|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 05  |................|
00000080  00 00 00 00 00 01 00 10  00 00 00 01 00 03 3f fc  |..............?.|
00000090  80 00 00 00 00 00 00 00  00 07 00 01 00 01 00 0b  |................|

hexdump -C Canvas3.04 | head
00000000  00 00 02 5a 00 00 00 1c  00 00 00 2a 00 00 00 0a  |...Z.......*....|
00000010  00 00 00 76 00 00 00 36  00 00 00 2e 00 00 00 1e  |...v...6........|
00000020  00 00 00 12 00 00 00 42  00 00 00 1a 00 00 00 82  |.......B........|
00000030  00 00 00 3c 00 68 00 02  00 00 00 1c 00 48 00 00  |...<.h.......H..|
00000040  40 02 90 00 00 00 00 00  00 00 00 00 00 00 00 00  |@...............|
00000050  00 01 01 00 01 03 00 00  00 20 00 40 00 60 00 80  |......... .@.`..|
00000060  00 c0 01 40 01 80 01 c0  02 40 02 80 00 00 00 00  |...@.....@......|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 01 00 00 00 01 00 10  00 00 00 01 00 03 3f fc  |..............?.|
00000090  80 00 00 00 00 00 00 00  00 07 00 01 00 01 00 0b  |................|

hexdump -C Canvas5-3.5-Sample1.CVS | head
00000000  00 00 01 58 00 00 01 30  00 00 00 2a 00 00 00 00  |...X...0...*....|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000030  00 00 00 00 00 69 00 02  00 00 01 30 00 48 00 00  |.....i.....0.H..|
00000040  40 02 90 00 00 00 00 00  00 00 00 00 00 00 00 00  |@...............|
00000050  00 01 01 01 00 00 00 00  00 20 00 40 00 60 00 80  |......... .@.`..|
00000060  00 c0 01 40 01 80 01 c0  02 40 02 80 00 00 00 00  |...@.....@......|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 01 00 00 00 01 00 10  00 00 00 01 00 03 3f fc  |..............?.|
00000090  80 00 00 00 00 00 00 00  00 07 00 01 00 01 00 01  |................|

hexdump -C C3-5-S01.CVS | head
00000000  78 11 00 00 10 00 00 00  2a 00 00 00 0a 00 00 00  |x.......*.......|
00000010  26 00 00 00 26 00 00 00  26 00 00 00 26 00 00 00  |&...&...&...&...|
00000020  96 00 00 00 2a 00 00 00  2e 00 00 00 32 00 00 00  |....*.......2...|
00000030  00 00 00 00 01 6b 01 00  50 14 00 00 28 00 00 00  |.....k..P...(...|
00000040  6e 00 00 00 5b 00 00 00  01 00 04 00 00 00 00 00  |n...[...........|
00000050  e8 13 00 00 12 0b 00 00  12 0b 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 80 00 00 80 00 00  |................|
00000070  00 80 80 00 80 00 00 00  80 00 80 00 80 80 00 00  |................|
00000080  c0 c0 c0 00 80 80 80 00  00 00 ff 00 00 ff 00 00  |................|
00000090  00 ff ff 00 ff 00 00 00  ff 00 ff 00 ff ff 00 00  |................|

In the version 2 & 3 samples you can see some patterns, which I thought would allow for proper identification, but looking at more samples I found differences. One pattern I was hopeful might be consistent was the hex values “002000400060008000C00140018001C002400280”, but there are some which don’t match this pattern. If the file is truly compressed, it will be hard to know which values would be consistent among all files. I have over 8,000 samples and have a signature that only excludes around 20, so it will have to do for now.

When we start with Version 5 we get into some more identifiable headers, there is some oddness with some samples. But with an ascii string like “CANVAS5”, it should be easy, right? Not so fast, in version 5 you can compress the file structure. This removes the easily identifiable “CANVAS5” string. But some have a small string at the tail end, but others do not. 

hexdump -C Canvas5-Sample1.CV5 | head
00000000  02 00 00 80 00 00 00 00  00 00 00 4e 96 00 00 4e  |...........N...N|
00000010  96 18 02 00 00 00 0e a8  da 43 41 4e 56 41 53 35  |.........CANVAS5|
00000020  00 01 00 00 00 00 00 05  03 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 21 00 00  00 21 00 00 00 79 00 00  |.....!...!...y..|
00000040  00 03 00 00 01 6b 00 00  00 03 00 00 00 01 ff ff  |.....k..........|
00000050  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

hexdump -C Canvas5-Sample3-cmp.CV5 | head
00000000  02 00 00 80 00 00 00 00  08 00 00 80 00 00 00 03  |................|
00000010  5c ff ff ff ff 00 00 40  22 00 00 03 50 10 00 89  |\......@"...P...|
00000020  07 60 bd 0f f0 00 00 10  03 04 10 56 00 20 05 00  |.`.........V. ..|
00000030  e0 18 02 10 35 04 30 4e  05 30 72 07 f0 a8 0d a1  |....5.0N.0r.....|
00000040  17 11 81 19 05 50 5c 00  60 0f 00 10 80 02 90 80  |.....P\.`.......|
00000050  03 f0 56 05 50 55 05 b0  75 12 51 29 05 e0 55 05  |..V.PU..u.Q)..U.|

hexdump -C Canvas5-Sample3-cmp.CV5 | tail
00001ff0  00 00 00 01 08 a5 ab c0  00 00 00 00 3f 89 2c 58  |............?.,X|
00002000  00 00 00 00 08 a5 ab 80  00 00 00 00 ff d4 11 e4  |................|
00002010  00 00 00 00 08 a5 ab 90  00 02 3e d8 ff d3 12 cc  |..........>.....|
00002020  00 00 00 00 00 00 00 00  00 02 3e d8 00 01 00 09  |..........>.....|
00002030  00 00 00 00 00 00 00 00  00 00 00 00 08 a5 ab f8  |................|
00002040  00 00 00 00 43 4e 56 35                           |....CNV5|

Canvas 6 uses a new extension, but has a similar structure to the file format. With compression as an option. But some of the compressed files on Windows has a reversed string, “5VNC“. So many Canvas 5 compressed look identical to Canvas 6 compressed, complicating identification.

hexdump -C Canvas6-Sample.CNV | head
00000000  01 00 80 00 00 90 07 cd  07 00 80 00 00 00 80 00  |................|
00000010  00 17 01 00 00 59 f5 0e  00 43 41 4e 56 41 53 36  |.....Y...CANVAS6|
00000020  00 01 00 00 00 00 06 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 21 7a 00  00 00 7a 00 00 00 03 00  |.....!z...z.....|
00000040  00 00 6e 01 00 00 03 00  00 00 01 00 00 00 ff ff  |..n.............|
00000050  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

hexdump -C Canvas6-Sample1-c.CNV | head
00000000  01 00 80 00 00 58 ea 2b  00 c2 1d 00 00 d0 09 00  |.....X.+........|
00000010  00 00 00 0f 2e 00 00 0b  07 00 00 09 c4 10 00 01  |................|
00000020  00 00 03 00 20 04 00 70  ff 00 80 05 00 c0 06 06  |.... ..p........|
00000030  50 20 03 00 0f 06 10 6b  00 a0 12 01 00 48 07 20  |P .....k.....H. |
00000040  6d 07 30 40 06 40 11 06  00 0b 05 00 10 00 10 71  |m.0@.@.........q|
00000050  01 40 21 00 00 59 01 00  0f 05 10 00 00 e1 14 00  |.@!..Y..........|

hexdump -C Canvas6-Sample1-c.CNV | tail
000016a0  00 00 00 12 f6 00 00 c0  f0 12 00 3c d0 80 7c 58  |...........<..|X|
000016b0  2f 14 00 00 00 00 00 bc  f4 8d 00 0f 00 00 00 00  |/...............|
000016c0  f1 12 00 7f 00 00 00 f8  2e 14 00 bc f4 8d 00 1c  |................|
000016d0  f2 12 00 04 f3 12 00 fc  d1 80 7c 09 04 00 00 00  |..........|.....|
000016e0  00 00 40 00 f2 12 00 ff  ff ff ff 00 f1 12 00 1c  |..@.............|
000016f0  f1 12 00 bc f4 8d 00 00  00 00 40 35 56 4e 43     |..........@5VNC|

While most have the “CANVAS6” string near the beginning, quite a few are missing the CNV5/5VNC string at the end. Instead, many have the string “%SI-0200” near the end, which I use in my signature suggestion. This structure remained the same from version 6 to 8. 

hexdump -C Canvas8-S01.CNV | head
00000000  02 00 00 80 00 00 12 b8  80 00 00 11 19 00 00 11  |................|
00000010  19 18 02 00 00 00 0e f5  59 43 41 4e 56 41 53 36  |........YCANVAS6|
00000020  00 01 00 00 00 00 00 08  01 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 21 00 00  00 00 00 00 00 00 00 00  |.....!..........|
00000040  00 03 00 00 00 00 00 00  00 03 00 00 00 01 00 00  |................|
00000050  00 01 ff ff ff ff 00 00  00 02 00 00 00 02 00 00  |................|

But…….. There are plenty without these strings, just the “%SI-0200” near the end.

hexdump -C TELEGRPH.CNV | head
00000000  02 00 00 80 00 00 00 00  08 00 00 80 00 00 00 3d  |...............=|
00000010  f2 ff ff ff ff 00 00 75  76 00 00 3d e6 10 00 ff  |.......uv..=....|
00000020  00 00 b3 0d 90 a9 03 b0  8a 07 f0 98 07 60 80 08  |.............`..|
00000030  d0 35 01 c0 58 01 e0 59  04 80 b8 03 90 38 02 f0  |.5..X..Y.....8..|
00000040  e2 00 20 0b 03 70 1d 03  20 36 0f 30 00 01 80 09  |.. ..p.. 6.0....|

hexdump -C TELEGRPH.CNV | tail
00006850  2b 2c f9 ae 30 00 00 00  20 00 00 00 01 00 00 00  |+,..0... .......|
00006860  0f 00 00 00 10 00 00 00  1e 00 00 00 07 00 00 00  |................|
00006870  64 65 6e 65 62 61 00 00  00 00 01 4c 25 53 49 2d  |deneba.....L%SI-|
00006880  30 32 30 30 6d 61 63 00  00 00 00 00 00 00 00 00  |0200mac.........|
00006890  00 00 00 00                                       |....|

In version 9 and forward we have an extension change to CVX, but the format is similar with the “CANVAS6” string, but is a slightly different offset. It is still used with the current version of Canvas X.

hexdump -C Canvas9-Sample1.cvx | head
00000000  00 00 00 00 00 00 00 00  00 00 02 00 00 80 00 07  |................|
00000010  d1 84 d0 00 00 80 00 00  00 80 00 18 02 00 00 00  |................|
00000020  0f b7 ef 43 41 4e 56 41  53 36 00 01 00 00 00 00  |...CANVAS6......|
00000030  00 09 00 00 00 03 34 00  00 00 04 00 00 00 00 00  |......4.........|
00000040  00 00 00 3c 42 45 47 49  4e 5f 50 52 45 56 49 45  |...<BEGIN_PREVIE|
00000050  57 5f 54 41 47 3e 21 00  00 00 75 00 00 00 79 00  |W_TAG>!...u...y.|
00000060  00 00 03 00 00 01 6b 00  00 00 03 00 00 00 01 ff  |......k.........|
00000070  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

hexdump -C Canvas9-Sample1-compressed.cvx | tail
00004090  00 00 e0 20 00 57 80 00  00 00 00 00 0a 13 00 09  |... .W..........|
000040a0  00 00 04 00 00 00 00 01  00 00 00 00 bf ff e0 80  |................|
000040b0  bf ff e0 40 01 8c 5e 00  02 4a 22 d0 00 00 01 60  |...@..^..J"....`|
000040c0  bf ff e0 40 00 5c 08 18  00 00 00 00 00 0d 84 80  |...@.\..........|
000040d0  43 61 6e 76 61 73 39 2d  53 61 6d 70 6c 65 31 2d  |Canvas9-Sample1-|
000040e0  63 6f 6d 70 72 65 73 73  65 64 2e 63 76 78 00 18  |compressed.cvx..|
000040f0  bf ff e0 70 0a 12 6a a0  02 43 22 b4 00 0c aa 9c  |...p..j..C".....|
00004100  bf ff e0 80 00 00 00 01  00 00 00 00 00 0d 84 80  |................|
00004110  bf ff e0 b0 43 4e 56 35                           |....CNV5|

hexdump -C CanvasX2019-S01.cvx | head
00000000  00 00 00 00 00 00 00 00  00 00 01 00 80 00 00 00  |................|
00000010  6e ab 03 00 80 00 00 00  80 00 00 17 01 00 00 ef  |n...............|
00000020  b7 0f 00 43 41 4e 56 41  53 36 00 01 00 00 00 00  |...CANVAS6......|
00000030  09 00 00 4d 01 00 00 eb  4c 00 00 41 00 00 00 31  |...M....L..A...1|
00000040  52 45 56 03 00 00 00 01  00 00 00 00 00 00 00 00  |REV.............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

This collection of file formats is very hard to make sense of. Some really great consistent patterns on many samples, with lots of exceptions. Super confusing. This software has had a long run, with the latter years staying pretty stagnate in terms of new development. It is worth defining and creating a signature for the consistent patterns, then we can dial in the variants over time?

The signatures I have built miss about 23 files in versions 1-3 out of the ~9000 samples I have and for Canvas 5, only some of the compressed files are currently not identified. But so far all my CNV and CVX files identify correctly, so probably good for now.

CanvasX dropped supported for the Macintosh, but did release an entirely different product called Canvas X Draw, which does support the Macintosh. Here is what a CVD file looks like:

hexdump -C CanvasXDraw7-Sample1.cvd | head
00000000  25 43 61 6e 76 61 73 43  56 44 09 31 2e 30 25 bb  |%CanvasCVD.1.0%.|
00000010  54 48 65 61 64 65 72 00  00 00 00 00 00 00 00 00  |THeader.........|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 bb 52 4d 61 63 4f 53  56 65 72 73 69 6f 6e 20  |..RMacOSVersion |
00000040  31 30 2e 31 33 2e 36 20  28 42 75 69 6c 64 20 31  |10.13.6 (Build 1|
00000050  37 47 31 34 30 34 32 29  31 30 2e 32 33 30 34 08  |7G14042)10.2304.|
00000060  00 00 00 70 6c 61 74 66  6f 72 6d 0a 73 00 00 00  |...platform.s...|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 00 00 00 05 00 00  00 02 00 00 00 00 00 00  |................|
00000090  00 08 00 00 00 6f 73 0a  73 00 00 00 00 00 00 00  |.....os.s.......|

There is also the matter of a Canvas Image, which the User Guide calls proxy images. They are Raster images used in placements within Canvas Documents. Should be easy to identify.

hexdump -C Canvas5-Sample1.CVI | head
00000000  00 00 00 01 44 41 44 35  50 52 4f 58 00 00 09 99  |....DAD5PROX....|
00000010  00 00 00 11 00 00 00 2d  00 00 00 03 00 00 00 08  |.......-........|
00000020  00 48 00 00 00 00 00 06  00 03 00 08 00 00 00 11  |.H..............|
00000030  00 00 00 2d 00 03 00 03  00 48 00 00 00 48 00 00  |...-.....H...H..|
00000040  00 00 00 00 00 00 00 00  00 00 00 11 00 00 00 2d  |...............-|
00000050  00 00 00 02 00 00 00 08  00 00 00 01 00 00 00 11  |................|
00000060  00 00 00 2d ff ff ff ff  ff ff ff ff ff ff ff ff  |...-............|
00000070  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|

Phew, if you held on for this whole post you must really like confusing file format structures. This format has been on my mind on and off for about 6 years. Hopefully these signatures will work for the vast majority of the Canvas files found in archives and personal systems. As always here is my GitHub with the signatures I am proposing and a few samples to get you confused.

MAGIX

Posted on by Thor Leave a comment

There are probably many reasons why a software developer might want to create a proprietary format to store their files in. The software may require special features that don’t fit into an existing format. I would hope a developer would try to use existing formats, or even better open formats, but for many reasons, which probably include profits, they choose to re-invent the wheel often.

MAGIX is a German company which started making software in 1994. In 2001 they developed their first video editing software which was called Movie Edit Pro. The software seems to be well received and is still in use today.

Like most video editing software, project files are used to store all the edits and links to video files. These are usually smaller text based, with many using XML as the project format. Not MAGIX, they decided to go with a different yet known format for their project files.

hexdump -C MAGIX15-s01.MVP | head
00000000  52 49 46 46 6c 37 01 00  53 45 4b 44 4d 56 50 48  |RIFFl7..SEKDMVPH|
00000010  08 00 00 00 00 00 00 00  00 00 00 00 4c 49 53 54  |............LIST|
00000020  0c 16 01 00 4d 56 50 4c  4c 49 53 54 00 16 01 00  |....MVPLLIST....|
00000030  56 49 50 4c 53 56 49 50  0c 07 00 00 00 dc 05 00  |VIPLSVIP........|
00000040  00 00 00 00 20 00 00 00  0c 00 00 00 80 bb 00 00  |.... ...........|
00000050  10 00 00 00 29 6b 55 e2  53 f8 3d 40 00 00 f0 42  |....)kU.S.=@...B|
00000060  01 00 00 00 bd 04 ef fe  00 00 01 00 06 00 08 00  |................|
00000070  00 00 01 00 06 00 08 00  00 00 01 00 3f 00 00 00  |............?...|
00000080  28 00 00 00 04 00 04 00  01 00 00 00 00 00 00 00  |(...............|
00000090  00 00 00 00 00 00 00 00  bd 8f 32 01 d0 02 00 00  |..........2.....|

Yes, they used the RIFF container format for their projects. Seems an odd choice, especially for video production although it is well suited for it. AVI is another video format which uses the RIFF container. The MVP project file uses the ID SEKD with the format MVPH. Earlier versions of Movie Edit Pro used a different extension. 

hexdump -C MAGIXv11-s01.MVD | head
00000000  52 49 46 46 38 57 00 00  53 45 4b 44 53 56 49 50  |RIFF8W..SEKDSVIP|
00000010  70 00 00 00 00 dc 05 00  00 00 00 00 04 00 00 00  |p...............|
00000020  02 00 00 00 80 bb 00 00  10 00 00 00 8e 23 d6 e2  |.............#..|
00000030  53 f8 3d 40 00 00 f0 42  01 00 00 00 bd 04 ef fe  |S.=@...B........|
00000040  00 00 01 00 00 00 06 00  00 00 04 00 00 00 06 00  |................|
00000050  00 00 04 00 3f 00 00 00  28 00 00 00 04 00 04 00  |....?...(.......|
00000060  01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  c8 1b 32 01 d0 02 00 00  e0 01 00 00 52 d7 da fb  |..2.........R...|
00000080  54 55 f5 3f 4c 49 53 54  04 00 00 00 70 68 79 73  |TU.?LIST....phys|
00000090  4c 49 53 54 d0 3d 00 00  74 72 6b 73 4c 49 53 54  |LIST.=..trksLIST|

The MVD format used on an earlier version of Movie Edit Pro is also a RIFF, and with the ID of SEKD, but has a format of SVIP.

RIFFpad can break down the chunks we see in an MVP file. Each of the LIST chunks has their own subchunks as well. I assume this his how the editing software stores each video/audio track references, etc. So I give it to MAGIX for at least using an understandable format to store their projects.

MAGIX has also used RIFF in many of its supporting formats. So far I have found mfx, afx, ifx, cfx, ctf, tfx, ufx, mmt, mmm, hdp, each having their own format:

hexdump -C 101_Loud.mfx | head
00000000  52 49 46 46 a8 6f 00 00  53 45 4b 44 4d 41 46 58  |RIFF.o..SEKDMAFX|
00000010  00 00 00 00 4c 49 53 54  94 6f 00 00 41 55 46 58  |....LIST.o..AUFX|
00000020  4c 49 53 54 88 6f 00 00  41 46 58 45 46 58 48 44  |LIST.o..AFXEFXHD|
00000030  20 00 00 00 00 00 25 0d  00 00 00 00 02 00 00 00  | .....%.........|
00000040  01 00 00 00 00 00 00 00  03 18 00 00 00 00 00 00  |................|
00000050  00 00 00 00 4c 49 53 54  54 6f 00 00 41 46 58 44  |....LISTTo..AFXD|
00000060  4c 49 53 54 50 6a 00 00  41 46 58 45 46 58 48 44  |LISTPj..AFXEFXHD|
00000070  20 00 00 00 00 00 25 0d  00 00 00 00 05 00 00 00  | .....%.........|
00000080  01 00 00 00 00 00 00 00  03 18 00 00 00 00 00 00  |................|
00000090  00 00 00 00 4c 49 53 54  1c 6a 00 00 41 46 58 44  |....LIST.j..AFXD|

Not sure the best way to manage all of these in terms of identification, as I am not sure what what is the purpose of each format. Maybe for now I’ll make a generic to catch them all as a MAGIX File.

ExtensionIDFORMAT
AFXSEKDSAFX
CFXSEKDSCFX
CTFSEKDSVIP
HDPSEKDSHDP
IFXSEKDSIFX
MFXSEKDMAFX
MMMSEKDSVIP
MMTSEKDSVIP
MVDSEKDSVIP
MVPSEKDMVPH
MXMMXMDmxmi
TFXSEKDSTFX
UFXSEKDSVIP

But, when it comes to their proprietary MAGIX Video format, I think they may have pushed things a little too far. Meet the MXV format: 

hexdump -C MAGIXv11-s01.mxv | head
00000000  4d 58 52 49 46 46 36 34  9a cb 2b 00 00 00 00 00  |MXRIFF64..+.....|
00000010  4d 58 4a 56 49 44 36 34  4d 58 4a 56 48 32 36 34  |MXJVID64MXJVH264|
00000020  70 00 00 00 00 00 00 00  70 00 00 00 03 00 00 00  |p.......p.......|
00000030  42 93 2b 00 00 00 00 00  f0 00 00 00 00 00 00 00  |B.+.............|
00000040  7b 2e 00 00 4b 00 00 00  01 00 00 00 00 00 00 00  |{...K...........|
00000050  8e 23 d6 e2 53 f8 3d 40  80 02 00 00 e0 01 00 00  |.#..S.=@........|
00000060  80 02 00 00 e0 01 00 00  04 00 00 00 43 15 00 00  |............C...|
00000070  f0 00 00 00 00 00 00 00  28 19 00 00 00 00 00 00  |........(.......|
00000080  55 55 55 55 55 55 f5 3f  00 00 00 00 00 00 00 00  |UUUUUU.?........|
00000090  7f dd 05 00 00 00 00 00  4d 58 4a 56 48 44 36 34  |........MXJVHD64|

I am not sure what I am looking at, is it a RIFF? Is it a RIFF variant like RF64? MAGIX claims the format is:

This is the MAGIX video format for quicker processing with MAGIX products. It offers very low loss of quality, but it cannot be played via conventional DVD players.

MAGIX Video Pro X6

A look around the internet doesn’t bring much up in reference to this format. Just my recent page on the format wiki. A search for MXRIFF64 bring up nothing. But a closer look at other strings within the MXV file reveal we are probably looking at some sort of MPEG format.

I was able to locate a project on GitHub which claims to be able to demux the MXV format. The software is written in GO and appears to indicate this format is chunked based and has most of the chunks figured out. So if you find yourself stuck with some MXV files and don’t want to use the latest from MAGIX, this might be the tool for you.

This demuxer also has an interesting file you can download. It is called a “GRAMMAR” file and can be loaded into hex viewers like Synalyze It! can show the parts of a file you load. Its a great way to explore a format!

None of these formats are found in PRONOM, project files are not usually kept in archives, but if would be good to know about the RIFF files if they do turn up. The video format is for sure something the archival world should know about. MediaInfo is currently not aware of this format, but seems like it might be an easy task.

As usual, you can see some samples and my proposal signatures on my GitHub.

Designer

Posted on by Thor Leave a comment

Micrografx / Corel Designer

Many software titles we have all used began life under a different brand or even title. Larger software companies gobble up smaller developers, some brands merge, and others change names for whatever reason. Adobe has bought many smaller companies over the years, sometimes developing the acquired software and other times burying the software to avoid competition. Pagemaker was bought to give InDesign life, many Macromedia titles were incorporated or shelved. Such is life in the software world.

In understanding a file format, often times you need to follow this trail backwards to understand when file formats changed and compatibility is dropped. Often times the formats remained the same, but the extension is changed. Or the software name changes and formats are updated, but the extension remains the same. There can also be multiple titles which all use a common format, further complicating the identification of the formats.

Let’s look closer at the a title which changed names and file formats a few times over the years. Micrografx was founded in 1982 and were pretty well known for their innovation in computer graphics. They have released many titles over the years, but one of the first was In*A*Vision graphic software for Windows 1.0 in 1986. This software used a format with the .PIC extension. A couple years later version 2, was renamed to Micrografx Designer and used the .DRW extension. This extension was also used by Micrografx Draw, another similar program.

Micrografx Designer continued to be released until version 9 which is when it was purchased by Corel who continued to release new versions, although it is said the software was just a variation of CorelDraw, and now Designer is part of the CorelDraw Technical Suite. Other Micrografx software such as Picture Publisher was discontinued and customers were encouraged to use Corel’s PaintShop Pro instead. Somewhere in the middle of all this, Micrografx spun off a separate business unit called iGrafx, which Designer was marketed under for a short time.

Let’s break down the names, extensions used, and format type.

  • In*A*Vision & Draw, binary format, PIC extension
  • Micrografx Designer & Draw, binary format, DRW extension
  • Micrografx Designer version 4, RIFF format, DS4 & MGX extension
  • Micrografx Designer versions 6-9, OLE Container format, DSF extension
  • Micrografx/Corel Designer versions 10-12, RIFF format, DES extension
  • Corel Designer version X4-Current, ZIP/XML format, DES extension

According to the 2021 Corel DesignerUser Guide:

Corel DESIGNER (DES, DSF, DS4, or DRW)

You can import Corel DESIGNER files. Files from version 10 and later have the filename extension .des. Files from Micrografx versions 6 to 9 have the filename extension .dsf. Version 4 files have the filename extension .ds4. The .drw filename extension is used for a Micrografx 2.x or 3.x file. Micrografx template files (DST) are also supported.

The PRONOM registry has a few of these formats with signatures and documented, but not all, let’s see where the gaps are.

PUIDFormat NameFormat VersionExtension
x-fmt/151 Micrografx Designer dsf
x-fmt/296 Micrografx Designer 3.1drw
x-fmt/47 Micrografx Draw 1-2drw
x-fmt/294 Micrografx Draw 3drw
x-fmt/295 Micrografx Draw 4drw, drt
fmt/1907Micrografx Icon File icn
fmt/1481Micrografx In-A-Vision Drawingpic

So from the PRONOM list, it appears we have good identification on the original PIC and DRW formats. Then the Designer DSF OLE container is taken care of as well. That leaves us with DS4 and DES formats.

hexdump -C DS41-S01.DS4 | head
00000000  52 49 46 46 6e 07 00 00  4d 47 58 20 69 74 70 64  |RIFFn...MGX itpd|
00000010  04 00 00 00 00 02 00 80  70 72 6f 70 23 00 00 00  |........prop#...|
00000020  1f 00 00 30 02 00 00 00  08 00 2c 40 44 00 11 20  |...0......,@D.. |
00000030  20 00 01 10 80 e0 00 00  91 08 21 e0 5c 82 90 72  | .........!.\..r|
00000040  05 ff c0 00 4c 49 53 54  10 04 00 00 64 69 74 6e  |....LIST....ditn|
00000050  74 68 6e 6c 03 04 00 00  57 01 00 30 00 00 08 00  |thnl....W..0....|
00000060  08 00 00 41 04 00 01 20  a4 00 82 10 72 14 40 48  |...A... ....r.@H|
00000070  00 58 20 84 04 32 10 40  00 12 c8 98 18 22 63 90  |.X ..2.@....."c.|
00000080  2b 91 32 36 47 08 20 c0  23 e4 80 90 92 22 46 49  |+.26G. .#...."FI|
00000090  09 29 26 24 e4 a0 94 92  a2 56 4b 09 69 2e 25 e4  |.)&$.....VK.i.%.|

Micrografx Designer 4 apparently uses the RIFF container format. The RIFF format is used with many different types of formats. The most common is the WAV format. CorelDRAW also uses the RIFF format so it makes sense they would use it as they took over from Micrografx.

Each RIFF format has a four byte identifier type after the first eight bytes which identify the RIFF. The DS4 file uses the code “MGX ” to identify itself. Which also appears to be used with their clipart format, MGX. We can use the same identification method we use for other RIFF’s to identify this format. 

hexdump -C Corel-DES10Sample.des | head
00000000  52 49 46 46 8a 57 00 00  44 45 53 41 76 72 73 6e  |RIFF.W..DESAvrsn|
00000010  02 00 00 00 7e 04 4c 49  53 54 54 0c 00 00 69 63  |....~.LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Starting with version 10 of Corel Designer, the RIFF format is used again and has a different type. With Version 10 using “DESA”, then for version 10.5:

hexdump -C Corel-DES10.5Sample.des | head 
00000000  52 49 46 46 cc 57 00 00  44 45 53 42 76 72 73 6e  |RIFF.W..DESBvrsn|
00000010  02 00 00 00 b0 04 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

The next version after 10.5 is version 12 and it shows a type:

hexdump -C Corel-DES12-Sample.des | head 
00000000  52 49 46 46 ce 57 00 00  44 45 53 43 76 72 73 6e  |RIFF.W..DESCvrsn|
00000010  02 00 00 00 e2 04 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

After version 12, Corel started using numbering consistent with their other products. The first being X4.

hexdump -C Corel-DESX4-Sample.des | head
00000000  50 4b 03 04 14 00 00 08  00 00 f8 bb c9 4e c3 4b  |PK...........N.K|
00000010  9c d1 2d 00 00 00 2d 00  00 00 08 00 00 00 6d 69  |..-...-.......mi|
00000020  6d 65 74 79 70 65 61 70  70 6c 69 63 61 74 69 6f  |metypeapplicatio|
00000030  6e 2f 78 2d 76 6e 64 2e  63 6f 72 65 6c 2e 64 65  |n/x-vnd.corel.de|
00000040  73 69 67 6e 65 72 2e 64  6f 63 75 6d 65 6e 74 2b  |signer.document+|
00000050  7a 69 70 50 4b 03 04 14  00 00 08 00 00 f8 bb c9  |zipPK...........|
00000060  4e 6f 38 b6 64 98 13 00  00 98 13 00 00 14 00 00  |No8.d...........|
00000070  00 63 6f 6e 74 65 6e 74  2f 72 69 66 66 44 61 74  |.content/riffDat|
00000080  61 2e 63 64 72 52 49 46  46 90 13 00 00 44 45 53  |a.cdrRIFF....DES|
00000090  45 76 72 73 6e 02 00 00  00 82 05 4c 49 53 54 54  |Evrsn......LISTT|

Well it looks like things changed, starting with X4 the format changed to a ZIP container. Let’s take a peak inside.

Path = Corel-DESX4-Sample.des
Type = zip
Physical Size = 8714

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:47 .....           45           45  mimetype
2019-06-09 22:31:47 .....         5016         5016  content/riffData.cdr
2019-06-09 22:31:47 .....       196662          239  metadata/thumbnails/thumbnail.bmp
2019-06-09 22:31:47 .....       151606          698  metadata/thumbnails/page1.bmp
2019-06-09 22:31:47 .....          596          259  metadata/textinfo.xml
2019-06-09 22:31:47 .....         4977         1314  metadata/metadata.xml
2019-06-09 22:31:47 .....           53           55  links.xml
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:47             358955         7626  7 files

Looks like the container holds a RIFF inside along with some thumbnails, metadata, and other things. The mimetype file simple holds “application/x-vnd.corel.designer.document+zip”. The riffData.cdr however looks like this:

hexdump -C Corel-DESX4-Sample/content/riffData.cdr | head
00000000  52 49 46 46 90 13 00 00  44 45 53 45 76 72 73 6e  |RIFF....DESEvrsn|
00000010  02 00 00 00 82 05 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Another RIFF, and seems to be in the same sequence, but going from version 12 to X4 we seemed to have skipped “DESD”. Maybe there was a developer version in between as they transitioned. Version X5 looks similar and has the RIFF sequence “DESF”. When we get to X6 the structure changes. 

Path = Corel-DESX6-Sample.des
Type = zip
Physical Size = 8568

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:21 .....           45           45  mimetype
2019-06-09 22:31:21 .....        12153         1098  content/data/data1.dat
2019-06-09 22:31:21 .....          439          224  content/data/masterPage.dat
2019-06-09 22:31:21 .....          613          265  content/data/page1.dat
2019-06-09 22:31:21 .....           34           28  content/dataFileList.dat
2019-06-09 22:31:21 .....          960          279  content/root.dat
2019-06-09 22:31:21 .....       196662          239  metadata/thumbnails/thumbnail.bmp
2019-06-09 22:31:21 .....       151606          698  metadata/thumbnails/page1.bmp
2019-06-09 22:31:21 .....          427          208  color/color.xml
2019-06-09 22:31:21 .....          596          259  metadata/textinfo.xml
2019-06-09 22:31:21 .....          103          100  color/docPalette.xml
2019-06-09 22:31:21 .....        14920         1444  styles/document.cdss
2019-06-09 22:31:21 .....         5500         1462  metadata/metadata.xml
2019-06-09 22:31:21 .....           53           55  links.xml
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:21             384111         6404  14 files

The mimetype remains the same, but we see additional files within the structure. Also the riffData.cdr file is missing. Looking at each file we can see the root.dat file is a RIFF and follows the same sequence. 

hexdump -C Corel-DESX6-Sample/content/root.dat | head
00000000  52 49 46 46 b8 03 00 00  44 45 53 47 66 76 65 72  |RIFF....DESGfver|
00000010  10 00 00 00 ff ff ff ff  08 00 00 00 5e 06 02 00  |............^...|
00000020  00 00 10 00 76 72 73 6e  10 00 00 00 ff ff ff ff  |....vrsn........|
00000030  02 00 00 00 5e 06 00 00  00 00 00 00 4c 49 53 54  |....^.......LIST|
00000040  7c 00 00 00 64 6f 63 20  6d 63 66 67 10 00 00 00  ||...doc mcfg....|
00000050  00 00 00 00 83 20 00 00  00 00 00 00 00 00 00 00  |..... ..........|
00000060  70 72 65 66 10 00 00 00  00 00 00 00 e6 0e 00 00  |pref............|
00000070  83 20 00 00 00 00 00 00  70 74 72 74 10 00 00 00  |. ......ptrt....|
00000080  00 00 00 00 10 00 00 00  69 2f 00 00 00 00 00 00  |........i/......|
00000090  4c 49 53 54 04 00 00 00  66 69 6c 74 4c 49 53 54  |LIST....filtLIST|

As we get to a more recent version. We can see the pattern continues.

hexdump -C Designer2022-s01/content/root.dat | head
00000000  52 49 46 46 88 06 00 00  44 45 53 4e 66 76 65 72  |RIFF....DESNfver|
00000010  10 00 00 00 ff ff ff ff  08 00 00 00 60 09 02 00  |............`...|
00000020  00 00 18 00 76 72 73 6e  10 00 00 00 ff ff ff ff  |....vrsn........|
00000030  02 00 00 00 60 09 00 00  00 00 00 00 4c 49 53 54  |....`.......LIST|
00000040  30 01 00 00 64 6f 63 20  6d 63 66 67 10 00 00 00  |0...doc mcfg....|
00000050  00 00 00 00 08 1f 00 00  00 00 00 00 00 00 00 00  |................|
00000060  70 72 65 66 10 00 00 00  00 00 00 00 ae 07 00 00  |pref............|
00000070  08 1f 00 00 00 00 00 00  70 74 72 74 10 00 00 00  |........ptrt....|
00000080  00 00 00 00 10 00 00 00  b6 26 00 00 00 00 00 00  |.........&......|
00000090  4c 49 53 54 4c 00 00 00  66 6e 74 74 66 6f 6e 74  |LISTL...fnttfont|

The last sample I have is for Corel Designer 2022, but there could be more. I created new signatures for all the samples I have, you can see them in my Github as usual. I decided to group some of the versions together to simplify things a bit, but if anyone thinks they should be broken out into individual versions, let me know.

Writing Center

Posted on by Thor Leave a comment

In honor of #Marchintosh, I threatened in an earlier post to discuss The Writing Center, one of the many writing programs marketed by the Learning Company for the Mac. This one was developed by Datapak Software, Inc and I think they wanted to watch the world burn.

This format was different enough from the Student Writing Center and the “Ultimate Writing & Creativity Center” to need its own post. Moreover, I am pretty sure the developers of this software were actively trying to frustrate anyone trying to document the format. Let me explain.

In the early Macintosh world, very rarely were extensions used. Current systems use extensions to link the file to an application which can open the file. On the Mac, the system would use special attributes called Type / Creator codes. These codes were registered with Apple so they would be unique to a specific software and type of file. The codes used the FourCC system and unfortunately Apple never released a full list of codes used. Some folks over the years have tried to document as many as they can. Many used simple understandable codes, for example, A Microsoft Word document has a Type / Creator of W6BN / MSWD. The creator code of MSWD is very readable, and the type code W6BN is unique to a document from version 6 of Microsoft Word.

This Sample Report file from The Writing Center, when investigated with the ResEdit tool show interesting Type / Creator codes. If we look at the hexadecimals values for the codes. The first four bytes are the Type code and the second set of 4 bytes are the Creator code.

xattr -p com.apple.FinderInfo "Sample Report" 
0000   0A 57 50 31 0A 1A 57 50 01 00 00 00 00 00 00 00    .WP1..WP........

getfileinfo "Sample Report" 
file: "Sample Report"
type: "\nWP1"
creator: "\n\^ZWP"
attributes: avbstclInmedz
created: 10/13/1990 00:10:54
modified: 07/25/1991 11:58:20

The first thing to know is the encoding for all Type / Creator codes is MacRoman, so if we look up the hexadecimal code for “0A” we learn it is the character for a new Line Feed, why in the world would you use the line feed character? The developers must have had a sense of humor, or are psychopaths, and I’m leaning toward the latter. Trying to put this character into any sort of spreadsheet or text based document with other codes throws everything off! When I try and use a spreadsheet with a group of codes and then use a script to look them up on the command line I get crazy formatting. Not to mentioned the second character in the creator code is “1A” which is a substitute character.

This is just one example of crazy characters being used in Type / Creator codes. Stay tuned for more on these in future discussions.

Even though the Type / Creator codes are very useful in identification of this format, often times the Finder attribute is lost. This can happen if the file is moved off an HFS disk, usually a network or through the internet. Then all we have is the binary data fork and a file with no extension. So finding a signature to identify this format is useful.

hexdump -C "Sample Report" | head
00000000  00 12 cf fc 00 00 05 78  00 00 00 00 01 18 01 eb  |.......x........|
00000010  ff ff ff c4 ff ff ff c4  00 00 02 82 00 00 02 28  |...............(|
00000020  00 00 00 00 00 00 00 00  00 00 05 76 00 00 00 30  |...........v...0|
00000030  00 00 02 70 00 aa 00 00  05 76 00 00 00 30 00 00  |...p.....v...0..|
00000040  02 70 00 aa 00 00 00 00  00 00 00 00 00 00 00 00  |.p..............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 12  |................|
00000070  d1 2c 00 00 05 3f 00 00  00 00 01 00 06 47 65 6e  |.,...?.......Gen|
00000080  65 76 61 00 00 00 00 00  00 00 00 00 00 00 00 00  |eva.............|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|

hexdump -C WC-s01 | head        
00000000  03 df cd 9c 00 00 00 09  00 00 00 00 02 c3 02 64  |...............d|
00000010  00 00 00 00 00 00 00 00  00 00 00 59 00 00 02 64  |...........Y...d|
00000020  00 00 00 00 00 00 00 00  00 00 00 07 00 00 00 00  |................|
00000030  00 00 00 00 00 79 00 00  00 07 00 00 00 00 00 00  |.....y..........|
00000040  00 00 00 79 00 00 00 00  00 00 00 00 00 00 00 00  |...y............|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 03 df  |................|
00000070  cd 78 00 00 00 00 00 00  00 00 01 00 06 47 65 6e  |.x...........Gen|
00000080  65 76 61 00 00 00 00 00  00 00 00 00 00 00 00 00  |eva.............|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|

Looking at the hexadecimal values of the header of a couple samples doesn’t initially look promising, the first few bytes are very different meaning there is no magic bytes at the beginning of the file. In fact the only thing the same is the mention of the Geneva font used in the document. Looking further into the files.

hexdump -C "Sample Report"       
00000000  00 12 cf fc 00 00 05 78  00 00 00 00 01 18 01 eb  |.......x........|
...
000000b0  00 00 00 00 00 00 00 02  84 28 ff ff 00 00 00 00  |.........(......|
000000c0  00 17 4e 26 00 12 d2 fc  00 00 00 00 00 12 d0 88  |..N&............|

hexdump -C WC-s01        
00000000  03 df cd 9c 00 00 00 09  00 00 00 00 02 c3 02 64  |...............d|
...
000000b0  00 00 00 00 00 00 00 02  84 28 ff ff 00 00 00 00  |.........(......|
000000c0  03 e3 a5 70 03 df cd 8c  00 00 00 00 03 df cd 64  |...p...........d|

hexdump -C Stationery 
00000000  00 12 d2 e8 00 00 00 02  00 00 00 00 01 17 01 ec  |................|
...
000000b0  00 00 00 00 00 00 00 02  84 20 ff ff 00 00 00 00  |......... ......|
000000c0  00 17 56 f8 00 12 cd f8  00 00 00 00 00 12 ce 40  |..V............@|

The only bytes I could find near the beginning that seemed semi consistent is the highlighted bytes above. I did however notice some consistent bytes at the end of each of the files.

hexdump -C "Sample Report" | tail                                                      
00007250  e5 00 02 e5 00 02 e5 00  02 e5 00 02 e5 00 02 e5  |................|
00007260  00 02 e5 00 02 e5 00 02  e5 00 02 e5 00 ff 00 07  |................|
00007270  00 00 00 05 04 31 2e 30  30 00 09 00 00 00 05 04  |.....1.00.......|
00007280  31 2e 30 30 00 08 00 00  00 05 04 31 2e 30 30 00  |1.00.......1.00.|
00007290  0a 00 00 00 05 04 31 2e  30 30 00 0b 00 00 00 02  |......1.00......|
000072a0  00 00 00 0c 00 00 00 10  00 00 00 00 00 00 00 00  |................|
000072b0  00 00 00 01 00 00 00 01  00 11 00 00 00 08 00 2b  |...............+|
000072c0  00 03 01 52 01 fd 00 13  00 00 00 02 00 00 7f ff  |...R............|
000072d0  00 00 00 00 00 00 72 dc  7f ff ff ff              |......r.....|

hexdump -C WC-s01 | tail                                                              
000003c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000003d0  01 00 00 80 0c 00 08 00  05 00 00 00 00 01 d2 03  |................|
000003e0  ee dc 3e 00 00 00 00 00  07 00 00 00 01 00 00 09  |..>.............|
000003f0  00 00 00 01 00 00 08 00  00 00 01 00 00 0a 00 00  |................|
00000400  00 01 00 00 0b 00 00 00  02 00 00 00 0c 00 00 00  |................|
00000410  10 00 00 00 00 00 00 00  00 00 00 00 01 00 00 00  |................|
00000420  01 00 11 00 00 00 08 00  2b 00 c7 02 fd 03 3a 00  |........+.....:.|
00000430  13 00 00 00 02 00 00 7f  ff 00 00 00 00 00 00 04  |................|
00000440  45 7f ff ff ff                                    |E....|

hexdump -C Stationery | tail
000039a0  00 02 e3 00 02 e3 00 02  e3 00 02 e3 00 02 e3 00  |................|
000039b0  02 e3 00 02 e3 00 02 e3  00 02 e3 00 02 e3 00 ff  |................|
000039c0  00 07 00 00 00 05 04 31  2e 30 30 00 09 00 00 00  |.......1.00.....|
000039d0  05 04 31 2e 30 30 00 08  00 00 00 05 04 31 2e 30  |..1.00.......1.0|
000039e0  30 00 0a 00 00 00 05 04  31 2e 30 30 00 0b 00 00  |0.......1.00....|
000039f0  00 02 00 00 00 0c 00 00  00 10 00 00 00 00 00 00  |................|
00003a00  00 00 00 00 00 01 00 00  00 01 00 11 00 00 00 08  |................|
00003a10  00 2b 00 03 01 51 01 fe  00 13 00 00 00 02 00 00  |.+...Q..........|
00003a20  7f ff 00 00 00 00 00 00  3a 2e 7f ff ff ff        |........:.....|

The four bytes at the end of each file by themselves would not be a good signature as there are many formats which end with a few “FF” sequences. But maybe combined with bytes near the beginning, a signature might be found. I added a couple samples to my Github page if you would like to take a look. In order to retain the extended attributes, I encoded the files as MacBinary.

lsar -L "Sample Report.bin"
Sample Report.bin: MacBinary
Sample Report: 
  Name:                    Sample Report
  Size:                    29.4 KB (29,404 bytes)
  Compressed size:         29.4 KB (29,440 bytes)
  Last modified:           Thursday, July 25, 1991 at 12:58:20 PM
  Created:                 Saturday, October 13, 1990 at 1:10:54 AM
  Mac OS type code:        ?WP1 (0x0a575031)
  Mac OS creator code:     ??WP (0x0a1a5750)
  Mac OS Finder flags:     0x0100
  Index in file:           0
  Length of embedded data: 29404
  Start of embedded data:  128
  Original archive entry:  Is an embedded MacBinary file: Yes

Melco

Posted on by Thor Leave a comment

I came across another CD-ROM the other day with some fun embroidery formats. It includes the HUS format I recently posted on, plus a few more.

Like I mentioned before, this is a format genre which is not normally seen in the archival world, but is fun to take a peek into the world of embroidery formats. The HUS format from Husqvarna was a unique proprietary format, but looking at another in this set, we see a common container format.

filename : 'CH1604.ofm'
filesize : 25600
modified : 2002-04-29T05:58:26-06:00
errors   : 
matches  :
  - ns      : 'pronom'
    id      : 'fmt/111'
    format  : 'OLE2 Compound Document Format'
    version : 
    mime    : 
    class   : 'Text (Structured)'
    basis   : 'byte match at 0, 30'

First, what is an OFM file? It is the native format for Melco branded embroidery machines. They have been around for a few years. Melco has been around since 1972, but i’m sure the format is much newer. The fact that it is in an OLE container would indicate it was created in the mid 1990’s.

Looking inside the OLE container:

Path = CH1604.ofm
Type = Compound
Physical Size = 25600
Extension = compound
Cluster Size = 512
Sector Size = 64

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
                    .....        19171        19456  EdsIV Object
                    .....         2502         2560  Design Icon
                    .....          130          192  Design Status
------------------- ----- ------------ ------------  ------------------------
                                 21803        22208  3 files

The EdsIV Object seems specific. Looking back at the web archive it looks like EDS IV was software available for the Melco products. In a user manual there are three formats associated with the software:

  • .CND – Condensed Format
  • .EXP – Expanded Format
  • .OFM – Project (Layout format)

The EdsIV Object file is unique and will work well for identification. There also seems to be some common patterns within the file that can further the correct identification.

hexdump -C EdsIV Object | head
00000000  03 00 00 00 03 00 00 00  00 00 00 00 00 00 ff ff  |................|
00000010  0b 00 0c 00 43 50 72 6a  44 65 66 61 75 6c 74 73  |....CPrjDefaults|
00000020  05 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 00 f0 3f  28 00 00 00 01 00 00 00  |.......?(.......|
00000040  7f 00 00 00 00 00 00 00  00 00 39 40 00 00 00 00  |..........9@....|
00000050  00 00 10 40 00 00 00 00  00 00 00 00 00 00 00 00  |...@............|
00000060  00 00 00 00 00 00 00 00  00 00 59 40 04 00 00 00  |..........Y@....|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 80 51 40  |..............Q@|
00000080  00 00 00 00 00 00 3e 40  00 00 00 00 00 00 2e 40  |......>@.......@|
00000090  00 00 00 00 00 80 56 40  00 00 00 00 00 80 51 40  |......V@......Q@|

The CND and EXP formats are a different matter. I ran Tridscan across all the CND samples and it could not detect one common pattern among them all. 

python tridscan.py *.csd

TrIDScan/Py v2.02 - (C) 2015-2016 By M.Pontello

File(s) to scan found: 60
Scanning for patterns...
Checking file 1/60 './Cf0103.csd'
Checking file 2/60 './Cr0005.csd'
  Pattern(s) found: 11
Checking file 3/60 './Fd0106.csd'
tridscan.py: Error: no patterns found!

Being a condensed format, I gather it might have some compression which makes for a difficult binary file to identify.

The EXP format on the other hand has a short pattern at the beginning:

hexdump -C CF0103.EXP | head
00000000  80 02 00 00 80 02 18 e7  80 02 19 e6 80 02 19 e6  |................|
00000010  80 02 19 e7 80 02 19 e6  80 02 19 e6 80 02 19 e6  |................|
00000020  80 02 19 e7 80 02 19 e6  80 02 19 e6 80 02 18 e7  |................|
00000030  00 00 fc 00 04 00 fc 00  04 ff fc 01 ed 00 ec 00  |................|
00000040  21 21 df de da 01 15 14  15 15 15 15 eb eb eb eb  |!!..............|
00000050  eb eb da 00 17 17 17 17  17 18 17 17 ea e9 e9 e9  |................|
00000060  e9 e8 e9 e9 ed 00 ec 00  18 18 19 19 18 19 19 19  |................|
00000070  18 18 e8 e8 e8 e7 e7 e7  e8 e7 e8 e8 fa 01 20 00  |.............. .|
00000080  21 00 20 01 21 00 20 00  f8 1e f7 1e f7 1f f7 1e  |!. .!. .........|
00000090  da 00 e6 e5 e5 e5 e5 e4  e5 e5 1a 1b 1b 1b 1b 1c  |................|

Currently Melco distributes a different software for use with their embroidery machines. Their DesignShop software also works with the OFM format. Downloading a copy of version 11 and using the trial version I get access to a few OFM sample files. Let’s see if they are the same.

hexdump -C BUBBLEBOY1.ofm | head
00000000  52 49 46 46 86 e5 01 00  4f 46 4d 38 76 72 73 6e  |RIFF....OFM8vrsn|
00000010  08 00 00 00 39 00 2e 00  30 00 30 00 6e 6f 74 65  |....9...0.0.note|
00000020  a8 00 00 00 ff fe ff 52  44 00 69 00 67 00 69 00  |.......RD.i.g.i.|
00000030  74 00 69 00 7a 00 65 00  72 00 20 00 3a 00 20 00  |t.i.z.e.r. .:. .|
00000040  41 00 45 00 30 00 38 00  33 00 0d 00 0a 00 46 00  |A.E.0.8.3.....F.|
00000050  61 00 62 00 72 00 69 00  63 00 20 00 3a 00 20 00  |a.b.r.i.c. .:. .|
00000060  54 00 77 00 69 00 6c 00  6c 00 20 00 0d 00 0a 00  |T.w.i.l.l. .....|
00000070  4d 00 45 00 4c 00 43 00  4f 00 20 00 2d 00 20 00  |M.E.L.C.O. .-. .|
00000080  41 00 43 00 54 00 49 00  4f 00 4e 00 20 00 49 00  |A.C.T.I.O.N. .I.|
00000090  4c 00 4c 00 55 00 53 00  54 00 52 00 41 00 54 00  |L.L.U.S.T.R.A.T.|

Well that is very different than the earlier example. We can see right away this is a different type of file, in fact the first few bytes tells us this another container format. The Resource Interchange File Format, is used in many various file formats, the most popular are WAVE, AVI, and CorelDRAW. It is a chunk based format and there are a few tools we can use to look closer.

Riffpad can open the file, but claims there is some extra data at the end. It does see four chunks and it gives us the code “OFM8”, which is what identifies this particular RIFF type.

I was also able to get some samples of version 10 of DesignShop and found they are the same OLE container. Also has the same “EdsIV Object” within the container. There is a small paragraph in the EdsIV user manual that indicates there are some versioning within the OFM format.

If you open an EDS III .OFM file and save it, it will be converted into an EDS IV .OFM file, which is no longer readable in EDS III.
Files saved in this version of EDS IV cannot be read by previous versions of EDS IV.

This version of EDS IV is capable of producing two types of OFM files. Files saved as “Melco Project File (.ofm)” can only be read with this version or higher versions of EDS IV. Files saved as “Melco Version 2.00 (.ofm)” can be read by any EDS IV user that has version 2.00.006 or higher software.

It never ceases to amaze me how many formats use the Compound Object Container format. Seems like more and more are documented often. For now, I made a signature to identify the OLE and RIFF version of OFM. I’ll keep my eye out for the older EDS III and other related formats. As always, you can find my signatures and a sample file on my GitHub.