TurboTax

Posted on by Thor Leave a comment

With all the different file formats that are found in everyday computing, most formats which find their way to my archive have historical value. We know we can’t keep everything and have to assign value to all we decide to keep in for the long term. Some files have sensitive data and we have to follow guidelines for their proper handling. Identification of files helps us know what type of data might be kept inside the format, so often I need to also identify formats we don’t plan on keeping.

I was recently looking through a large digital collection and a report on the files which did not identify in the initial scan. A few popped out to me because of their extension, TAX. Tax records are one thing we need to identify so we can properly handle them, but not likely keep in our repository.

These tax files come from the popular US based TurboTax software. The software gets a new version for every year as tax laws are constantly changing. The software has also been around since 1984, so there are many versions to be aware of. Add to the fact there are personal and business versions along with DOS, Windows, and Macintosh versions, identification might get complicated. None of which are documented in the PRONOM registry. Wikidata is aware of a couple of the extensions, but does not have any signatures to help in identification.

Luckily, this collection of files I was processing had a number of years worth of records. Using them and a few others I was able to put together a decent timeline of formats used, at least from the early 1990’s on. The format seemed to settle on the .TAX extension around the 1994 Windows version. Before this, a group of files in DOS together stored the data. Let’s look at a sample of the 1994 file from Windows.

% hexdump -C TT1994.TAX | head
00000000  54 75 72 62 6f 54 61 78  0d 0a 46 6f 72 6d 61 74  |TurboTax..Format|
00000010  3d 57 49 4e 0d 0a 56 65  72 73 69 6f 6e 3d 31 33  |=WIN..Version=13|
00000020  0d 0a 45 6e 67 69 6e 65  56 65 72 73 53 74 72 3d  |..EngineVersStr=|
00000030  36 2e 30 30 2e 31 0d 0a  46 6f 72 6d 73 65 74 3d  |6.00.1..Formset=|
00000040  53 31 39 39 34 55 53 31  30 34 30 0d 0a 43 65 6e  |S1994US1040..Cen|
00000050  74 73 3d 59 65 73 0d 0a  53 68 6f 77 43 6f 6d 6d  |ts=Yes..ShowComm|
00000060  61 73 3d 59 65 73 0d 0a  53 68 6f 77 43 6f 6c 6c  |as=Yes..ShowColl|
00000070  61 70 73 69 62 6c 65 57  6f 72 6b 53 68 65 65 74  |apsibleWorkSheet|
00000080  73 3d 59 65 73 0d 0a 44  61 74 61 56 65 72 73 69  |s=Yes..DataVersi|
00000090  6f 6e 3d 31 0d 0a 46 6f  72 6d 46 69 6c 65 53 75  |on=1..FormFileSu|

I love these easy to identify format headers, but then jump to the next year, 1995, and the format changes.

% hexdump -C TT1995.TAX | head
00000000  c0 45 01 5f 0a 00 00 35  b5 06 36 2e 30 30 2e 31  |.E._...5..6.00.1|
00000010  00 00 c7 00 02 00 02 0d  00 00 00 b4 00 00 00 d9  |................|
00000020  00 0e 53 31 39 39 35 55  53 31 30 34 30 50 45 52  |..S1995US1040PER|
00000030  01 01 01 00 00 00 01 00  01 00 00 35 b5 00 0a c8  |...........5....|
00000040  00 01 00 01 09 00 00 00  cf 00 06 00 06 1d 00 00  |................|
00000050  00 3e 00 00 00 3e 00 00  00 64 00 00 00 64 00 00  |.>...>...d...d..|
00000060  00 7e 00 00 00 ce 13 7a  65 7a 50 65 72 73 69 73  |.~.....zezPersis|
00000070  74 65 6e 74 53 74 61 74  75 73 00 65 00 64 00 01  |tentStatus.e.d..|
00000080  00 00 00 00 00 00 ce 12  7a 74 6c 50 65 72 73 69  |........ztlPersi|
00000090  73 74 46 69 6c 65 44 61  74 61 00 00 00 00 00 00  |stFileData......|

The nice easy to read header is gone, but some other patterns start to appear. It seems most of the files from these early versions also used a code near the beginning that may help. “S1995US1040PER”, is similar to the “S1994US1040” in the 1994 file. One could assume the “1040” is the tax form most Americans are used to, along with “US” preceding the number. Then at the end of the string we see “PER”. This may refer to different versions of the Tax software, a Personal for the individual, and a possibly other versions for business as well. I believe TurboTax also had versions for Canadians as well, so there may be many variations on this string. This could get complex. Let’s jump ahead to a 1999 file.

% hexdump -C TurboTax1999.tax | head 
00000000  c0 45 01 5f 0a 00 00 54  6a 16 4c 39 31 30 32 31  |.E._...Tj.L91021|

00000030  00 0e 53 31 39 39 39 55  53 31 30 34 30 50 45 52  |..S1999US1040PER|
00000040  00 00 01 00 00 00 25 00  00 00 00 00 00 00 00 00  |......%.........|
00000050  01 19 12 8f f1 00 0a 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 c8  |................|
00000080  00 04 00 04 15 00 00 00  ec 05 00 00 c3 07 00 00  |................|
00000090  a3 08 00 00 c4 00 05 46  31 30 34 30 00 00 00 01  |.......F1040....|

The same string is visible, but if course with the year “1999”. We can also see a pattern with the first 4 bytes, “c0 45 01 5f” which seem to be consistent with the 1995 file. The file I have for 1998 is consistent as well. Jumping to the new millennium, we see a change.

% hexdump -C TurboTax2000.tax | head
00000000  c0 45 01 64 0a 00 00 2e  4f 18 4c 30 30 39 32 37  |.E.d....O.L00927|

00000030  00 dc 00 0b 53 32 30 30  30 55 53 31 31 32 30 00  |....S2000US1120.|
00000040  00 01 00 00 00 09 00 00  00 00 00 00 00 00 00 01  |................|

Two changes we see with this file. One, the ASCII string is different. S2000US1120, 1120 being the U.S. Corporation Income Tax Return. So this version of the software was different. The other change is the first 4 bytes. They changed to “c0 45 01 64”, with the last byte changing from 5F to 64. Jumping to 2003, we see the same values.

% hexdump -C TurboTax2003.tax | head 
00000000  c0 45 01 64 0d 00 00 80  1b 26 54 59 30 33 5f 4c  |.E.d.....&TY03_L|

00000040  58 03 00 dc 00 0e 53 32  30 30 33 55 53 31 30 34  |X.....S2003US104|
00000050  30 50 45 52 00 00 01 00  00 6a c6 00 00 00 00 00  |0PER.....j......|

Back to a 1040 form, but with the same header as the 2000 file. I am removing some lines, just to be safe and not exposing any personal data. In 2004 we see a major change in the format.

% hexdump -C TurboTax2004.tax | head 
00000000  54 54 46 4e 01 01 6f 68  dc 62 00 00 00 00 4b 01  |TTFN..oh.b....K.|

Again, removing some lines to ensure safety. This header is very different and their is no human readable ASCII in the file, which means it is binary and probably encoded. This header is new, TTFN is what I assume references TurboTax format? file? or possibly, “Turbo Tax Financial Network“?

This header is then used for the next few years ending in 2013, but before we get there, the extension makes a change as well. In 2008, instead of the simple .TAX extension, the software begins to save the tax file with the extension .TAX2008. I don’t have a 2008 document, but I do have a sample 2009 document.

% hexdump -C TurboTax2009.tax2009 | head
00000000  54 54 46 4e 01 01 b5 68  02 24 00 00 00 00 4b 0b  |TTFN...h.$....K.|
00000010  01 01 19 13 01 01 01 52  01 01 01 0b 01 01 4e 7a  |.......R......Nz|

With the last to use the TTFN header in 2013.

% hexdump -C TurboTax2013.tax2013 | head
00000000  54 54 46 4e 01 01 87 22  6a ec 00 00 00 00 50 bd  |TTFN..."j.....P.|

2014 is where I get a little confused. I have one file which uses the TTFN header and another which uses what becomes the standard going forward. But definitely in 2015, the format starts using the ZIP container as a structure for the format. Here is a sample from 2015

% hexdump -C TurboTax2015.tax2015 | head
00000000  50 4b 03 04 2d 00 02 00  08 00 e5 a6 51 48 ba 4d  |PK..-.......QH.M|
00000010  43 67 15 06 00 00 10 06  00 00 0c 00 14 00 6d 61  |Cg............ma|
00000020  6e 69 66 65 73 74 2e 78  6d 6c 01 00 10 00 00 00  |nifest.xml......|

If we take a look inside the ZIP container of a 2017 dummy sample.

% 7z l TurboTax2017.tax2017
7-Zip [64] 17.05 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.05 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

Scanning the drive for archives:
1 file, 769814 bytes (752 KiB)                                              

Listing archive: TurboTax2017.tax2017

--
Path = TurboTax2017.tax2017
Type = zip
WARNINGS:
Headers Error
Physical Size = 769814

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2026-03-28 20:25:38 .....          576          581  manifest.xml
2026-03-28 20:25:38 .....       768688       768923  084A702A-CD3D-4623-B8B7-EE4800BB151F
------------------- ----- ------------ ------------  ------------------------
2026-03-28 20:25:38             769264       769504  2 files

Warnings: 1

The files all seem to have a manifest.xml and a unique identifier. 7-Zip also mentions a header issue with the ZIP files. Something maybe done on purpose? Now comes the odd part, the manifest.xml file does not render as an XML file, it is binary.

% hexdump -C TurboTax2017/manifest.xml | head
00000000  a1 b1 fe fb 37 18 dd 9c  08 2d 9c 86 23 00 10 fa  |....7....-..#...|
00000010  12 60 92 bb dc 92 a5 df  1a 24 16 4e a9 28 89 80  |.`.......$.N.(..|
00000020  64 33 66 55 c5 93 f0 68  44 d0 7c f9 56 86 42 2c  |d3fU...hD.|.V.B,|
00000030  80 ba 8a 95 2a 82 6d 32  75 84 b1 f1 e2 18 93 5c  |....*.m2u......\|
00000040  82 4d 18 f9 ed 23 4f dc  d6 b5 7f f2 20 1e 30 59  |.M...#O..... .0Y|
00000050  d5 7f 47 7d aa f5 8d bd  8b 10 20 ec 8a c7 43 df  |..G}...... ...C.|
00000060  52 90 a9 70 4d 68 b4 76  fa c8 37 85 f5 56 25 82  |R..pMh.v..7..V%.|
00000070  ea 16 06 54 b0 b4 bc 43  16 fb 70 7b 7a 79 a5 8b  |...T...C..p{zy..|
00000080  3c 79 7d ef ac 32 fc 35  ce 0f fa a2 6f e7 c3 a4  |<y}..2.5....o...|
00000090  92 a1 a4 c8 83 dd 9f 32  f4 ea d3 1a eb 89 15 a3  |.......2........|

Of the samples I have which have a manifest.xml, they all begin with “a1 b1 fe fb”. Which apparently is the header for an AES CBC encrypted file. A clever person was able to decrypt the file to reveal the actual XML.

TurboTax isn’t sold on physical disk anymore, but you can download the current tax year version from their website. I am not a user of their product so I am not sure if the latest version still saves files in the same way. If you do use it currently, I would love to know if it is still the same.

So to recap, the headers are:

  • 1994 “TurboTax Format=WIN Version=13
  • 1995-99 “C045015F”
  • 2000-03 “C0450164”
  • 2004-13 “TTFN”
  • 2014-current “ZIP Container”

This should be enough to create five new signatures for identification. Extensions will be a problem since they change very year, but we can add them to the list. With these signatures we can now identify all the tax files we have and set them aside if not needed.

DaVinci Resolve

Posted on by Thor Leave a comment

A previous post was about LUTs, the little files needed to color grade your photo’s and video’s. One of the best systems for color grading video in use by professionals today is DaVinci Resolve. The system originally was all hardware based, but in the 2004 as computers were able to process higher quality video, da Vinci Systems released new digital systems.

Like most professional multimedia editing software, projects are used to manage work and DaVinci Resolve is no different. Projects are generally where all the settings for the project are stored, but don’t generally store the actual media used in the project. Project files are often XML with unique schema’s, but other pack a little more into the project file. 

hexdump -C project.drp | head
00000000  50 4b 03 04 14 00 08 00  08 00 f2 54 90 5a ef 18  |PK.........T.Z..|
00000010  b0 25 47 0c 00 00 db 1b  00 00 0b 00 00 00 70 72  |.%G...........pr|
00000020  6f 6a 65 63 74 2e 78 6d  6c 9d 58 d9 72 5b 37 12  |oject.xml.X.r[7.|
00000030  7d cf 57 68 f4 7e 4d ec  4b 8a 51 ca b1 92 89 aa  |}.Wh.~M.K.Q.....|
00000040  2c db 65 29 79 9d 6a 00  0d 85 09 45 aa 48 4a 71  |,.e)y.j....E.HJq|
00000050  fe 7e 0e ee 42 51 94 9c  68 c6 29 85 17 0d a0 d1  |.~..BQ..h.).....|
00000060  e8 3e bd 61 fe fd 97 db  e5 c9 03 6f b6 8b f5 ea  |.>.a.......o....|
00000070  bb 53 f9 46 9c 9e f0 2a  af cb 62 75 f3 dd e9 2f  |.S.F...*..bu.../|
00000080  d7 3f 75 e1 f4 fb b3 6f  e6 ff ea ba f3 f4 f6 ee  |.?u....o........|
00000090  ee 57 de 60 55 7c 23 df  98 37 42 48 79 7a 72 9e  |.W.`U|#..7BHyzr.|

DaVinci Resolve keeps all projects in a database, but you can export them to a project file. A DaVinci Resolve Project file uses a ZIP container to store all the project settings in one file. Let’s see what also might be inside.

Path = project.drp
Type = zip
Physical Size = 543860

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2018-02-27 20:25:08 .....      1010030       287793  project.xml
2018-02-27 20:25:08 .....        21173         6856  MediaPool/Master/000_Timelines/MpFolder.xml
2018-02-27 20:25:08 .....       492690        28067  MediaPool/Master/001_Audio/MpFolder.xml
2018-02-27 20:25:08 .....        20177         3588  MediaPool/Master/002_gfx/MpFolder.xml
2018-02-27 20:25:08 .....        11025         2611  MediaPool/Master/003_VO/MpFolder.xml
2018-02-27 20:25:08 .....        98309         7042  MediaPool/Master/004_ScreenCaptures Consolidated/MpFolder.xml
2018-02-27 20:25:08 .....      1278493        66424  MediaPool/Master/005_Video H264/MpFolder.xml
2018-02-27 20:25:08 .....         1995          748  MediaPool/Master/MpFolder.xml
2018-02-27 20:25:08 .....      1638204       137086  SeqContainer/909a0a2c-4183-4310-9f78-6e15c3c59cb4.xml
2018-02-27 20:25:08 .....         8806         1169  Gallery.xml
2018-02-27 20:25:08 .....        12697          696  media.dat
------------------- ----- ------------ ------------  ------------------------
2018-02-27 20:25:08            4593599       542080  11 files

Looks like a lot of XML! The consistent XML in all the DRP files is the apply named “project.xml” along with “Gallery.xml”.

cat project.xml | head
<?xml version="1.0" encoding="UTF-8"?>
<!--DbAppVer="19.1.4.0011" DbPrjVer="14"-->
<SM_Project DbId="db65f2ee-2bff-41cd-b478-f96c26e9609f">
 <FieldsBlob>000000010000000700000026005400650078007400520065006e006400650072004900740065006d005600650063004200410000000c00ffffffff0000002400520065006e0064006500720043006100630068006500560065007200730069006f006e0000000200000000010000001e00500072006f006a00650063007400460065006100740075007200650073000000050000000000000000010000002e00500072006f006a00650063007400440062004d006900670072006100740069006f006e00530074006100740065000000040000000000000000030000002e0049007300500072006f006a0065006300740041006700650049006e004d006900630072006f00530065006300730000000100010000001400470061006c006c0065007200790052006500660000000a000000004800330033003400320034003300380036002d0034006400330030002d0034003600610035002d0061006100340033002d006100330035003200620066006500370038003200640063000000260046007500730069006f006e00530069007a0069006e006700560065007200730069006f006e000000020000000002</FieldsBlob>
 <LockId/>
 <User>86f03abc-9354-47d9-9006-a55b6b1d49cf</User>
 <Folder/>
 <UserId>-1</UserId>
 <SysId>6CB133A11B81</SysId>
 <ProjectId>0</ProjectId>

It appears the version of DaVinci Resolve is pretty important. If you try and open a DRP file without using the most up-to-date software you might run into problems. From what I can see, every time a new major version is released, the updates to the XML cause the project error when imported. So knowing the version of the DRP file can be a critical piece of metadata needed in understanding the format. There are some helpful apps created by DaVinci Resolve users you can try, or you can try a little python script to report back the version used in a DRP or whole folder of DRP files.

There is one other file used by the DaVinci Resolve software. It uses the DRT extension and is for exporting and importing single timelines to the software. Like a DRP it is a simple project file that only points to the media used in the project and only stores the settings needed. 

Path = timeline.drt
Type = zip
Physical Size = 215159

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2021-04-21 21:16:42 .....        45726         8888  project.xml
2021-04-21 21:16:42 .....       670306       198698  MediaPool/Master/MpFolder.xml
2021-04-21 21:16:42 .....        98268         7089  SeqContainer/7eb849f3-41cb-4e3f-baa8-d5b134b57aa7.xml
------------------- ----- ------------ ------------  ------------------------
2021-04-21 21:16:42             814300       214675  3 files

This DRT file also has a project.xml file, but doesn’t have the Gallery.xml file we normally find in a DRP file. We can use this to distinguish the difference. The project.xml is the same as the DRP, so this distinction is important. 

cat project.xml |head
<?xml version="1.0" encoding="UTF-8"?>
<!--DbAppVer="17.1.1.0009" DbPrjVer="10"-->
<SM_Project DbId="ec6cb2e2-0b3c-43b8-8f90-a5fcb973af3b">
 <FieldsBlob>00000001000000040000002e00500072006f006a00650063007400440062004d006900670072006100740069006f006e00530074006100740065000000040000000000000000020000002e0049007300500072006f006a0065006300740041006700650049006e004d006900630072006f00530065006300730000000100010000001400470061006c006c0065007200790052006500660000000a000000004800660030003800380038003300390038002d0066006400620037002d0034006300320036002d0061003700310032002d003300360038006200300036003300300065003400330031000000260046007500730069006f006e00530069007a0069006e006700560065007200730069006f006e000000020000000002</FieldsBlob>
 <LockId/>
 <User>04d71873-a504-40c6-bde5-41709691a2c9</User>
 <Folder/>
 <UserId>-1</UserId>
 <SysId>94F6D6F3F60F</SysId>
 <ProjectId>0</ProjectId>

In both formats they use the XML root tag of “SM_Project”, this can also be used to define a signature for the two formats as “project.xml” could be used with a different format and we don’t want there to be a false identification.

I was able to trace back the use of the DRP format back to DaVinci Resolve version 9. In version 8, it appears projects are exported using the name and extension, “Default Project.resolve.zip”. From what I could find, DaVinci Resolve version 9 was a big re-write and so it makes sense to settle on more useful extension. The project.xml file in a version 8 format is slightly different.

cat project.xml | head
<SM_Project DbId="9ba0c4dc-d99c-4b7f-b0da-d254d91e34e2" DbAppVer="8.2 (#153)">
 <LockId></LockId>
 <User>159415b8-7515-43bf-b5f5-00d98949434b</User>
 <UserId>-1</UserId>
 <SysId>7cd1c388ea29</SysId>
 <ProjectId>0</ProjectId>
 <RevivalTaskSetID>-1</RevivalTaskSetID>
 <PlayHeadsSplitDisplay>false</PlayHeadsSplitDisplay>
 <pGallery>
  <Gallery::GyGallery DbId="9884d8ff-096e-4df0-b833-0e75e6e07e15">

Still uses the “SM_Project” root tag, but displays the DbAppVer information differently. It would be good to find more examples of the version 8 and earlier to see how this format has evolved over time. For now, I have created a signature you can test if you happen to have any DRP files in your archive.

Designer

Posted on by Thor Leave a comment

Micrografx / Corel Designer

Many software titles we have all used began life under a different brand or even title. Larger software companies gobble up smaller developers, some brands merge, and others change names for whatever reason. Adobe has bought many smaller companies over the years, sometimes developing the acquired software and other times burying the software to avoid competition. Pagemaker was bought to give InDesign life, many Macromedia titles were incorporated or shelved. Such is life in the software world.

In understanding a file format, often times you need to follow this trail backwards to understand when file formats changed and compatibility is dropped. Often times the formats remained the same, but the extension is changed. Or the software name changes and formats are updated, but the extension remains the same. There can also be multiple titles which all use a common format, further complicating the identification of the formats.

Let’s look closer at the a title which changed names and file formats a few times over the years. Micrografx was founded in 1982 and were pretty well known for their innovation in computer graphics. They have released many titles over the years, but one of the first was In*A*Vision graphic software for Windows 1.0 in 1986. This software used a format with the .PIC extension. A couple years later version 2, was renamed to Micrografx Designer and used the .DRW extension. This extension was also used by Micrografx Draw, another similar program.

Micrografx Designer continued to be released until version 9 which is when it was purchased by Corel who continued to release new versions, although it is said the software was just a variation of CorelDraw, and now Designer is part of the CorelDraw Technical Suite. Other Micrografx software such as Picture Publisher was discontinued and customers were encouraged to use Corel’s PaintShop Pro instead. Somewhere in the middle of all this, Micrografx spun off a separate business unit called iGrafx, which Designer was marketed under for a short time.

Let’s break down the names, extensions used, and format type.

  • In*A*Vision & Draw, binary format, PIC extension
  • Micrografx Designer & Draw, binary format, DRW extension
  • Micrografx Designer version 4, RIFF format, DS4 & MGX extension
  • Micrografx Designer versions 6-9, OLE Container format, DSF extension
  • Micrografx/Corel Designer versions 10-12, RIFF format, DES extension
  • Corel Designer version X4-Current, ZIP/XML format, DES extension

According to the 2021 Corel DesignerUser Guide:

Corel DESIGNER (DES, DSF, DS4, or DRW)

You can import Corel DESIGNER files. Files from version 10 and later have the filename extension .des. Files from Micrografx versions 6 to 9 have the filename extension .dsf. Version 4 files have the filename extension .ds4. The .drw filename extension is used for a Micrografx 2.x or 3.x file. Micrografx template files (DST) are also supported.

The PRONOM registry has a few of these formats with signatures and documented, but not all, let’s see where the gaps are.

PUIDFormat NameFormat VersionExtension
x-fmt/151 Micrografx Designer dsf
x-fmt/296 Micrografx Designer 3.1drw
x-fmt/47 Micrografx Draw 1-2drw
x-fmt/294 Micrografx Draw 3drw
x-fmt/295 Micrografx Draw 4drw, drt
fmt/1907Micrografx Icon File icn
fmt/1481Micrografx In-A-Vision Drawingpic

So from the PRONOM list, it appears we have good identification on the original PIC and DRW formats. Then the Designer DSF OLE container is taken care of as well. That leaves us with DS4 and DES formats.

hexdump -C DS41-S01.DS4 | head
00000000  52 49 46 46 6e 07 00 00  4d 47 58 20 69 74 70 64  |RIFFn...MGX itpd|
00000010  04 00 00 00 00 02 00 80  70 72 6f 70 23 00 00 00  |........prop#...|
00000020  1f 00 00 30 02 00 00 00  08 00 2c 40 44 00 11 20  |...0......,@D.. |
00000030  20 00 01 10 80 e0 00 00  91 08 21 e0 5c 82 90 72  | .........!.\..r|
00000040  05 ff c0 00 4c 49 53 54  10 04 00 00 64 69 74 6e  |....LIST....ditn|
00000050  74 68 6e 6c 03 04 00 00  57 01 00 30 00 00 08 00  |thnl....W..0....|
00000060  08 00 00 41 04 00 01 20  a4 00 82 10 72 14 40 48  |...A... ....r.@H|
00000070  00 58 20 84 04 32 10 40  00 12 c8 98 18 22 63 90  |.X ..2.@....."c.|
00000080  2b 91 32 36 47 08 20 c0  23 e4 80 90 92 22 46 49  |+.26G. .#...."FI|
00000090  09 29 26 24 e4 a0 94 92  a2 56 4b 09 69 2e 25 e4  |.)&$.....VK.i.%.|

Micrografx Designer 4 apparently uses the RIFF container format. The RIFF format is used with many different types of formats. The most common is the WAV format. CorelDRAW also uses the RIFF format so it makes sense they would use it as they took over from Micrografx.

Each RIFF format has a four byte identifier type after the first eight bytes which identify the RIFF. The DS4 file uses the code “MGX ” to identify itself. Which also appears to be used with their clipart format, MGX. We can use the same identification method we use for other RIFF’s to identify this format. 

hexdump -C Corel-DES10Sample.des | head
00000000  52 49 46 46 8a 57 00 00  44 45 53 41 76 72 73 6e  |RIFF.W..DESAvrsn|
00000010  02 00 00 00 7e 04 4c 49  53 54 54 0c 00 00 69 63  |....~.LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Starting with version 10 of Corel Designer, the RIFF format is used again and has a different type. With Version 10 using “DESA”, then for version 10.5:

hexdump -C Corel-DES10.5Sample.des | head 
00000000  52 49 46 46 cc 57 00 00  44 45 53 42 76 72 73 6e  |RIFF.W..DESBvrsn|
00000010  02 00 00 00 b0 04 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

The next version after 10.5 is version 12 and it shows a type:

hexdump -C Corel-DES12-Sample.des | head 
00000000  52 49 46 46 ce 57 00 00  44 45 53 43 76 72 73 6e  |RIFF.W..DESCvrsn|
00000010  02 00 00 00 e2 04 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

After version 12, Corel started using numbering consistent with their other products. The first being X4.

hexdump -C Corel-DESX4-Sample.des | head
00000000  50 4b 03 04 14 00 00 08  00 00 f8 bb c9 4e c3 4b  |PK...........N.K|
00000010  9c d1 2d 00 00 00 2d 00  00 00 08 00 00 00 6d 69  |..-...-.......mi|
00000020  6d 65 74 79 70 65 61 70  70 6c 69 63 61 74 69 6f  |metypeapplicatio|
00000030  6e 2f 78 2d 76 6e 64 2e  63 6f 72 65 6c 2e 64 65  |n/x-vnd.corel.de|
00000040  73 69 67 6e 65 72 2e 64  6f 63 75 6d 65 6e 74 2b  |signer.document+|
00000050  7a 69 70 50 4b 03 04 14  00 00 08 00 00 f8 bb c9  |zipPK...........|
00000060  4e 6f 38 b6 64 98 13 00  00 98 13 00 00 14 00 00  |No8.d...........|
00000070  00 63 6f 6e 74 65 6e 74  2f 72 69 66 66 44 61 74  |.content/riffDat|
00000080  61 2e 63 64 72 52 49 46  46 90 13 00 00 44 45 53  |a.cdrRIFF....DES|
00000090  45 76 72 73 6e 02 00 00  00 82 05 4c 49 53 54 54  |Evrsn......LISTT|

Well it looks like things changed, starting with X4 the format changed to a ZIP container. Let’s take a peak inside.

Path = Corel-DESX4-Sample.des
Type = zip
Physical Size = 8714

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:47 .....           45           45  mimetype
2019-06-09 22:31:47 .....         5016         5016  content/riffData.cdr
2019-06-09 22:31:47 .....       196662          239  metadata/thumbnails/thumbnail.bmp
2019-06-09 22:31:47 .....       151606          698  metadata/thumbnails/page1.bmp
2019-06-09 22:31:47 .....          596          259  metadata/textinfo.xml
2019-06-09 22:31:47 .....         4977         1314  metadata/metadata.xml
2019-06-09 22:31:47 .....           53           55  links.xml
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:47             358955         7626  7 files

Looks like the container holds a RIFF inside along with some thumbnails, metadata, and other things. The mimetype file simple holds “application/x-vnd.corel.designer.document+zip”. The riffData.cdr however looks like this:

hexdump -C Corel-DESX4-Sample/content/riffData.cdr | head
00000000  52 49 46 46 90 13 00 00  44 45 53 45 76 72 73 6e  |RIFF....DESEvrsn|
00000010  02 00 00 00 82 05 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Another RIFF, and seems to be in the same sequence, but going from version 12 to X4 we seemed to have skipped “DESD”. Maybe there was a developer version in between as they transitioned. Version X5 looks similar and has the RIFF sequence “DESF”. When we get to X6 the structure changes. 

Path = Corel-DESX6-Sample.des
Type = zip
Physical Size = 8568

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:21 .....           45           45  mimetype
2019-06-09 22:31:21 .....        12153         1098  content/data/data1.dat
2019-06-09 22:31:21 .....          439          224  content/data/masterPage.dat
2019-06-09 22:31:21 .....          613          265  content/data/page1.dat
2019-06-09 22:31:21 .....           34           28  content/dataFileList.dat
2019-06-09 22:31:21 .....          960          279  content/root.dat
2019-06-09 22:31:21 .....       196662          239  metadata/thumbnails/thumbnail.bmp
2019-06-09 22:31:21 .....       151606          698  metadata/thumbnails/page1.bmp
2019-06-09 22:31:21 .....          427          208  color/color.xml
2019-06-09 22:31:21 .....          596          259  metadata/textinfo.xml
2019-06-09 22:31:21 .....          103          100  color/docPalette.xml
2019-06-09 22:31:21 .....        14920         1444  styles/document.cdss
2019-06-09 22:31:21 .....         5500         1462  metadata/metadata.xml
2019-06-09 22:31:21 .....           53           55  links.xml
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:21             384111         6404  14 files

The mimetype remains the same, but we see additional files within the structure. Also the riffData.cdr file is missing. Looking at each file we can see the root.dat file is a RIFF and follows the same sequence. 

hexdump -C Corel-DESX6-Sample/content/root.dat | head
00000000  52 49 46 46 b8 03 00 00  44 45 53 47 66 76 65 72  |RIFF....DESGfver|
00000010  10 00 00 00 ff ff ff ff  08 00 00 00 5e 06 02 00  |............^...|
00000020  00 00 10 00 76 72 73 6e  10 00 00 00 ff ff ff ff  |....vrsn........|
00000030  02 00 00 00 5e 06 00 00  00 00 00 00 4c 49 53 54  |....^.......LIST|
00000040  7c 00 00 00 64 6f 63 20  6d 63 66 67 10 00 00 00  ||...doc mcfg....|
00000050  00 00 00 00 83 20 00 00  00 00 00 00 00 00 00 00  |..... ..........|
00000060  70 72 65 66 10 00 00 00  00 00 00 00 e6 0e 00 00  |pref............|
00000070  83 20 00 00 00 00 00 00  70 74 72 74 10 00 00 00  |. ......ptrt....|
00000080  00 00 00 00 10 00 00 00  69 2f 00 00 00 00 00 00  |........i/......|
00000090  4c 49 53 54 04 00 00 00  66 69 6c 74 4c 49 53 54  |LIST....filtLIST|

As we get to a more recent version. We can see the pattern continues.

hexdump -C Designer2022-s01/content/root.dat | head
00000000  52 49 46 46 88 06 00 00  44 45 53 4e 66 76 65 72  |RIFF....DESNfver|
00000010  10 00 00 00 ff ff ff ff  08 00 00 00 60 09 02 00  |............`...|
00000020  00 00 18 00 76 72 73 6e  10 00 00 00 ff ff ff ff  |....vrsn........|
00000030  02 00 00 00 60 09 00 00  00 00 00 00 4c 49 53 54  |....`.......LIST|
00000040  30 01 00 00 64 6f 63 20  6d 63 66 67 10 00 00 00  |0...doc mcfg....|
00000050  00 00 00 00 08 1f 00 00  00 00 00 00 00 00 00 00  |................|
00000060  70 72 65 66 10 00 00 00  00 00 00 00 ae 07 00 00  |pref............|
00000070  08 1f 00 00 00 00 00 00  70 74 72 74 10 00 00 00  |........ptrt....|
00000080  00 00 00 00 10 00 00 00  b6 26 00 00 00 00 00 00  |.........&......|
00000090  4c 49 53 54 4c 00 00 00  66 6e 74 74 66 6f 6e 74  |LISTL...fnttfont|

The last sample I have is for Corel Designer 2022, but there could be more. I created new signatures for all the samples I have, you can see them in my Github as usual. I decided to group some of the versions together to simplify things a bit, but if anyone thinks they should be broken out into individual versions, let me know.

Gone in a Flash

Posted on by Thor Leave a comment

This week I am at the annual iPres digital preservation conference. It is an amazing week of meeting colleagues and old friends who share the same passion of digital preservation. Outside of this community and my co-workers, talking about file formats and digital preservation usually bores people to death and I can hear some of them mumble under their breath, “nerd”! I term I am happy to accept.

At the conference, which is in lovely Urbana-Champaign Illinois this year, I am trying to soak in all the amazing talks and conversations about the challenges facing our work. There were a couple great workshops on Persistent Identifiers and Digital Object Storage Criteria. The presentations I made were of course on File Formats, documentation, and obsolescence. One talk before my panel conversation was about the ubiquitous Adobe Flash format.

The paper, “Around for Decades, Gone in a Flash: How we dealt with Flash objects and the National Archives of the Netherlands” was presented by Lotte Wijsman and Marin Rappard. They knew they had flash objects in their web archives and wanted to go through the process of how they might be preserved and accessed. They started out looking for any files with “FLA”, “SWF”, and “FLV” as extensions. This proved problematic as there were references to those extensions within other documents and objects. They then used DROID to identify the flash formats. “SWF” has quite a number of format PUID’s.

PUIDFormat NameFormat VersionExtension
fmt/104Macromedia Flash1swf,
fmt/105Macromedia Flash2swf,
fmt/106Macromedia Flash3swf,
fmt/107Macromedia Flash4swf,
fmt/108Macromedia Flash5swf,
fmt/109Macromedia Flash6swf,
fmt/110Macromedia Flash7swf,
fmt/505Adobe Flash8swf,
fmt/506Adobe Flash9swf,
fmt/507Adobe Flash10swf,
fmt/757Adobe Flash11swf,
fmt/758Adobe Flash12swf,
fmt/759Adobe Flash13swf,
fmt/760Adobe Flash14swf,
fmt/761Adobe Flash15swf,
fmt/762Adobe Flash16swf,
fmt/763Adobe Flash17swf,
fmt/764Adobe Flash18swf,
fmt/765Adobe Flash19swf,
fmt/766Adobe Flash20swf,
fmt/767Adobe Flash21swf,
fmt/768Adobe Flash22swf,
fmt/769Adobe Flash23swf,
fmt/770Adobe Flash24swf,
fmt/771Adobe Flash25swf,
fmt/772Adobe Flash26swf,
fmt/773Adobe Flash27swf,
fmt/774Adobe Flash28swf,
fmt/775Adobe Flash29swf,
fmt/776Adobe Flash30swf,

Even the Macromedia/Adobe Flash Video format has a PRONOM PUID, x-fmt/382.

The format missing from PRONOM is the FLA format. FLA is the native format for Macromedia/Adobe Flash for saving the source project of your Flash document. SWF files are compiled from the FLA source. This means the the SWF will be the most common format found on the web and in public places, but the FLA format might be more often found on local drives and working folders.

The Flash format and software was actually created by Future Wave software in 1996 as FutureSplash Animator, but was shortly bought by Macromedia later that year and Flash 1.0 was born. FutureSplash used the extension .SPA, but Macromedia changed it to FLA.

The format was initially based on the Microsoft Compound File Format or the OLE container format. 

oledir Flash4-S01.fla 
oledir 0.54 - http://decalage.info/python/oletools
OLE directory entries in file Flash4-S01.fla:
----+------+-------+----------------------+-----+-----+-----+--------+------
id  |Status|Type   |Name                  |Left |Right|Child|1st Sect|Size  
----+------+-------+----------------------+-----+-----+-----+--------+------
0   |<Used>|Root   |Root Entry            |-    |-    |1    |5       |4416  
1   |<Used>|Stream |Contents              |2    |-    |-    |6       |4013  
2   |<Used>|Stream |Page 1                |-    |-    |-    |0       |329   
3   |unused|Empty  |                      |-    |-    |-    |0       |0     
----+----------------------------+------+--------------------------------------
id  |Name                        |Size  |CLSID                                 
----+----------------------------+------+--------------------------------------
0   |Root Entry                  |-     |597CAA70-72AA-11CF-831E-524153480000  
1   |Contents                    |4013  |                                      
2   |Page 1                      |329   |

The FLA format stayed with OLE until Adobe Flash CS5, which the format changed to use a ZIP container to store all the content.

Flash5.5-S01.fla
Type = zip
Physical Size = 216632

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2022-07-09 11:57:46 .....           25           25  mimetype
2022-07-09 11:57:46 .....            9            9  Flash5.5-S01.xfl
2022-07-09 11:57:46 D....            0            0  LIBRARY
2022-07-09 11:57:46 D....            0            0  META-INF
2022-07-09 11:57:46 .....        49267         3936  DOMDocument.xml
2022-07-09 11:57:48 .....         9735         1103  META-INF/metadata.xml
2022-07-09 11:57:48 .....         8093         2222  PublishSettings.xml
2022-07-09 11:57:48 .....            0            0  MobileSettings.xml
2022-07-09 11:57:48 D....            0            0  LIBRARY/Mouth shape graphic symbols
2022-07-09 11:57:48 D....            0            0  LIBRARY/Voice
2022-07-09 11:57:48 .....       151006       151006  bin/M 1 1252032698.dat
2022-07-09 11:57:48 .....        99707        15311  LIBRARY/mouth.xml
2022-07-09 11:57:48 .....        16510         4534  LIBRARY/Mouth shape graphic symbols/A I.xml
2022-07-09 11:57:48 .....        14334         4086  LIBRARY/Mouth shape graphic symbols/C D G K N R S Th Y Z.xml
2022-07-09 11:57:48 .....        14531         4040  LIBRARY/Mouth shape graphic symbols/E.xml
2022-07-09 11:57:48 .....        15846         4007  LIBRARY/Mouth shape graphic symbols/F V D Th.xml
2022-07-09 11:57:48 .....        13093         3542  LIBRARY/Mouth shape graphic symbols/L D Th.xml
2022-07-09 11:57:48 .....         2106          751  LIBRARY/Mouth shape graphic symbols/M B P Closed.xml
2022-07-09 11:57:48 .....        14130         3949  LIBRARY/Mouth shape graphic symbols/O.xml
2022-07-09 11:57:48 .....        11082         2951  LIBRARY/Mouth shape graphic symbols/Open_Rest.xml
2022-07-09 11:57:48 .....        14847         4066  LIBRARY/Mouth shape graphic symbols/U.xml
2022-07-09 11:57:48 .....         8139         2202  LIBRARY/Mouth shape graphic symbols/W Q.xml
2022-07-09 11:57:48 .....        15768         3914  LIBRARY/panda.xml
2022-07-09 11:57:48 .....        10477         1064  LIBRARY/sample graph.xml
2022-07-09 11:57:48 .....          538          538  bin/SymDepend.cache
------------------- ----- ------------ ------------  ------------------------
2022-07-09 11:57:48             469243       213256  21 files, 4 folders

The move to a ZIP container included a new format, XFL. This XFL file is a simple text file with the text “PROXY-CS5″. In the DOMDocument.xml file we find an XML namespace, xmlns=”http://ns.adobe.com/xfl/2008/” and a version of the XFL structure, xflVersion=”2.1″.

This ZIP compressed FLA file is still being used in the current Adobe Animate software, which no longer uses the flash technology and uses more modern web formats like HTML5 to display the animations.

I took each version and made a PRONOM signature, which you can find here with samples. These container signatures should cover all the major changes for the format, but there is a problem……..

Listing archive: Flash5.5-S01v5.fla

--
Path = Flash5.5-S01v5.fla
Type = zip
ERRORS:
Headers Error
Physical Size = 216581
Embedded Stub Size = 63
Characteristics = Local

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2022-07-09 11:57:46 .....           25           25  mimetype
2022-07-09 11:57:46 D....            0            0  LIBRARY
2022-07-09 11:57:46 D....            0            0  META-INF
2022-07-09 11:57:46 .....        48556         3742  DOMDocument.xml
2022-07-09 11:57:48 .....        10133         1112  META-INF/metadata.xml
2022-07-09 11:57:48 .....         8115         2219  PublishSettings.xml
2022-07-09 11:57:48 .....            0            0  MobileSettings.xml
2022-07-09 11:57:48 D....            0            0  LIBRARY/Mouth shape graphic symbols
2022-07-09 11:57:48 D....            0            0  LIBRARY/Voice
2022-07-09 11:57:48 .....       151006       151006  bin/M 1 1252032698.dat
2022-07-09 11:57:48 .....        99551        15319  LIBRARY/mouth.xml
2022-07-09 11:57:48 .....        16580         4536  LIBRARY/Mouth shape graphic symbols/A I.xml
2022-07-09 11:57:48 .....        14404         4089  LIBRARY/Mouth shape graphic symbols/C D G K N R S Th Y Z.xml
2022-07-09 11:57:48 .....        14531         4044  LIBRARY/Mouth shape graphic symbols/E.xml
2022-07-09 11:57:48 .....        15848         4008  LIBRARY/Mouth shape graphic symbols/F V D Th.xml
2022-07-09 11:57:48 .....        13024         3546  LIBRARY/Mouth shape graphic symbols/L D Th.xml
2022-07-09 11:57:48 .....         2106          752  LIBRARY/Mouth shape graphic symbols/M B P Closed.xml
2022-07-09 11:57:48 .....        14200         3955  LIBRARY/Mouth shape graphic symbols/O.xml
2022-07-09 11:57:48 .....        11152         2963  LIBRARY/Mouth shape graphic symbols/Open_Rest.xml
2022-07-09 11:57:48 .....        14777         4069  LIBRARY/Mouth shape graphic symbols/U.xml
2022-07-09 11:57:48 .....         8287         2228  LIBRARY/Mouth shape graphic symbols/W Q.xml
2022-07-09 11:57:48 .....        15768         3914  LIBRARY/panda.xml
2022-07-09 11:57:48 .....        10477         1064  LIBRARY/sample graph.xml
2022-07-09 11:57:48 .....          538          538  bin/SymDepend.cache
2022-07-09 11:57:46 .....           25           25  mimetype
2022-07-09 11:58:18 .....            9            9  Flash5.5-S01v5.xfl
------------------- ----- ------------ ------------  ------------------------
2022-07-09 11:58:18             469112       213163  22 files, 4 folders

Turns out majority of the samples I have from many versions of Adobe Flash after CS5 have a ZIP Header error. When using the new signatures in DROID, the samples with the header errors will fail in the DROID’s zip library processing. The DROID logs shows this issue:

Could not process the potential container format (ZIP): file:///Flash5.5-S01v5.fla	
Expected 25 more entries in the Central Directory!

The Central Directory header in a ZIP file is quite important to the proper function of the ZIP container. Wikipedia has a great explanation of the header. You may notice in the listing above the file “mimetype” is shown twice which is probably the extra entries the parser wasn’t expecting.

So currently the identification of majority of these FLA formats is on hold until a way is discovered to ignore the error and continue the container identification in DROID.