MAGIX

March 22, 2024 by Thor Leave a comment

There are probably many reasons why a software developer might want to create a proprietary format to store their files in. The software may require special features that don’t fit into an existing format. I would hope a developer would try to use existing formats, or even better open formats, but for many reasons, which probably include profits, they choose to re-invent the wheel often.

MAGIX is a German company which started making software in 1994. In 2001 they developed their first video editing software which was called Movie Edit Pro. The software seems to be well received and is still in use today.

Like most video editing software, project files are used to store all the edits and links to video files. These are usually smaller text based, with many using XML as the project format. Not MAGIX, they decided to go with a different yet known format for their project files.

hexdump -C MAGIX15-s01.MVP | head
00000000  52 49 46 46 6c 37 01 00  53 45 4b 44 4d 56 50 48  |RIFFl7..SEKDMVPH|
00000010  08 00 00 00 00 00 00 00  00 00 00 00 4c 49 53 54  |............LIST|
00000020  0c 16 01 00 4d 56 50 4c  4c 49 53 54 00 16 01 00  |....MVPLLIST....|
00000030  56 49 50 4c 53 56 49 50  0c 07 00 00 00 dc 05 00  |VIPLSVIP........|
00000040  00 00 00 00 20 00 00 00  0c 00 00 00 80 bb 00 00  |.... ...........|
00000050  10 00 00 00 29 6b 55 e2  53 f8 3d 40 00 00 f0 42  |....)kU.S.=@...B|
00000060  01 00 00 00 bd 04 ef fe  00 00 01 00 06 00 08 00  |................|
00000070  00 00 01 00 06 00 08 00  00 00 01 00 3f 00 00 00  |............?...|
00000080  28 00 00 00 04 00 04 00  01 00 00 00 00 00 00 00  |(...............|
00000090  00 00 00 00 00 00 00 00  bd 8f 32 01 d0 02 00 00  |..........2.....|

Yes, they used the RIFF container format for their projects. Seems an odd choice, especially for video production although it is well suited for it. AVI is another video format which uses the RIFF container. The MVP project file uses the ID SEKD with the format MVPH. Earlier versions of Movie Edit Pro used a different extension.

hexdump -C MAGIXv11-s01.MVD | head
00000000  52 49 46 46 38 57 00 00  53 45 4b 44 53 56 49 50  |RIFF8W..SEKDSVIP|
00000010  70 00 00 00 00 dc 05 00  00 00 00 00 04 00 00 00  |p...............|
00000020  02 00 00 00 80 bb 00 00  10 00 00 00 8e 23 d6 e2  |.............#..|
00000030  53 f8 3d 40 00 00 f0 42  01 00 00 00 bd 04 ef fe  |S.=@...B........|
00000040  00 00 01 00 00 00 06 00  00 00 04 00 00 00 06 00  |................|
00000050  00 00 04 00 3f 00 00 00  28 00 00 00 04 00 04 00  |....?...(.......|
00000060  01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  c8 1b 32 01 d0 02 00 00  e0 01 00 00 52 d7 da fb  |..2.........R...|
00000080  54 55 f5 3f 4c 49 53 54  04 00 00 00 70 68 79 73  |TU.?LIST....phys|
00000090  4c 49 53 54 d0 3d 00 00  74 72 6b 73 4c 49 53 54  |LIST.=..trksLIST|

The MVD format used on an earlier version of Movie Edit Pro is also a RIFF, and with the ID of SEKD, but has a format of SVIP.

RIFFpad can break down the chunks we see in an MVP file. Each of the LIST chunks has their own subchunks as well. I assume this his how the editing software stores each video/audio track references, etc. So I give it to MAGIX for at least using an understandable format to store their projects.

MAGIX has also used RIFF in many of its supporting formats. So far I have found mfx, afx, ifx, cfx, ctf, tfx, ufx, mmt, mmm, hdp, each having their own format:

hexdump -C 101_Loud.mfx | head
00000000  52 49 46 46 a8 6f 00 00  53 45 4b 44 4d 41 46 58  |RIFF.o..SEKDMAFX|
00000010  00 00 00 00 4c 49 53 54  94 6f 00 00 41 55 46 58  |....LIST.o..AUFX|
00000020  4c 49 53 54 88 6f 00 00  41 46 58 45 46 58 48 44  |LIST.o..AFXEFXHD|
00000030  20 00 00 00 00 00 25 0d  00 00 00 00 02 00 00 00  | .....%.........|
00000040  01 00 00 00 00 00 00 00  03 18 00 00 00 00 00 00  |................|
00000050  00 00 00 00 4c 49 53 54  54 6f 00 00 41 46 58 44  |....LISTTo..AFXD|
00000060  4c 49 53 54 50 6a 00 00  41 46 58 45 46 58 48 44  |LISTPj..AFXEFXHD|
00000070  20 00 00 00 00 00 25 0d  00 00 00 00 05 00 00 00  | .....%.........|
00000080  01 00 00 00 00 00 00 00  03 18 00 00 00 00 00 00  |................|
00000090  00 00 00 00 4c 49 53 54  1c 6a 00 00 41 46 58 44  |....LIST.j..AFXD|

Not sure the best way to manage all of these in terms of identification, as I am not sure what what is the purpose of each format. Maybe for now I’ll make a generic to catch them all as a MAGIX File.

Extension	ID	FORMAT
AFX	SEKD	SAFX
CFX	SEKD	SCFX
CTF	SEKD	SVIP
HDP	SEKD	SHDP
IFX	SEKD	SIFX
MFX	SEKD	MAFX
MMM	SEKD	SVIP
MMT	SEKD	SVIP
MVD	SEKD	SVIP
MVP	SEKD	MVPH
MXM	MXMD	mxmi
TFX	SEKD	STFX
UFX	SEKD	SVIP

But, when it comes to their proprietary MAGIX Video format, I think they may have pushed things a little too far. Meet the MXV format:

hexdump -C MAGIXv11-s01.mxv | head
00000000  4d 58 52 49 46 46 36 34  9a cb 2b 00 00 00 00 00  |MXRIFF64..+.....|
00000010  4d 58 4a 56 49 44 36 34  4d 58 4a 56 48 32 36 34  |MXJVID64MXJVH264|
00000020  70 00 00 00 00 00 00 00  70 00 00 00 03 00 00 00  |p.......p.......|
00000030  42 93 2b 00 00 00 00 00  f0 00 00 00 00 00 00 00  |B.+.............|
00000040  7b 2e 00 00 4b 00 00 00  01 00 00 00 00 00 00 00  |{...K...........|
00000050  8e 23 d6 e2 53 f8 3d 40  80 02 00 00 e0 01 00 00  |.#..S.=@........|
00000060  80 02 00 00 e0 01 00 00  04 00 00 00 43 15 00 00  |............C...|
00000070  f0 00 00 00 00 00 00 00  28 19 00 00 00 00 00 00  |........(.......|
00000080  55 55 55 55 55 55 f5 3f  00 00 00 00 00 00 00 00  |UUUUUU.?........|
00000090  7f dd 05 00 00 00 00 00  4d 58 4a 56 48 44 36 34  |........MXJVHD64|

I am not sure what I am looking at, is it a RIFF? Is it a RIFF variant like RF64? MAGIX claims the format is:

This is the MAGIX video format for quicker processing with MAGIX products. It offers very low loss of quality, but it cannot be played via conventional DVD players.
MAGIX Video Pro X6

A look around the internet doesn’t bring much up in reference to this format. Just my recent page on the format wiki. A search for MXRIFF64 bring up nothing. But a closer look at other strings within the MXV file reveal we are probably looking at some sort of MPEG format.

I was able to locate a project on GitHub which claims to be able to demux the MXV format. The software is written in GO and appears to indicate this format is chunked based and has most of the chunks figured out. So if you find yourself stuck with some MXV files and don’t want to use the latest from MAGIX, this might be the tool for you.

This demuxer also has an interesting file you can download. It is called a “GRAMMAR” file and can be loaded into hex viewers like Synalyze It! can show the parts of a file you load. Its a great way to explore a format!

None of these formats are found in PRONOM, project files are not usually kept in archives, but if would be good to know about the RIFF files if they do turn up. The video format is for sure something the archival world should know about. MediaInfo is currently not aware of this format, but seems like it might be an easy task.

As usual, you can see some samples and my proposal signatures on my GitHub.

Designer

March 15, 2024 by Thor Leave a comment

Micrografx / Corel Designer

Many software titles we have all used began life under a different brand or even title. Larger software companies gobble up smaller developers, some brands merge, and others change names for whatever reason. Adobe has bought many smaller companies over the years, sometimes developing the acquired software and other times burying the software to avoid competition. Pagemaker was bought to give InDesign life, many Macromedia titles were incorporated or shelved. Such is life in the software world.

In understanding a file format, often times you need to follow this trail backwards to understand when file formats changed and compatibility is dropped. Often times the formats remained the same, but the extension is changed. Or the software name changes and formats are updated, but the extension remains the same. There can also be multiple titles which all use a common format, further complicating the identification of the formats.

Let’s look closer at the a title which changed names and file formats a few times over the years. Micrografx was founded in 1982 and were pretty well known for their innovation in computer graphics. They have released many titles over the years, but one of the first was In*A*Vision graphic software for Windows 1.0 in 1986. This software used a format with the .PIC extension. A couple years later version 2, was renamed to Micrografx Designer and used the .DRW extension. This extension was also used by Micrografx Draw, another similar program.

Micrografx Designer continued to be released until version 9 which is when it was purchased by Corel who continued to release new versions, although it is said the software was just a variation of CorelDraw, and now Designer is part of the CorelDraw Technical Suite. Other Micrografx software such as Picture Publisher was discontinued and customers were encouraged to use Corel’s PaintShop Pro instead. Somewhere in the middle of all this, Micrografx spun off a separate business unit called iGrafx, which Designer was marketed under for a short time.

Let’s break down the names, extensions used, and format type.

In*A*Vision & Draw, binary format, PIC extension
Micrografx Designer & Draw, binary format, DRW extension
Micrografx Designer version 4, RIFF format, DS4 & MGX extension
Micrografx Designer versions 6-9, OLE Container format, DSF extension
Micrografx/Corel Designer versions 10-12, RIFF format, DES extension
Corel Designer version X4-Current, ZIP/XML format, DES extension

According to the 2021 Corel DesignerUser Guide:

Corel DESIGNER (DES, DSF, DS4, or DRW)

You can import Corel DESIGNER files. Files from version 10 and later have the filename extension .des. Files from Micrografx versions 6 to 9 have the filename extension .dsf. Version 4 files have the filename extension .ds4. The .drw filename extension is used for a Micrografx 2.x or 3.x file. Micrografx template files (DST) are also supported.

The PRONOM registry has a few of these formats with signatures and documented, but not all, let’s see where the gaps are.

PUID	Format Name	Format Version	Extension
x-fmt/151	Micrografx Designer		dsf
x-fmt/296	Micrografx Designer	3.1	drw
x-fmt/47	Micrografx Draw	1-2	drw
x-fmt/294	Micrografx Draw	3	drw
x-fmt/295	Micrografx Draw	4	drw, drt
fmt/1907	Micrografx Icon File		icn
fmt/1481	Micrografx In-A-Vision Drawing		pic

So from the PRONOM list, it appears we have good identification on the original PIC and DRW formats. Then the Designer DSF OLE container is taken care of as well. That leaves us with DS4 and DES formats.

hexdump -C DS41-S01.DS4 | head
00000000  52 49 46 46 6e 07 00 00  4d 47 58 20 69 74 70 64  |RIFFn...MGX itpd|
00000010  04 00 00 00 00 02 00 80  70 72 6f 70 23 00 00 00  |........prop#...|
00000020  1f 00 00 30 02 00 00 00  08 00 2c 40 44 00 11 20  |...0......,@D.. |
00000030  20 00 01 10 80 e0 00 00  91 08 21 e0 5c 82 90 72  | .........!.\..r|
00000040  05 ff c0 00 4c 49 53 54  10 04 00 00 64 69 74 6e  |....LIST....ditn|
00000050  74 68 6e 6c 03 04 00 00  57 01 00 30 00 00 08 00  |thnl....W..0....|
00000060  08 00 00 41 04 00 01 20  a4 00 82 10 72 14 40 48  |...A... ....r.@H|
00000070  00 58 20 84 04 32 10 40  00 12 c8 98 18 22 63 90  |.X ..2.@....."c.|
00000080  2b 91 32 36 47 08 20 c0  23 e4 80 90 92 22 46 49  |+.26G. .#...."FI|
00000090  09 29 26 24 e4 a0 94 92  a2 56 4b 09 69 2e 25 e4  |.)&$.....VK.i.%.|

Micrografx Designer 4 apparently uses the RIFF container format. The RIFF format is used with many different types of formats. The most common is the WAV format. CorelDRAW also uses the RIFF format so it makes sense they would use it as they took over from Micrografx.

Each RIFF format has a four byte identifier type after the first eight bytes which identify the RIFF. The DS4 file uses the code “MGX ” to identify itself. Which also appears to be used with their clipart format, MGX. We can use the same identification method we use for other RIFF’s to identify this format.

hexdump -C Corel-DES10Sample.des | head
00000000  52 49 46 46 8a 57 00 00  44 45 53 41 76 72 73 6e  |RIFF.W..DESAvrsn|
00000010  02 00 00 00 7e 04 4c 49  53 54 54 0c 00 00 69 63  |....~.LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Starting with version 10 of Corel Designer, the RIFF format is used again and has a different type. With Version 10 using “DESA”, then for version 10.5:

hexdump -C Corel-DES10.5Sample.des | head 
00000000  52 49 46 46 cc 57 00 00  44 45 53 42 76 72 73 6e  |RIFF.W..DESBvrsn|
00000010  02 00 00 00 b0 04 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

The next version after 10.5 is version 12 and it shows a type:

hexdump -C Corel-DES12-Sample.des | head 
00000000  52 49 46 46 ce 57 00 00  44 45 53 43 76 72 73 6e  |RIFF.W..DESCvrsn|
00000010  02 00 00 00 e2 04 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

After version 12, Corel started using numbering consistent with their other products. The first being X4.

hexdump -C Corel-DESX4-Sample.des | head
00000000  50 4b 03 04 14 00 00 08  00 00 f8 bb c9 4e c3 4b  |PK...........N.K|
00000010  9c d1 2d 00 00 00 2d 00  00 00 08 00 00 00 6d 69  |..-...-.......mi|
00000020  6d 65 74 79 70 65 61 70  70 6c 69 63 61 74 69 6f  |metypeapplicatio|
00000030  6e 2f 78 2d 76 6e 64 2e  63 6f 72 65 6c 2e 64 65  |n/x-vnd.corel.de|
00000040  73 69 67 6e 65 72 2e 64  6f 63 75 6d 65 6e 74 2b  |signer.document+|
00000050  7a 69 70 50 4b 03 04 14  00 00 08 00 00 f8 bb c9  |zipPK...........|
00000060  4e 6f 38 b6 64 98 13 00  00 98 13 00 00 14 00 00  |No8.d...........|
00000070  00 63 6f 6e 74 65 6e 74  2f 72 69 66 66 44 61 74  |.content/riffDat|
00000080  61 2e 63 64 72 52 49 46  46 90 13 00 00 44 45 53  |a.cdrRIFF....DES|
00000090  45 76 72 73 6e 02 00 00  00 82 05 4c 49 53 54 54  |Evrsn......LISTT|

Well it looks like things changed, starting with X4 the format changed to a ZIP container. Let’s take a peak inside.

Path = Corel-DESX4-Sample.des
Type = zip
Physical Size = 8714

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:47 .....           45           45  mimetype
2019-06-09 22:31:47 .....         5016         5016  content/riffData.cdr
2019-06-09 22:31:47 .....       196662          239  metadata/thumbnails/thumbnail.bmp
2019-06-09 22:31:47 .....       151606          698  metadata/thumbnails/page1.bmp
2019-06-09 22:31:47 .....          596          259  metadata/textinfo.xml
2019-06-09 22:31:47 .....         4977         1314  metadata/metadata.xml
2019-06-09 22:31:47 .....           53           55  links.xml
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:47             358955         7626  7 files

Looks like the container holds a RIFF inside along with some thumbnails, metadata, and other things. The mimetype file simple holds “application/x-vnd.corel.designer.document+zip”. The riffData.cdr however looks like this:

hexdump -C Corel-DESX4-Sample/content/riffData.cdr | head
00000000  52 49 46 46 90 13 00 00  44 45 53 45 76 72 73 6e  |RIFF....DESEvrsn|
00000010  02 00 00 00 82 05 4c 49  53 54 54 0c 00 00 69 63  |......LISTT...ic|
00000020  63 70 69 63 63 64 48 0c  00 00 00 00 0c 48 4c 69  |cpiccdH......HLi|
00000030  6e 6f 02 10 00 00 6d 6e  74 72 52 47 42 20 58 59  |no....mntrRGB XY|
00000040  5a 20 07 ce 00 02 00 09  00 06 00 31 00 00 61 63  |Z .........1..ac|
00000050  73 70 4d 53 46 54 00 00  00 00 49 45 43 20 73 52  |spMSFT....IEC sR|
00000060  47 42 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |GB..............|
00000070  f6 d6 00 01 00 00 00 00  d3 2d 48 50 20 20 00 00  |.........-HP  ..|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Another RIFF, and seems to be in the same sequence, but going from version 12 to X4 we seemed to have skipped “DESD”. Maybe there was a developer version in between as they transitioned. Version X5 looks similar and has the RIFF sequence “DESF”. When we get to X6 the structure changes.

Path = Corel-DESX6-Sample.des
Type = zip
Physical Size = 8568

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:21 .....           45           45  mimetype
2019-06-09 22:31:21 .....        12153         1098  content/data/data1.dat
2019-06-09 22:31:21 .....          439          224  content/data/masterPage.dat
2019-06-09 22:31:21 .....          613          265  content/data/page1.dat
2019-06-09 22:31:21 .....           34           28  content/dataFileList.dat
2019-06-09 22:31:21 .....          960          279  content/root.dat
2019-06-09 22:31:21 .....       196662          239  metadata/thumbnails/thumbnail.bmp
2019-06-09 22:31:21 .....       151606          698  metadata/thumbnails/page1.bmp
2019-06-09 22:31:21 .....          427          208  color/color.xml
2019-06-09 22:31:21 .....          596          259  metadata/textinfo.xml
2019-06-09 22:31:21 .....          103          100  color/docPalette.xml
2019-06-09 22:31:21 .....        14920         1444  styles/document.cdss
2019-06-09 22:31:21 .....         5500         1462  metadata/metadata.xml
2019-06-09 22:31:21 .....           53           55  links.xml
------------------- ----- ------------ ------------  ------------------------
2019-06-09 22:31:21             384111         6404  14 files

The mimetype remains the same, but we see additional files within the structure. Also the riffData.cdr file is missing. Looking at each file we can see the root.dat file is a RIFF and follows the same sequence.

hexdump -C Corel-DESX6-Sample/content/root.dat | head
00000000  52 49 46 46 b8 03 00 00  44 45 53 47 66 76 65 72  |RIFF....DESGfver|
00000010  10 00 00 00 ff ff ff ff  08 00 00 00 5e 06 02 00  |............^...|
00000020  00 00 10 00 76 72 73 6e  10 00 00 00 ff ff ff ff  |....vrsn........|
00000030  02 00 00 00 5e 06 00 00  00 00 00 00 4c 49 53 54  |....^.......LIST|
00000040  7c 00 00 00 64 6f 63 20  6d 63 66 67 10 00 00 00  ||...doc mcfg....|
00000050  00 00 00 00 83 20 00 00  00 00 00 00 00 00 00 00  |..... ..........|
00000060  70 72 65 66 10 00 00 00  00 00 00 00 e6 0e 00 00  |pref............|
00000070  83 20 00 00 00 00 00 00  70 74 72 74 10 00 00 00  |. ......ptrt....|
00000080  00 00 00 00 10 00 00 00  69 2f 00 00 00 00 00 00  |........i/......|
00000090  4c 49 53 54 04 00 00 00  66 69 6c 74 4c 49 53 54  |LIST....filtLIST|

As we get to a more recent version. We can see the pattern continues.

hexdump -C Designer2022-s01/content/root.dat | head
00000000  52 49 46 46 88 06 00 00  44 45 53 4e 66 76 65 72  |RIFF....DESNfver|
00000010  10 00 00 00 ff ff ff ff  08 00 00 00 60 09 02 00  |............`...|
00000020  00 00 18 00 76 72 73 6e  10 00 00 00 ff ff ff ff  |....vrsn........|
00000030  02 00 00 00 60 09 00 00  00 00 00 00 4c 49 53 54  |....`.......LIST|
00000040  30 01 00 00 64 6f 63 20  6d 63 66 67 10 00 00 00  |0...doc mcfg....|
00000050  00 00 00 00 08 1f 00 00  00 00 00 00 00 00 00 00  |................|
00000060  70 72 65 66 10 00 00 00  00 00 00 00 ae 07 00 00  |pref............|
00000070  08 1f 00 00 00 00 00 00  70 74 72 74 10 00 00 00  |........ptrt....|
00000080  00 00 00 00 10 00 00 00  b6 26 00 00 00 00 00 00  |.........&......|
00000090  4c 49 53 54 4c 00 00 00  66 6e 74 74 66 6f 6e 74  |LISTL...fnttfont|

The last sample I have is for Corel Designer 2022, but there could be more. I created new signatures for all the samples I have, you can see them in my Github as usual. I decided to group some of the versions together to simplify things a bit, but if anyone thinks they should be broken out into individual versions, let me know.

Embedded WAVE, thanks HP 👋

May 26, 2023 by Thor 1 Comment

Digital Preservation is all about identifying risks. This is done through a process which includes identification, validation, and metadata extraction. The more you know about the digital data you need to preserve over time, the more you can do to minimize those risks with the goal of making the data accessible over time.

Many formats are pretty straight forward, they are identifiable through a header and then have some binary bits or plain text that is readable by certain software. Others are more complicated. A common practice for more complex needs is to use a container. Word processing programs started out with plain text with maybe some formatting codes mixed in, then many moved to the Microsoft OLE container so you could have additional content embedded in a single file. Today file formats such as DOCX use a ZIP container, which houses all the text, images, formatting and anything else the format supports. Knowing what the format is and knowing what it may contain is important to preservation.

I collect older digital cameras, specifically cameras with unique file formats, raw and otherwise. When I picked up a HP (Hewlett-Packard) point and shoot camera awhile back, I was initially unimpressed as it would only capture in a JPEG format and only 3 quality settings. While looking at a copy of the manual, I saw the camera was capable of capturing audio clips or voice memos for each photo taken. This can be handy when taking many photos and need a reminder about the context. This was not unique to HP, as many cameras could do this, normally a JPG was captured and the Audio would have the same name connecting the two. But when I recorded some audio on my little HP, placed the SD card in my computer, I couldn’t find the additional audio file. I also not the only one to ask about this.

There are many types of JPG files. Raw Streams, JPEG File Interchange Format (JFIF), and Exchangeable Image File Format (EXIF). Normally these formats have raster image data sprinkled with metadata. I have seen JPEG files embedded into other formats and containers, such as MP3, PDF, etc, but JPEG’s are not container formats. Or so I thought…..

View of HP Photosmart 433 folder in HP Photo & Imaging Gallery

Lets take a look at an image I took with my HP Photosmart 433. We’ll start with identification:

siegfried   : 1.10.1
scandate    : 2023-05-25T12:27:04-06:00
signature   : default.sig
created     : 2023-05-22T08:43:02-06:00
identifiers : 
  - name    : 'pronom'
    details : 'DROID_SignatureFile_V112.xml; container-signature-20230510.xml'
---
filename : 'GitHub/digicam_corpus/HP/Photosmart 433/IM000959.JPG'
filesize : 178922
modified : 2023-05-25T11:23:32-06:00
errors   : 
matches  :
  - ns      : 'pronom'
    id      : 'x-fmt/391'
    format  : 'Exchangeable Image File Format (Compressed)'
    version : '2.2'
    mime    : 'image/jpeg'
    class   : 'Image (Raster)'
    basis   : 'extension match jpg; byte match at [[0 16] [366 12] [178907 2]] (signature 2/2)'
    warning :

IM000959.JPG was identified as x-fmt/391 which is a compressed Exchangeable Image File Format. version 2.2. Pretty straight forward. Next lets look at validation:

Jhove (Rel. 1.28.0, 2023-05-18)
 Date: 2023-05-25 12:35:16 MDT
 RepresentationInformation: GitHub/digicam_corpus/HP/Photosmart 433/IM000959.JPG
  ReportingModule: JPEG-hul, Rel. 1.5.4 (2023-03-16)
  LastModified: 2023-05-25 11:23:32 MDT
  Size: 178922
  Format: JPEG
  Status: Well-Formed and valid
  SignatureMatches:
   JPEG-hul
  ErrorMessage: Tag 41492 out of sequence
   ID: TIFF-HUL-2
   Offset: 606
  MIMEtype: image/jpeg
  JPEGMetadata: 
   CompressionType: Huffman coding, Baseline DCT
   Images: 
    Number: 1
    Image: 
     NisoImageMetadata: 
      FormatName: image/jpeg
      ByteOrder: big_endian
      CompressionScheme: JPEG
      ImageWidth: 640
      ImageHeight: 480
      ColorSpace: YCbCr
      DateTimeCreated: 2021-11-16T09:04:04
      ScannerManufacturer: Hewlett-Packard
      ScannerModelName: hp PhotoSmart 43x series
      DigitalCameraManufacturer: Hewlett-Packard
      DigitalCameraModelName: hp PhotoSmart 43x series
      FNumber: 4
      ................................
     Exif: 
      ExifVersion: 0220
      FlashpixVersion: 0100
      ColorSpace: sRGB
      ComponentsConfiguration: 1, 2, 3, 0
      CompressedBitsPerPixel: 1.568
      PixelXDimension: 640
      PixelYDimension: 480
      MakerNote: 0, 97, 48, 101, 114, 32, 78, 111, 116, 101, 115, 0, 0, 0, 0, 0
      DateTimeOriginal: 2021:11:16 09:04:04
      DateTimeDigitized: 2021:11:16 09:04:04
   ApplicationSegments: APP1, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2, APP2

I removed a few lines to show important parts, but we get some similar information about the format, a JPEG with EXIF version 2.2. We also learn that HP improperly ordered their tags and put Tag 41492 out of sequence, but we can ignore that for now. Looking close at the output does not give us any indication of audio formats. There is a clue when we see the mention of a Flashpix version and additional Application Segments.

Since this is an image with EXIF data, lets also take a look at the output of Exiftool.

ExifTool Version Number         : 12.62
File Name                       : IM000959.JPG
Directory                       : .
File Size                       : 179 kB
File Modification Date/Time     : 2023:05:25 11:23:32-06:00
File Access Date/Time           : 2023:05:25 11:24:42-06:00
File Inode Change Date/Time     : 2023:05:25 11:24:39-06:00
File Permissions                : -rwxr-xr-x
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Little-endian (Intel, II)
Image Description               : IM000959.JPG
Make                            : Hewlett-Packard
Camera Model Name               : hp PhotoSmart 43x series
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 1.400
Modify Date                     : 2021:11:16 09:04:04
Y Cb Cr Positioning             : Co-sited
Copyright                       : Copyright 2002-2003
Exposure Time                   : 1/29
F Number                        : 4.0
ISO                             : 100
Exif Version                    : 0220
Date/Time Original              : 2021:11:16 09:04:04
Create Date                     : 2021:11:16 09:04:04
Components Configuration        : Y, Cb, Cr, -
Compressed Bits Per Pixel       : 1.567552083
Shutter Speed Value             : 1/30
Aperture Value                  : 4.0
Exposure Compensation           : 0
Max Aperture Value              : 4.0
Subject Distance                : 1 m
Metering Mode                   : Average
Light Source                    : Unknown
Flash                           : Auto, Did not fire
Focal Length                    : 5.7 mm
Warning                         : [minor] Unrecognized MakerNotes
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 640
Exif Image Height               : 480
Interoperability Index          : R98 - DCF basic file (sRGB)
Interoperability Version        : 0100
Digital Zoom Ratio              : 1
Subject Location                : 0
Compression                     : JPEG (old-style)
Thumbnail Offset                : 2046
Thumbnail Length                : 7112
Code Page                       : Unicode UTF-16, little endian
Used Extension Numbers          : 1, 31
Extension Name                  : Audio
Extension Class ID              : 10000100-6FC0-11D0-BD01-00609719A180
Extension Persistence           : Always Valid
Audio Stream                    : (Binary data 117820 bytes, use -b option to extract)
Image Width                     : 640
Image Height                    : 480
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:2 (2 1)
Aperture                        : 4.0
Image Size                      : 640x480
Megapixels                      : 0.307
Shutter Speed                   : 1/29
Thumbnail Image                 : (Binary data 7112 bytes, use -b option to extract)
Focal Length                    : 5.7 mm
Light Value                     : 8.9

Ohh, what do we have here? Exiftool mentions an audio stream. An audio stream inside the JPEG? How is this possible? The Flashpix format was originally developed by Kodak in which collaborated with HP. This was later added to the EXIF specifications. Below is an screenshot from the Exif Version 2.2 spec.

Exiftool mentioned Flashpix and additional APP2 segments. Lets take a look at the raw file in a hex editor.

Ahhh….. In one of the App2 segments we can see something familiar. A RIFF WAVE header! Lets see if we can extract the WAVE file.

exiftool -b -AudioStream IM000959.JPG > IM000959.WAV

mediainfo IM000959.WAV
General
Complete name                            : IM000959.WAV
Format                                   : Wave
Format settings                          : WaveFormatEx
File size                                : 115 KiB
Duration                                 : 10 s 681 ms
Overall bit rate mode                    : Constant
Overall bit rate                         : 88.2 kb/s

Audio
Format                                   : ADPCM
Codec ID                                 : 11
Codec ID/Hint                            : Intel
Duration                                 : 10 s 681 ms
Bit rate mode                            : Constant
Bit rate                                 : 88.2 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 22.05 kHz
Bit depth                                : 4 bits
Stream size                              : 115 KiB (100%)

MediaInfo can give us details on the embedded WAVE file, which is pretty terrible quality but is a PCM audio stream.

Embedded audio inside a raster image is not common. Most software which can render a JPEG image will most likely ignore the embedded WAVE and not even give a warning it exists. IM000959.JPG opens fine in Adobe Photoshop, but saving to a new format or making any edits will delete the WAVE file. Imagemagick also will remove the WAVE with any editing with no warning.

In order to ensure the embedded audio stream is preserved we first need to know it is there, this is where tools like exiftool can be used to extract metadata from the file and the image can be associated with having an audio stream and handled differently than any other JPEG file. More work is needed, Exiftool may mention an Audio Stream, but currently does not have the ability to pull any data from the stream.