One of the important parts about Digital Preservation is to gather significant properties of the digital files we hope to preserve. This can allow us to base our risk assessments off of more data than just an extension. For example, a TIFF file is a mighty good preservation format. Well documented and adopted by the preservation community, and with hundreds if not thousands of software tools to render and make use of the format. But if a TIFF file uses compression like LZW, or if it happens to have multiple pages, those are good things to know about. Most formats might have a stable set of properties, but sometimes can have properties which adds more risk to the format becoming difficult to render or migrate.
A DNG or Digital Negative developed by Adobe was supposed to solve the issues with proprietary RAW digital camera formats. Rendering a PhaseOne IIQ file often times requires the full CaptureOne software which can be expensive. Adobe spends quite a bit of resources in adding support to its Camera RAW toolkit and adding the ability to take majority of these RAW formats and move them into a DNG. There is also more and more camera manufacturers who image directly to a DNG as their native RAW format. This is the case for Apple’s ProRAW format which uses the DNG specification.
Another manufacturer is the Insta360 camera’s. Their 360 camera’s can use two lenses to capture 180 degrees from each and then stitch into a 360 photo or video. They can capture compressed images and videos, but also in RAW. Because of the two lenses and sensors, their DNG’s can get quite large. For this reason I recently asked PRONOM to adjust their signatures to allow for a bigger offset of DNG information in the larger RAW images.
exiftool IMG_20230913_141939_00_039.dng ExifTool Version Number : 12.70 File Name : IMG_20230913_141939_00_039.dng File Size : 143 MB File Type : DNG File Type Extension : dng MIME Type : image/x-adobe-dng Exif Byte Order : Little-endian (Intel, II) Subfile Type : Full-resolution image Image Width : 5984 Image Height : 11968 Bits Per Sample : 16 Compression : Uncompressed Photometric Interpretation : Color Filter Array Make : Arashi Vision Camera Model Name : Insta360 X3
DNG files are actually based on the TIFF format, TIFF/EP to be precise, which means there is some good history behind the format and understanding of its structure. DNG does add many new tags and new features, so there is much more going on. Here is a TIFFInfo view of a DNG. Lots of new tags…..
tiffinfo IMG_20230913_141939_00_039.dng TIFFReadDirectory: Warning, Unknown field with tag 33421 (0x828d) encountered. TIFFReadDirectory: Warning, Unknown field with tag 33422 (0x828e) encountered. TIFFReadDirectory: Warning, Unknown field with tag 50937 (0xc6f9) encountered. TIFFReadDirectory: Warning, Unknown field with tag 50938 (0xc6fa) encountered. TIFFReadDirectory: Warning, Unknown field with tag 50940 (0xc6fc) encountered. TIFFReadDirectory: Warning, Unknown field with tag 51009 (0xc741) encountered. TIFFReadDirectory: Warning, Unknown field with tag 51107 (0xc7a3) encountered. === TIFF directory 0 === TIFF Directory at offset 0x889946c (143234156) Subfile Type: (0 = 0x0) Image Width: 5984 Image Length: 11968 Bits/Sample: 16 Sample Format: unsigned integer Compression Scheme: None Photometric Interpretation: 32803 (0x8023) Orientation: row 0 top, col 0 lhs Samples/Pixel: 1 Rows/Strip: 11968 Planar Configuration: single image plane Make: Arashi Vision Model: Insta360 X3 Software: v1.0.69_build1 DateTime: 2023:09:13 14:19:40 Tag 33421: 2,2 Tag 33422: 1,2,0,1 EXIFIFDOffset: 0x8 GPSIFDOffset: 0x3e6 DNGVersion: 1,3,0,0 DNGBackwardVersion: 1,3,0,0 UniqueCameraModel: Insta360 X3
An IFD (Image File Directory) is the building block of a TIFF file. A TIFF file can have multiple IFD’s within a single file. But an IFD can also be a thumbnail, metadata or GPS info. For a DNG, they use the IFD structure as well, but often, the first IFD is a lower resolution of the full image.
<File:FileType>DNG</File:FileType> <File:FileTypeExtension>dng</File:FileTypeExtension> <File:MIMEType>image/x-adobe-dng</File:MIMEType> <File:ExifByteOrder>Little-endian (Intel, II)</File:ExifByteOrder> <IFD0:SubfileType>Reduced-resolution image</IFD0:SubfileType> <IFD0:ImageWidth>256</IFD0:ImageWidth> <IFD0:ImageHeight>171</IFD0:ImageHeight> <IFD0:BitsPerSample>8 8 8</IFD0:BitsPerSample> <IFD0:Compression>Uncompressed</IFD0:Compression> <IFD0:PhotometricInterpretation>RGB</IFD0:PhotometricInterpretation> <IFD0:Make>Canon</IFD0:Make> <IFD0:Model>Canon EOS RP</IFD0:Model> ... <SubIFD:SubfileType>Full-resolution image</SubIFD:SubfileType> <SubIFD:ImageWidth>6384</SubIFD:ImageWidth> <SubIFD:ImageHeight>4224</SubIFD:ImageHeight> <SubIFD:BitsPerSample>16</SubIFD:BitsPerSample> <SubIFD:Compression>JPEG</SubIFD:Compression>
But not always the same way.
<IFD0:SubfileType>Full-resolution image</IFD0:SubfileType> <IFD0:ImageWidth>5984</IFD0:ImageWidth> <IFD0:ImageHeight>11968</IFD0:ImageHeight> <IFD0:BitsPerSample>16</IFD0:BitsPerSample> <IFD0:Compression>Uncompressed</IFD0:Compression> <IFD0:PhotometricInterpretation>Color Filter Array</IFD0:PhotometricInterpretation> <IFD0:Make>Arashi Vision</IFD0:Make> <IFD0:Model>Insta360 X3</IFD0:Model> <IFD0:SubfileType>Reduced-resolution image</IFD0:SubfileType> <IFD0:ImageWidth>4032</IFD0:ImageWidth> <IFD0:ImageHeight>3024</IFD0:ImageHeight> <IFD0:BitsPerSample>8 8 8</IFD0:BitsPerSample> <IFD0:Compression>JPEG</IFD0:Compression> <IFD0:PhotometricInterpretation>YCbCr</IFD0:PhotometricInterpretation> <IFD0:Make>Apple</IFD0:Make> <IFD0:Model>iPhone 13 Pro</IFD0:Model> ... <SubIFD:SubfileType>Full-resolution image</SubIFD:SubfileType> <SubIFD:ImageWidth>4032</SubIFD:ImageWidth> <SubIFD:ImageHeight>3024</SubIFD:ImageHeight> <SubIFD:BitsPerSample>12 12 12</SubIFD:BitsPerSample> <SubIFD:Compression>JPEG</SubIFD:Compression>
It can get confusing, especially for tools we use to extract metadata and significant properties from a DNG for preservation. Within Rosetta, the preservation system I use at work, there is no dedicated DNG extractor, so we use JHOVE, as it is the tool we use for our TIFF images. This presents a problem as the process only extracts properties for the first IFD assuming it is the main IFD, but in many cases it reports back the image is much smaller in pixel dimensions than it actually is. More work is needed to improve extracting correct significant properties for DNG and other RAW image formats.
Adobe released a new version of DNG this year. In June, DNG version 1.7.0.0 was finalized. The new version brought a few new features, two of which are including JPEG XL compression and a new HDR colorimetric value. In order to add JPEG XL compression DNG version 1.7 is required. Here is how one looks in exiftool, created with Adobe DNG Converter 16.1.
exiftool _MG_9375_1.dng ExifTool Version Number : 12.70 File Name : _MG_9375_1.dng File Size : 5.4 MB File Type : DNG File Type Extension : dng MIME Type : image/x-adobe-dng Exif Byte Order : Little-endian (Intel, II) Make : Canon Camera Model Name : Canon EOS DIGITAL REBEL XT Preview Image Start : 91884 Orientation : Rotate 270 CW Rows Per Strip : 171 Preview Image Length : 10305 Software : Adobe DNG Converter 16.1 (Macintosh) Modify Date : 2023:12:18 11:45:06 Artist : unknown Image Width : 3516 Image Height : 2328 Bits Per Sample : 16 Compression : JPEG XL DNG Version : 1.7.1.0 DNG Backward Version : 1.7.1.0
I had recently submitted a new signature for DNG 1.7 to PRONOM, but I found this new DNG version falls outside the signature I created. I had made the assumption all DNG’s report their version based on the last two values of 0.0, so I created the signature to look for 1.7.0.0. This is wrong now that I can see an example of version 1.7.1.0.
In order to fix the issue, I would need to change all the DNG signatures to remove the last two bytes so:
12C601000400000001070000 would change to 12C60100040000000107
This would allow for identification if some DNG files have a point version.
The pace at which manufacturers are producing camera’s with new features is much faster than the Digital Preservation community can keep up with. As new technologies get released, we play catch up trying to identify new formats and variations to existing ones. I guess that is job security?