Digital Negatives

One of the important parts about Digital Preservation is to gather significant properties of the digital files we hope to preserve. This can allow us to base our risk assessments off of more data than just an extension. For example, a TIFF file is a mighty good preservation format. Well documented and adopted by the preservation community, and with hundreds if not thousands of software tools to render and make use of the format. But if a TIFF file uses compression like LZW, or if it happens to have multiple pages, those are good things to know about. Most formats might have a stable set of properties, but sometimes can have properties which adds more risk to the format becoming difficult to render or migrate.

A DNG or Digital Negative developed by Adobe was supposed to solve the issues with proprietary RAW digital camera formats. Rendering a PhaseOne IIQ file often times requires the full CaptureOne software which can be expensive. Adobe spends quite a bit of resources in adding support to its Camera RAW toolkit and adding the ability to take majority of these RAW formats and move them into a DNG. There is also more and more camera manufacturers who image directly to a DNG as their native RAW format. This is the case for Apple’s ProRAW format which uses the DNG specification.

Another manufacturer is the Insta360 camera’s. Their 360 camera’s can use two lenses to capture 180 degrees from each and then stitch into a 360 photo or video. They can capture compressed images and videos, but also in RAW. Because of the two lenses and sensors, their DNG’s can get quite large. For this reason I recently asked PRONOM to adjust their signatures to allow for a bigger offset of DNG information in the larger RAW images.

exiftool IMG_20230913_141939_00_039.dng 
ExifTool Version Number         : 12.70
File Name                       : IMG_20230913_141939_00_039.dng
File Size                       : 143 MB
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution image
Image Width                     : 5984
Image Height                    : 11968
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Make                            : Arashi Vision
Camera Model Name               : Insta360 X3

DNG files are actually based on the TIFF format, TIFF/EP to be precise, which means there is some good history behind the format and understanding of its structure. DNG does add many new tags and new features, so there is much more going on. Here is a TIFFInfo view of a DNG. Lots of new tags…..

tiffinfo IMG_20230913_141939_00_039.dng 
TIFFReadDirectory: Warning, Unknown field with tag 33421 (0x828d) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 33422 (0x828e) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50937 (0xc6f9) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50938 (0xc6fa) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 50940 (0xc6fc) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 51009 (0xc741) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 51107 (0xc7a3) encountered.
=== TIFF directory 0 ===
TIFF Directory at offset 0x889946c (143234156)
  Subfile Type: (0 = 0x0)
  Image Width: 5984 Image Length: 11968
  Bits/Sample: 16
  Sample Format: unsigned integer
  Compression Scheme: None
  Photometric Interpretation: 32803 (0x8023)
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 1
  Rows/Strip: 11968
  Planar Configuration: single image plane
  Make: Arashi Vision
  Model: Insta360 X3
  Software: v1.0.69_build1
  DateTime: 2023:09:13 14:19:40
  Tag 33421: 2,2
  Tag 33422: 1,2,0,1
  EXIFIFDOffset: 0x8
  GPSIFDOffset: 0x3e6
  DNGVersion: 1,3,0,0
  DNGBackwardVersion: 1,3,0,0
  UniqueCameraModel: Insta360 X3

An IFD (Image File Directory) is the building block of a TIFF file. A TIFF file can have multiple IFD’s within a single file. But an IFD can also be a thumbnail, metadata or GPS info. For a DNG, they use the IFD structure as well, but often, the first IFD is a lower resolution of the full image.

 <File:FileType>DNG</File:FileType>
 <File:FileTypeExtension>dng</File:FileTypeExtension>
 <File:MIMEType>image/x-adobe-dng</File:MIMEType>
 <File:ExifByteOrder>Little-endian (Intel, II)</File:ExifByteOrder>
 <IFD0:SubfileType>Reduced-resolution image</IFD0:SubfileType>
 <IFD0:ImageWidth>256</IFD0:ImageWidth>
 <IFD0:ImageHeight>171</IFD0:ImageHeight>
 <IFD0:BitsPerSample>8 8 8</IFD0:BitsPerSample>
 <IFD0:Compression>Uncompressed</IFD0:Compression>
 <IFD0:PhotometricInterpretation>RGB</IFD0:PhotometricInterpretation>
 <IFD0:Make>Canon</IFD0:Make>
 <IFD0:Model>Canon EOS RP</IFD0:Model>
...
 <SubIFD:SubfileType>Full-resolution image</SubIFD:SubfileType>
 <SubIFD:ImageWidth>6384</SubIFD:ImageWidth>
 <SubIFD:ImageHeight>4224</SubIFD:ImageHeight>
 <SubIFD:BitsPerSample>16</SubIFD:BitsPerSample>
 <SubIFD:Compression>JPEG</SubIFD:Compression>

But not always the same way.

 <IFD0:SubfileType>Full-resolution image</IFD0:SubfileType>
 <IFD0:ImageWidth>5984</IFD0:ImageWidth>
 <IFD0:ImageHeight>11968</IFD0:ImageHeight>
 <IFD0:BitsPerSample>16</IFD0:BitsPerSample>
 <IFD0:Compression>Uncompressed</IFD0:Compression>
 <IFD0:PhotometricInterpretation>Color Filter Array</IFD0:PhotometricInterpretation>
 <IFD0:Make>Arashi Vision</IFD0:Make>
 <IFD0:Model>Insta360 X3</IFD0:Model>

 <IFD0:SubfileType>Reduced-resolution image</IFD0:SubfileType>
 <IFD0:ImageWidth>4032</IFD0:ImageWidth>
 <IFD0:ImageHeight>3024</IFD0:ImageHeight>
 <IFD0:BitsPerSample>8 8 8</IFD0:BitsPerSample>
 <IFD0:Compression>JPEG</IFD0:Compression>
 <IFD0:PhotometricInterpretation>YCbCr</IFD0:PhotometricInterpretation>
 <IFD0:Make>Apple</IFD0:Make>
 <IFD0:Model>iPhone 13 Pro</IFD0:Model>
...
 <SubIFD:SubfileType>Full-resolution image</SubIFD:SubfileType>
 <SubIFD:ImageWidth>4032</SubIFD:ImageWidth>
 <SubIFD:ImageHeight>3024</SubIFD:ImageHeight>
 <SubIFD:BitsPerSample>12 12 12</SubIFD:BitsPerSample>
 <SubIFD:Compression>JPEG</SubIFD:Compression>

It can get confusing, especially for tools we use to extract metadata and significant properties from a DNG for preservation. Within Rosetta, the preservation system I use at work, there is no dedicated DNG extractor, so we use JHOVE, as it is the tool we use for our TIFF images. This presents a problem as the process only extracts properties for the first IFD assuming it is the main IFD, but in many cases it reports back the image is much smaller in pixel dimensions than it actually is. More work is needed to improve extracting correct significant properties for DNG and other RAW image formats.

Adobe released a new version of DNG this year. In June, DNG version 1.7.0.0 was finalized. The new version brought a few new features, two of which are including JPEG XL compression and a new HDR colorimetric value. In order to add JPEG XL compression DNG version 1.7 is required. Here is how one looks in exiftool, created with Adobe DNG Converter 16.1.

exiftool _MG_9375_1.dng 
ExifTool Version Number         : 12.70
File Name                       : _MG_9375_1.dng
File Size                       : 5.4 MB
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Canon
Camera Model Name               : Canon EOS DIGITAL REBEL XT
Preview Image Start             : 91884
Orientation                     : Rotate 270 CW
Rows Per Strip                  : 171
Preview Image Length            : 10305
Software                        : Adobe DNG Converter 16.1 (Macintosh)
Modify Date                     : 2023:12:18 11:45:06
Artist                          : unknown
Image Width                     : 3516
Image Height                    : 2328
Bits Per Sample                 : 16
Compression                     : JPEG XL
DNG Version                     : 1.7.1.0
DNG Backward Version            : 1.7.1.0

I had recently submitted a new signature for DNG 1.7 to PRONOM, but I found this new DNG version falls outside the signature I created. I had made the assumption all DNG’s report their version based on the last two values of 0.0, so I created the signature to look for 1.7.0.0. This is wrong now that I can see an example of version 1.7.1.0.

In order to fix the issue, I would need to change all the DNG signatures to remove the last two bytes so:

12C601000400000001070000 would change to 12C60100040000000107

This would allow for identification if some DNG files have a point version.

The pace at which manufacturers are producing camera’s with new features is much faster than the Digital Preservation community can keep up with. As new technologies get released, we play catch up trying to identify new formats and variations to existing ones. I guess that is job security?

One thought on “Digital Negatives

Leave a Reply

Your email address will not be published. Required fields are marked *