Scheduling EXport

During a recent review of some help files for some older Final Draft software I came across this Q&A.

Needless to say, I was intrigued, but let me give you a pro tip. Googling MovieMagic and “SEX” does not bring back results related to file formats. Also, probably best not to search at work.

Movie Magic refers to software developed by Write Brothers/Screenplay. The main software, Screenwriter is a word processor built specifically for writing screen plays for TV, Movies, theater, etc. The software was first developed in 1983 and quickly replaced typewriters as the favorite for writing the very specific formatting required by screenplays.

Screenwriter version 6 uses the extension MMSW and DEF for templates. Let’s have a look at one under the hood.

hexdump -C Screenwriterv6-01.mmsw | head
00000000  53 63 72 65 65 6e 77 72  69 74 65 72 57 69 6e 56  |ScreenwriterWinV|
00000010  65 72 2e 20 36 2e 30 30  20 00 00 00 00 00 00 00  |er. 6.00 .......|
00000020  00 00 00 0c 4e 4f 4e 41  4d 45 32 2e 4d 4d 53 57  |....NONAME2.MMSW|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000050  00 00 00 00 00 00 00 00  0f 0a 0a 00 00 00 00 0a  |................|
00000060  01 0c 00 00 0f 0a 0a 01  00 00 01 0a 01 0c 00 00  |................|
00000070  19 18 00 00 00 00 04 0a  01 0c 00 00 25 0a 0a 01  |............%...|
00000080  00 00 03 0a 01 0c 00 00  0f 0a 0a 01 00 00 06 0a  |................|
00000090  01 0c 00 00 1e 23 00 00  00 00 05 0a 01 0c 00 00  |.....#..........|

The header is easy to interpret, Version 6 is the latest version of the software. The rest of the file is non-human readable binary so not much to look at. The screenplay website has a chart for figuring out compatibility for all the versions and extensions.

The other major version used the SCW extension.

hexdump -C ScreenWriter4-s01.scw | head
00000000  53 63 72 65 65 6e 77 72  69 74 65 72 77 69 6e 76  |Screenwriterwinv|
00000010  65 72 2e 20 34 2e 31 31  61 00 00 00 00 00 00 00  |er. 4.11a.......|
00000020  00 00 00 11 53 63 72 65  65 6e 57 72 69 74 65 72  |....ScreenWriter|
00000030  34 2d 73 30 31 00 00 00  00 00 00 00 00 00 00 00  |4-s01...........|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  0f 0a 0a 00 00 00 00 0a  |................|
00000060  00 0c 00 00 0f 0a 0a 01  00 00 01 0a 00 0c 00 00  |................|
00000070  19 18 00 00 00 00 04 0a  00 0c 00 00 25 0a 0a 01  |............%...|
00000080  00 00 03 0a 00 0c 00 00  0f 0a 0a 01 00 00 06 0a  |................|
00000090  00 0c 00 00 1e 23 00 00  00 00 05 0a 00 0c 00 00  |.....#..........|

hexdump -C Screenwriterv6-01.scw | head
00000000  53 63 72 65 65 6e 77 72  69 74 65 72 57 69 6e 56  |ScreenwriterWinV|
00000010  65 72 2e 20 34 2e 39 30  00 00 00 00 00 00 00 00  |er. 4.90........|
00000020  00 00 00 0c 4e 4f 4e 41  4d 45 32 2e 4d 4d 53 57  |....NONAME2.MMSW|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Files from the earlier version appear to be structured the same way. Although files generated on a Macintosh still seem to retain the “win” in the header, but with a capital “Win” and “Ver”. Looking at the older ScriptThing format it is similar.

hexdump -C NONAME1.SCR | head
00000000  53 63 72 69 70 74 54 68  69 6e 67 20 56 65 72 2e  |ScriptThing Ver.|
00000010  20 32 2e 31 39 00 00 00  00 00 07 4e 4f 4e 41 4d  | 2.19......NONAM|
00000020  45 31 2e 53 43 57 00 00  00 00 00 00 00 00 00 00  |E1.SCW..........|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 07 00 01  |................|
00000050  00 0f 49 0b 00 00 0f 49  0b 01 01 19 3b 01 00 04  |..I....I....;...|
00000060  25 49 0b 01 03 0f 49 0b  01 06 1e 30 01 00 05 0f  |%I....I....0....|
00000070  49 0b 01 02 0f 49 0b 01  0a 0f 49 0b 01 0b 0f 49  |I....I....I....I|
00000080  0b 04 14 00 00 02 02 02  02 02 00 00 01 02 00 37  |...............7|
00000090  0b 28 43 4f 4e 54 49 4e  55 45 44 29 00 0a 43 4f  |.(CONTINUED)..CO|

hexdump -C MULTIMEDIA DEMO.SCW | head
00000000  53 63 72 69 70 74 54 68  69 6e 67 20 57 69 6e 56  |ScriptThing WinV|
00000010  65 72 2e 20 31 2e 32 35  64 00 08 44 49 43 54 46  |er. 1.25d..DICTF|
00000020  49 4c 45 1a 4d 75 6c 74  69 6d 65 64 69 61 20 44  |ILE.Multimedia D|
00000030  65 6d 6f 20 53 63 72 69  70 74 2e 53 43 52 00 00  |emo Script.SCR..|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  0a 0e 0a 00 00 00 00 0a  |................|
00000060  00 0c 00 00 0a 0d 0a 01  00 00 01 0a 00 0c 00 05  |................|
00000070  14 1b 00 00 00 00 04 0a  00 0c 00 00 20 0d 0a 01  |............ ...|
00000080  00 00 03 0a 00 0c 00 00  0c 17 0a 01 00 00 06 0a  |................|
00000090  00 0c 00 01 19 26 00 00  00 00 05 0a 00 0c 00 00  |.....&..........|

Screenwriting software is very specialized, the formatting is important as well as keeping track of characters, locations, props, etc. An important part of filming a movie based on a script is to schedule the different scenes and which characters are needed for each scene. This scheduling can be generated by generating a Scheduling EXport. Not sure who decided on this extension, but there it is.

hexdump -C Screenwriterv6-01.sex | head
00000000  53 53 49 2a 00 23 00 00  00 e2 00 00 00 00 43 61  |SSI*.#........Ca|
00000010  73 74 20 4d 65 6d 62 65  72 73 00 45 78 74 72 61  |st Members.Extra|
00000020  73 00 53 74 75 6e 74 73  00 56 65 68 69 63 6c 65  |s.Stunts.Vehicle|
00000030  73 00 50 72 6f 70 73 00  53 70 65 63 69 61 6c 20  |s.Props.Special |
00000040  45 66 66 65 63 74 73 00  43 6f 73 74 75 6d 65 73  |Effects.Costumes|
00000050  00 4d 61 6b 65 75 70 00  4c 69 76 65 73 74 6f 63  |.Makeup.Livestoc|
00000060  6b 00 41 6e 69 6d 61 6c  20 48 61 6e 64 6c 65 72  |k.Animal Handler|
00000070  00 4d 75 73 69 63 00 53  6f 75 6e 64 00 53 65 74  |.Music.Sound.Set|
00000080  20 44 72 65 73 73 69 6e  67 00 47 72 65 65 6e 65  | Dressing.Greene|
00000090  72 79 00 53 70 65 63 69  61 6c 20 45 71 75 69 70  |ry.Special Equip|

Scheduling Export files begin with “SSI“, which I assume refers to Screenplay Systems, Inc. They begin with listing all the different things which need scheduling in plain text. In Screenwriter 6 there are a couple of export types with the same extension. One called Gorilla Scheduling and another called CompanyMOVE ShowPlanner, but both are identical to the Movie Magic file, so I am not quite sure their purpose. Maybe there would be more to discern from a whole script instead of the simple samples I made for this purpose.

This was a fun format to research, although I had to be careful of the terms I used! You can check out my proposed signatures and samples on my GitHub.

Sibelius

Music notation software is among the earliest software for desktop computers. SCORE in 1987, Finale came around in 1988, Capella in 1992, and Sibelius in 1993. Many others came and went during this time. Music notation software was so much more than the typical word processing or desktop publishing system. Specialized fonts were needed to display the music notation and there are many other variables for different instruments allowing individuals and others the ability to create complicated compositions in an inexpensive way.

Sibelius [SI] + [BAY] + [LEE] + [UHS] was originally developed for the Acorn system in 1986, then released on Windows and Macintosh in 1998-99. The software became very popular and in 2006 was purchased by the software giant AVID. The software was used enough to get a preservation assessment by the British Library in 2017 and draft status format description by the Library of Congress, written by the amazing Ashley!

Both reviews of the format emphasize the proprietary nature of the file format which has been used since the early versions. Aside from the early Acorn release, the Windows and Macintosh versions used a binary format with the SIB extension. They are actually quite easy to identify.

hexdump -C Sibelius-s01.sib | head
00000000  0f 53 49 42 45 4c 49 55  53 00 00 40 00 02 00 a4  |.SIBELIUS..@....|
00000010  f1 ed 00 00 00 30 00 00  00 02 00 00 00 01 00 00  |.....0..........|
00000020  00 2a 00 00 00 00 00 00  00 00 0f 53 49 42 45 4c  |.*.........SIBEL|
00000030  49 55 53 00 00 40 00 02  00 00 00 00 00 00 00 3a  |IUS..@.........:|
00000040  00 00 00 00 38 a1 28 06  b3 d2 2f 66 03 04 16 4e  |....8.(.../f...N|
00000050  5f 5c 8d f3 95 27 3e f1  2a 1b 68 de 08 81 e8 9a  |_\...'>.*.h.....|
00000060  ea 1c bf dd 54 0e 92 8d  4d be e3 34 ed 42 78 36  |....T...M..4.Bx6|
00000070  d2 e1 67 7b 8d f7 98 6a  3a 70 c4 8b 0b 08 7b 26  |..g{...j:p....{&|
00000080  f9 45 00 00 00 00 48 71  7c 4c 98 df 0b 38 7d 9d  |.E....Hq|L...8}.|
00000090  2a 2d 84 9c a4 39 0f 4d  da a2 cc 97 ad 3d b0 55  |*-...9.M.....=.U|

This is exactly how PRONOM and other identification methods determine they are Sibelius files. PRONOM has assigned the format fmt/696 and is looking for the hexadecimal bytes 0F534942454C495553.

The problem with this identification method is that all the Sibelius files are identified as such, regardless of version. As mentioned by Ashley, version of the software used is highly important as new features were added all the time making backwards compatibility difficult. Add in the fact that there were different releases for each version which would limit these features even more and I can see how a musician could get very frustrated. If you created a score in Sibelius 5 and tried to open in Sibelius 5 Student version, you may find your composition lacking in many ways. The only way to avoid compatibility issues is to always open in the latest “Ultimate” version. Sibelius Ultimate can open all versions of the SIB format back to version 2. The software even has an export feature which allows you to export back to a previous version stripping what is necessary to ensure compatibility.

Sibelius export to previous version

For those with a bunch of SIB files in their archives, how would you know which software version created the file? Well lets take a closer look at the bytes and see if we can find some patterns. Let it be known, I am not reverse engineering the format, just looking for patterns which will allow for proper identification!

I am not the first person to ask this question, many others want to know the versions of their SIB files. Thankfully others have found some clues on which bytes hold the version information. It seems we can determine the version based on 4 bytes shortly after the SIBELIUS string. Specifically bytes 10-13.

hexdump -C Sibelius2-s01.sib | head
00000000  0f 53 49 42 45 4c 49 55  53 00 00 08 00 22 00 47  |.SIBELIUS....".G|
00000010  98 4c 00 00 00 3a 00 00  00 00 4e 81 49 34 41 2c  |.L...:....N.I4A,|
00000020  fa 76 62 f9 71 53 a9 93  0f 54 1e 20 6c 63 61 4d  |.vb.qS...T. lcaM|
00000030  f7 b2 b0 a7 5d bd 82 3a  0d 86 02 8b f2 89 d2 a0  |....]..:........|
00000040  83 1f 8d e0 37 1b ed 1c  6a 8b 82 08 4b 6d 64 60  |....7...j...Kmd`|
00000050  71 59 e8 aa ef b1 3c df  5c 25 0a 9f 66 50 69 de  |qY....<.\%..fPi.|
00000060  2a d3 4e 2a cd 97 88 06  67 5f 50 64 0f 8f 86 2b  |*.N*....g_Pd...+|
00000070  08 0d 3f f7 80 26 e0 63  f6 7d 4e f8 e7 c0 3f fc  |..?..&.c.}N...?.|
00000080  7a 77 ea b3 4a b9 30 59  13 47 6e 09 0a 0b ae 3c  |zw..J.0Y.Gn....<|
00000090  c1 93 85 f6 41 f8 58 22  4b 92 35 3f b2 f5 3f 9d  |....A.X"K.5?..?.|

From what others have gathered and updating it with more recent versions I have come up with a list.

VersionHex 10-13
Sibelius 1.200 00 00 0E
Sibelius 2.x00 08 xx xx 
Sibelius 3.x00 0A xx xx 
Sibelius 4.x00 1B xx xx 
Sibelius 5.000 2D 00 03 
Sibelius 5.100 2D 00 0D 
Sibelius 5.2.x – 5.400 2D 00 10 
Sibelius 6.0.x00 36 00 01 
Sibelius 6.100 36 00 17 
Sibelius 6.200 36 00 1E 
Sibelius 7.000 39 00 0C 
Sibelius 7.0.1 – 7.0.200 39 00 0E 
Sibelius 7.0.300 39 00 13 
Sibelius 7.1.000 39 00 15 
Sibelius 7.1.2 – 7.1.300 39 00 16 
Sibelius 7.5.x00 3D 00 0E 
Sibelius 8.0.0 – 8.0.100 3D 00 10 
Sibelius 8.1.x00 3E 00 00 
Sibelius 8.200 3E 00 01 
Sibelius 8.300 3E 00 02 
Sibelius 8.4.x00 3E 00 06 
Sibelius 8.5.x00 3E 00 07 
Sibelius 8.6.x, 8.7.0, 8.7.100 3F 00 00 
Sibelius 8.7.2, 2018.1, 2018.4.x, 2018.5, 2018.6, 2018.700 3F 00 01 
Sibelius 2018.11, 2018.1200 3F 00 02
Sibelius 2019.100 3F 00 04
Sibelius 2019.4.x, 2019.5, 2019.7, 2019.900 3F 00 06
Sibelius 2019.1200 3F 00 07
Sibelius 8.6-2019.1200 3F 00 0A
Sibelius 2020.100 3F 00 0B
Sibelius 2020.3, 2020.600 40 00 01
Sibelius 2020.900 40 00 02
Sibelius 2022.500 40 00 03
Sibelius 2022.1100 41 00 02
Sibelius 2022.1200 42 00 00
Sibelius 2023.300 42 00 01
Sibelius 2023.800 43 00 07
Sibelius 2024.3.100 44 00 01

That is a lot of versions and I feel there may be some gaps that still need to be identified. It appears that the first two bytes are the major version and the second set of bytes is the minor version. Although it looks like a few major version bytes span across a few software versions. With this chart, one could be very specific in identifying which Sibelius version wrote the file, but for archiving purposes it seems we can group many of these capturing just the major version. The export screenshot above seems to have broken down significant changes and grouped similar formats together, the biggest being 8.6 through 2019.12. A comparison of “student” and “first” formats don’t have any obvious bytes which indicate as such, so for now they are all lumped together.

There is one other similar format which needs to be mentioned. Sibelius Scorch was a product made to share scores online. This has been replaced with Sibelius Cloud Publishing, but for awhile was the best way to share a score with others in a way that protected the original. I have no idea how they were made, but sites like scorestreet.net and sibeliusmusic.com were sites you could upload your score to for sharing. Some SCO files appear to have a PDF embedded within them for proper printing.

hexdump -C smd_h_0000000000097761.sco | head
00000000  0f 43 43 53 43 4f 52 43  48 00 00 36 00 1e 00 c0  |.CCSCORCH..6....|
00000010  d4 55 00 00 00 30 00 00  00 01 00 00 00 01 00 00  |.U...0..........|
00000020  00 22 0f 43 43 53 43 4f  52 43 48 00 00 36 00 1e  |.".CCSCORCH..6..|
00000030  00 00 00 00 00 00 00 3a  00 00 00 00 03 56 11 b9  |.......:.....V..|
00000040  70 dc fe 90 50 48 30 df  eb 39 88 23 8e 88 78 bf  |p...PH0..9.#..x.|
00000050  da ab ab 5b e2 13 98 89  66 eb 94 67 8d 16 00 00  |...[....f..g....|
00000060  00 00 cf 6f 0c 67 85 ec  57 90 e5 c1 ea 8a eb 9f  |...o.g..W.......|
00000070  c8 13 d2 1d 75 bd a5 9f  eb b9 ef 1d 25 79 45 2c  |....u.......%yE,|
00000080  05 bb 74 41 e8 8f 27 6a  01 07 d0 f5 3b 17 ce 87  |..tA..'j....;...|
00000090  7b c2 82 d9 41 6b 82 2f  d8 b8 17 32 fa d3 59 05  |{...Ak./...2..Y.|

I am not sure the best way to handle all the different versions within the PRONOM registry. I went ahead and made a few signatures based on the export dialog of Sibelius 2024. Even with combining a few together, it leaves us with 17 new PUID’s. Maybe further discussion can refine these down a bit more? Regardless, each file can be associated with a specific Sibelius version, making it easier to open and migrate if needed without fear of opening in the wrong version. Take a look at some samples and my signatures on my GitHub page and let me know if there is a better way.

Shorten

I was recently going through some of my old CD-R’s and came across this 11 year old fun memory.

I remember going to this 2003 Toad the Wet Sprocket concert in Salt Lake City with some friends, I had seen this band perform before, but this was the first time I was able to get a recording of the show. Normally having a recording of a concert of a well known band was a little shady, but for some bands, they not only allow recording of their live concerts, but they encourage it. There has been a few bands over the years who have this philosophy, one most have heard of is the Grateful Dead, because of all the tape trading, the band’s numerous concerts will live on forever.

The scene of recording concerts is still alive and well, and if you are into recording and sharing it is expected you share in a lossless audio format. The world of lossless audio is definitely in the minority of all those who listen to music on the daily. Most of us have been placated with the infinite playlists on services like Apple Music, Spotify, and Amazon Music. Most probably don’t care about owning music anymore, but for the few who consider themselves Audiophiles, having a lossless audio file is the only choice.

When it comes to formats, there are a few lossless formats to choose from, they all come with some advantages as well as some downsides. WAV files contain the full PCM audio stream, and while internet bandwidth today can handle full uncompressed audio, it can still be beneficial to use some compression for archiving or sharing over the web.

The most common lossless format today is the Free Lossless Audio Codec or FLAC, but there are also quite a few who like the Apple Lossless Audio Codec. Both offer many advantages, especially with metadata, cuesheets, and can contain cover album art. But many years ago another lossless format was most often used with bootleg recordings and audio sharing.

Shorten was one of the first lossless formats, developed by Tony Robinson in 1993 for SoftSound. It could cut the size in half of a typical 16-bit WAV file. It achieved this by using Huffman coding, kinda the same way a JPEG works, by reducing the frequency of how often patterns occur. Today FLAC and ALAC have replaced this format and offer improved features and support. Many audio players have dropped support for shorten making it difficult to use this old format.

The Shorten format uses the .SHN extension. It is one of the formats listed on the Library of Congress Sustainability of Digital Formats with the ID fdd000199, although a couple links don’t appear to work as it hasn’t been updated since 2011. Support was ended for this format and many of the links found on various websites are for broken, usually referencing the etree wiki. Much of which is archived on the Internet Archive.

Let’s take a look at the what makes up a lossless compressed SHN file. A quick look at a sample header:

hexdump -C test.shn | head
00000000  61 6a 6b 67 02 fb b1 70  09 f9 25 59 52 a4 d1 a8  |ajkg...p..%YR...|
00000010  dd cf 85 5a 01 57 a0 d5  a8 b6 6b 6d d2 41 10 80  |...Z.W....km.A..|
00000020  40 20 10 18 04 0a 01 44  d6 40 20 11 0d 8c 0a 01  |@ .....D.@ .....|
00000030  04 80 44 20 16 4b 0d d2  c3 b8 f8 55 a0 11 80 59  |..D .K.....U...Y|
00000040  98 56 1d b1 79 51 9f 39  f1 12 d2 d3 75 5c cd 08  |.V..yQ.9....u\..|
00000050  06 25 68 6b 52 5e 9f 4c  39 cd c1 32 c4 0d a9 b7  |.%hkR^.L9..2....|
00000060  69 34 56 f0 96 fa 46 89  a2 6e 8c ba d5 d0 58 de  |i4V...F..n....X.|
00000070  f5 44 5b aa 61 82 c7 85  88 37 d6 ee cb ab 4e 44  |.D[.a....7....ND|
00000080  91 19 b7 38 d4 20 ae 98  98 d1 2c 4a 4e 88 dd 3e  |...8. ....,JN..>|
00000090  36 68 1b 59 a8 7d 84 23  76 0a 84 21 a1 cd 80 8e  |6h.Y.}.#v..!....|

The first four bytes seem to be consistent among my samples. It makes me wonder if the ascii values have something to do with the author, Anthony (Tony) J. Robinson. In the source code for the shorten software, the file shorten.h defines the ascii “ajkg” as the magic header for the SHN format. Also found in current ffmpeg code. Although the tools don’t have much to say about them.

mediainfo test.shn 
General
Complete name                            : test.shn
Format                                   : Shorten
Format version                           : 2
File size                                : 3.17 MiB

Audio
Format                                   : Shorten
Compression mode                         : Lossless

ffprobe -i test.shn
Input #0, shn, from 'test.shn':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Audio: shorten, 44100 Hz, 2 channels, s16p

Using the older SHNTOOL, we can get more information.

shntool info test.shn  
-------------------------------------------------------------------------------
File name:                    test.shn
Handled by:                   shn format module
Length:                       0:32.23
WAVE format:                  0x0001 (Microsoft PCM)
Channels:                     2
Bits/sample:                  16
Samples/sec:                  44100
Average bytes/sec:            176400
Rate (calculated):            176400
Block align:                  4
Header size:                  44 bytes
Data size:                    5697720 bytes
Chunk size:                   5697756 bytes
Total size (chunk size + 8):  5697764 bytes
Actual file size:             3325489
File is compressed:           yes
Compression ratio:            0.5836
CD-quality properties:
  CD quality:                 yes
  Cut on sector boundary:     no
  Sector misalignment:        1176 bytes
  Long enough to be burned:   yes
WAVE properties:
  Non-canonical header:       no
  Extra RIFF chunks:          no
Possible problems:
  File contains ID3v2 tag:    no
  Data chunk block-aligned:   yes
  Inconsistent header:        no
  File probably truncated:    unknown
  Junk appended to file:      unknown
  Odd data size has pad byte: n/a
Extra shn-specific info:
  Seekable:                   yes

Many Shorten Audio Files are found out there in archives and file sharing sites, so even though the format isn’t used to create new files, it will still be around for awhile. My GitHub has my signature proposal and a couple of samples.