Player identification (pattern searching)

Telling which music routine — which sid-player-routine — a .sid file was made with, by looking for a short sequence of code bytes that only that routine contains. Stefano Tognon set the method out in sidin #3 and #5 (2003–04) together with Lada 'Ray' Lostak's database of about 450 engines; the same idea, with a far longer list, is what sidid does for hvsc today (sources: s-sidin03-pattern-searching-1, s-sidin05-pattern-searching-2, s-sidid-player-list).

Why a tune "has" a player#

A SID file carries the player code together with the song data (sid-format). The data changes with every tune, the code does not — the same routine is linked into every song made with a given editor or driver, only at a different address. A run of opcodes that handles something peculiar to that routine (its command bytes, its instrument layout, how it does portamento) is therefore a fingerprint, provided the parts that change with relocation are ignored (source: s-sidin03-pattern-searching-1).

Making a signature (SIDin #3)#

• With source or a disassembly you understand: choose code that deals with the routine's own instruments, patterns or note format, or with an effect. The galway-player treats byte-code values ≥ $C0 as instructions, so its fetch loop LDY #$00 / LDA ($xx),Y / CMP #$C0 / BCC is a seven-byte signature (six fixed bytes and one wildcard — "very little", so it may collide with another engine); the matt-gray-player's double CMP #$FB portamento test is a longer, "very characteristic" one (source: s-sidin03-pattern-searching-1).
• Without: disassemble two or more tunes known to use the player (the author says so, or you made a test tune in the editor), align the listings side by side, and take a stretch that all versions of interest share and their siblings lack. Tognon's example separates JCH NewPlayer v1 (Hoppin, Brown Ice), v2 (Cavern) and v3 (2Cvee) this way (chordian); understanding what the stretch does is not required — "we only need a pattern that find that player" (source: s-sidin03-pattern-searching-1).
• Wildcards: absolute addresses are relative to where the player is linked, so they become .. (BD ?? ?? B9 ?? ??); only I/O addresses ($D4xx) stay; zero-page operands are wildcarded too because some players let the user choose them; a branch offset is dropped when its target lies outside the pattern. In Perl . is any byte, \x.. a fixed one, and the test is $data =~ /$Search/s (source: s-sidin03-pattern-searching-1).

The Lostack database (SIDin #5)#

One Player { … } block per engine (source: s-sidin05-pattern-searching-2):

• The offsets are relative to the scan position: the database descends from the C64 program Advanced Music Searcher (AMS 5.0.415), which stepped through memory at fixed intervals (most probably $100, since players load at $1000 "but not a $1001"). On a PC a SID player, emulator or rip tool can scan byte by byte instead — with the risk that four-byte AMS entries then produce false matches; Simon White's scanner ran all 450 signatures over the whole HVSC "in less than 5 seconds" (source: s-sidin05-pattern-searching-2).
• What the positions are for: a player program can show live where the routine is in the song, and it can find a tune's true length from the track end marks ("usually $FF and $FE" — one restarts the track, the other silences the voice): with restarts the tune has looped only when all three voices have wrapped, with stops it has ended once SecPos stops changing (orderlist). Players with a single track for all voices get the same address three times (source: s-sidin05-pattern-searching-2).
• Worked example: FUTURE COMPOSER 1.0 — ID=$0006:$AD / $0007:$74 / $0009:$C9 / $000A:$02, OrderPos=$0921,$0922,$0923, SecPos=$0924,$0924,$0926; the IDs are the opcode and low operand byte of LDA $2174 and the CMP #$02 at the start of a tune loaded at $1800, low bytes surviving because tunes are relocated in $100 steps; "more than 1000 tunes in HVSC use this player" (future-composer) (source: s-sidin05-pattern-searching-2).

Signatures quoted in the sources#

From here to SIDId#

Lostak's database was GPL, read only by his own player (www.unreal64.net), and its four-byte AMS entries were its weakness; Tognon planned an extended syntax (relative addresses inside instructions, using the program counter) and tools that propose patterns, and started hvmec so that test tunes could be made in the editors themselves — "you can use the players for founding the right pattern" (sources: s-sidin05-pattern-searching-2, s-sidin05-hvmec). Lasse Öörni's sidid is the form the idea took: sidid.cfg holds byte patterns for about 1 300 named players and editors, with sub-variants in parentheses, and sidid.nfo the author, year and CSDb reference per entry — the list HVSC uses to label a file's player (source: s-sidid-player-list).

To verify#

• (unverified) Whether SIDId's pattern syntax descends from the Lostack format or was written independently; the wiki's SIDId summary describes the list, not its history.
• (unverified) The magazine's "TSID" (Tognon's Time Sid Manager, announced in the news pages) is a separate HVSC statistics tool, not this identification project; the two share only his web directory ice00/tsid/.

Related#

sidid · sid-player-routine · hvsc · hvmec · sid-format · orderlist · future-composer · galway-player · matt-gray-player · chordian · sidin · siddump

Sources#

s-sidin03-pattern-searching-1 · s-sidin05-pattern-searching-2 · s-sidin05-hvmec · s-sidid-player-list