
Here it finally is - the _REAL_ .XM format. This is basically the  
XM.TXTfile that comes with Fasttracker II, but with a lot of comments  
andcorrections added. All modifications are clearly marked and the  
originaltext is still there, so you can see how much they really managed  
to screwup.. Funny - for example, we still haven't found a single error in  
theScream Tracker 3 module formats. Probably tells you something about  
theattitide of the coders...

If you look through this file you'll probably understand why we  
musicsystem coders have had so much trouble in supporting the format. It  
ispretty hard to play a module correctly if you can't even load it in  
thefirst place. I hope this also serves to stop the "why no XM  
support"complaints in Usenet and various diskmags. And, of course, I think  
thatmost anyone who is planning to support XM modules or already has  
someworking code will find this useful. I haven't included any  
informationabout the effects or playing in general as it would have made  
this filefar too long (and I was lazy in the first place), but that may  
followlater if there is interest. FT2 manages to screw up quite a few  
effectswhen compared to Protracker and there are several other oddities as  
well.For example:

1. Protracker note retrig (the E-command) does not retrig in 0-ticks(when  
note data is played just like PT) _except_ when the infobyte is 0.Then it  
retrigs always on tick 0.
2. Envelope position is resetted _always_ when there is an  
instrumentnumber, even in tone portamento effects. So no wonder tone  
portamentoscan sound wrong...
3. Sample Offset is not Protracker compatible (small wonder).

And so on. Anyway, here is the format. Have fun and e-mail any  
comments(preferably to me, it's faster). Also keep on lookout for the new  
32-bitMIDAS with XM-support. Hopefully some beta will be released next  
weekend,along with a new MIDAS Module Player which will play XMs better  
thananything except FT2 (Cubic Player screws up quite a few effects...).

Oh. Final plea for help (once again): Anyone know about  
instrumentautovibrato? Or got Mr.H's e-mail address?

...cut...

       The XM module format description for XM files version $0104.

       By Mr.H of Triton in 1994.

       - Be prepared! Are you sure you want to know? :-)

<< A LOT of comments, corrections and notes added by Guru and
Alfred of Sahara Surfers in 1995. All additions are included in
double angle brackets << like this >> and everything outside
them is left intact. Note the HUGE amount of errors in the
original text. Triton: Perhaps you'd like to include THIS in the
next FT2 release? :-)

Not copyrighted, released into public domain (at least the
additions by us). Feel free to do what you wish. Credits please.

Remember - the added information is based on REAL XM files. >>

 ============================================================================


   ******************************
   *   The XM file structure:   *
   ******************************

   Offset Length Type

      0     17   (char) ID text: 'Extended module: '
                                << Nope: 'Extended Module: ' >>
     17     20   (char) Module name, padded with zeroes
     37      1   (char) $1a
     38     20   (char) Tracker name
     58      2   (word) Version number, hi-byte major and low-byte minor
                        The current format is version $0103
                        << Nope, it's $0104. Format versions
                        below $0104 have a LOT of differences.
                        Check this field! >>

     60      4  (dword) Header size
                        << Calculated FROM THIS OFFSET, not from
                        the beginning of the file! >>
     +4      2   (word) Song length (in patten order table)
     +6      2   (word) Restart position
     +8      2   (word) Number of channels (2,4,6,8,10,...,32)
    +10      2   (word) Number of patterns (max 256)
    +12      2   (word) Number of instruments (max 128)
    +14      2   (word) Flags: bit 0: 0 = Amiga frequency table (see below);
                                      1 = Linear frequency table
    +16      2   (word) Default tempo
    +18      2   (word) Default BPM
    +20    256   (byte) Pattern order table

                        Patterns:
                        ---------

      ?      4  (dword) Pattern header length
     +4      1   (byte) Packing type (always 0)
     +5      2   (word) Number of rows in pattern (1..256)
     +7      2   (word) Packed patterndata size
                        << Note! This is zero if the pattern is
                        completely empty and no pattern data
                        follows! >>

      ?      ?          Packed pattern data

                        Instruments:
                        ------------

      ?      4  (dword) Instrument size << header that is >>
     +4     22   (char) Instrument name
    +26      1   (byte) Instrument type (always 0)
                        << This seems pretty random, don't
                        assume it's zero >>
    +27      2   (word) Number of samples in instrument

   If the number of samples > 0, then the this will follow:

   !     +29      4  (dword) Sample header size
   !     +33     96   (byte) Sample number for all notes
   !    +129     48   (byte) Points for volume envelope
   !    +177     48   (byte) Points for panning envelope
   !    +225      1   (byte) Number of volume points
   !    +226      1   (byte) Number of panning points
   !    +227      1   (byte) Volume sustain point
   !    +228      1   (byte) Volume loop start point
   !    +229      1   (byte) Volume loop end point
   !    +230      1   (byte) Panning sustain point
   !    +231      1   (byte) Panning loop start point
   !    +232      1   (byte) Panning loop end point
   !    +233      1   (byte) Volume type: bit 0: On; 1: Sustain; 2: Loop
   !    +234      1   (byte) Panning type: bit 0: On; 1: Sustain; 2: Loop
   !    +235      1   (byte) Vibrato type
   !    +236      1   (byte) Vibrato sweep
   !    +237      1   (byte) Vibrato depth
   !    +238      1   (byte) Vibrato rate
   !    +239      2   (word) Volume fadeout
   !    +241      2   (word) Reserved
        << "Instrument Size" field tends to be more than the
        actual size of the structure documented here (it
        includes also the extended instrument sample header
        above). So remember to check it and skip the additional
        bytes before the first sample header >>
   !
   !                         Sample headers:
   !                         ---------------
   !
   !       ?      4  (dword) Sample length
   !      +4      4  (dword) Sample loop start
   !      +8      4  (dword) Sample loop length
   !     +12      1   (byte) Volume
   !     +13      1   (byte) Finetune (signed byte -16..+15)
   !     +14      1   (byte) Type: Bit 0-1: 0 = No loop, 1 = Forward loop,
   !                                        2 = Ping-pong loop;
   !                                     4: 16-bit sampledata
   !     +15      1   (byte) Panning (0-255)
   !     +16      1   (byte) Relative note number (signed byte)
   !     +17      1   (byte) Reserved
   !     +18     22   (char) Sample name

        << Note! After the instrument header the file contains
        ALL sample headers for the instrument followed by the
        sample data for all samples >>
   !
   !                         Sample data:
   !                         ------------
   !
   !       ?      ?          Sample data (signed): The samples are stored
   !                         as delta values. To convert to real data:
   !
   !                         old=0;
   !                         for i=1 to len
   !                            new=sample[i]+old;
   !                            sample[i]=new;
   !                            old=new;



   ***********************
   *   Pattern format:   *
   ***********************

   The patterns are stored as ordinary MOD patterns, except that each
   note is stored as 5 bytes:

      ?      1   (byte) Note (0-71, 0 = C-0)
                        << Hah. Actually note numbers are 1-96,
                        97 being the key-off note (try to find
                        that in the documentation! >>
     +1      1   (byte) Instrument (0-128)
                        << More like 1-128 >>
     +2      1   (byte) Volume column byte (see below)
     +3      1   (byte) Effect type
     +4      1   (byte) Effect parameter

   A simle packing scheme is also adopted, so that the patterns not become
   TOO large: Since the MSB in the note value is never used, if is used for
   the compression. If the bit is set, then the other bits are interpreted
   as follows:

      bit 0 set: Note follows
          1 set: Instrument follows
          2 set: Volume column byte follows
          3 set: Effect follows
          4 set: Guess what!

   It is very simple, but far from optimal. If you want a better,
   you can always repack the patterns in your loader.


   ******************************
   *   Volumes and envelopes:   *
   ******************************

   The volume formula:

   FinalVol=(FadeOutVol/65536)*(EnvelopeVol/64)*(GlobalVol/64)*(Vol/64)*Scale;

   << The FadeOutVol is originally 65535 and is decremented by
   instrument.fadeout each tick after note is released >>

   The panning formula:

   FinalPan=Pan+(EnvelopePan-32)*(128-Abs(Pan-128))/32;

   << Note that the panning envelope values range from 0-64, not
   -128 - +127 >>

      Envelope:
      ---------

   The envelopes are processed once per frame, instead of every frame where
   no new notes are read. This is also true for the instrument vibrato and
   the fadeout. Since I am so lazy and the tracker is rather self-explaining
   I am not going to write any more for the moment.

   << Anyone have ANY idea how instrument autovibrato works? >>


   ********************************
   *   Periods and frequencies:   *
   ********************************

   PatternNote = 0..95 (0 = C-0, 95 = B-7)
        << Blah: Note = 1..96, 1 = C-0, 97 = key off >>

   FineTune = -128..+127 (-128 = -1 halftone, +127 = +127/128 halftones)
   RelativeTone = -96..95 (0 => C-4 = C-4)

   RealNote = PatternNote + RelativeTone; (0..118, 0 = C-0, 118 = A#9)

      Linear frequence table:
      -----------------------

   Period = 10*12*16*4 - Note*16*4 - FineTune/2;
   Frequency = 8363*2^((6*12*16*4 - Period) / (12*16*4));

      Amiga frequence table:
      ----------------------

   Period = (PeriodTab[(Note MOD 12)*8 + FineTune/16]*(1-Frac(FineTune/16)) +
             PeriodTab[(Note MOD 12)*8 + FineTune/16]*(Frac(FineTune/16)))
            *16/2^(Note DIV 12);
      (The period is interpolated for finer finetune values)
   Frequency = 8363*1712/Period;

   << The interpolation code above doesn't work because of
   several reasons:
        1. It does not interpolate (try adding 1 to the
        PeriodTab index in the second line)
        2. It may go past the table beginning for negative
        finetune values
   Write your own interpolation routine instead - it's not that
   hard. >>

   PeriodTab = Array[0..12*8-1] of Word = (
      907,900,894,887,881,875,868,862,856,850,844,838,832,826,820,814,
      808,802,796,791,785,779,774,768,762,757,752,746,741,736,730,725,
      720,715,709,704,699,694,689,684,678,675,670,665,660,655,651,646,
      640,636,632,628,623,619,614,610,604,601,597,592,588,584,580,575,
      570,567,563,559,555,551,547,543,538,535,532,528,524,520,516,513,
      508,505,502,498,494,491,487,484,480,477,474,470,467,463,460,457);

   << Note! The period table is made for 1-based note numbers, so in
      practise it contains the period values for B-3 to G#4. Fun. >>


   *************************
   *   Standard effects:   *
   *************************

      0      Appregio           << ;) >>
      1  (*) Porta up
      2  (*) Porta down
      3  (*) Tone porta
      4  (*) Vibrato
      5  (*) Tone porta+Volume slide
      6  (*) Vibrato+Volume slide
      7  (*) Tremolo
      8      Set panning
      9      Sample offset
      A  (*) Volume slide
      B      Position jump
      C      Set volume
      D      Pattern break
      E1 (*) Fine porta up
      E2 (*) Fine porta down
      E3     Set gliss control
      E4     Set vibrato control
      E5     Set finetune
      E6     Set loop begin/loop
      E7     Set tremolo control
      E9     Retrig note
      EA (*) Fine volume slide up
      EB (*) Fine volume slide down
      EC     Note cut
      ED     Note delay
      EE     Pattern delay
      F      Set tempo/BPM
      G      Set global volume
      H  (*) Global volume slide
      K      Key off            << also note number 97 >>
      L      Set envelope position
      P  (*) Panning slide
      R  (*) Multi retrig note
      T      Tremor
      X1 (*) Extra fine porta up
      X2 (*) Extra fine porta down

      (*) = If the command byte is zero, the last nonzero byte for the
            command should be used.

<< The commands are reasonably Protracker compatible although
not all PT "features" (some might call them replay routine bugs)
are implemented. >>

   *********************************
   *   Effects in volume column:   *
   *********************************

   All effects in the volume column should work as the standard effects.
   The volume column is interpreted before the standard effects, so
   some standard effects may override volume column effects.

   Value      Meaning

      0       Do nothing
    $10-$50   Set volume Value-$10
      :          :        :
      :          :        :
    $60-$6f   Volume slide down
    $70-$7f   Volume slide up
    $80-$8f   Fine volume slide down
    $90-$9f   Fine volume slide up
    $a0-$af   Set vibrato speed
    $b0-$bf   Vibrato
    $c0-$cf   Set panning
    $d0-$df   Panning slide left
    $e0-$ef   Panning slide right
    $f0-$ff   Tone porta


<< And here goes one more section that this document really
should have had:

   ********************
   *  General layout  *
   ********************

   The layout of a typical XM is like this:

   XM header (up to and excluding header size field, now 60 bytes)
 Rest of the header (length including header size field)
   Pattern 0 header (length in header)
 Pattern 0 data (length in header)
   Pattern 1 header (length in header)
 Pattern 1 data (length in header)
   ... (* number of patterns)
   Instrument 0 header (length in size field is this and next one together)
 if (numSample > 0)
                Extra header
   Sample 0 header (size in instrument extra header)
   Sample 1 header (size in instrument extra header)
   ... (* number of samples in instrument)
    Sample 0 data (length in sample header)
    Sample 1 data (length in sample header)
    ... (* number of samples in instrument)
   Instrument 1 header (length in size field is this and next one together)
        ... (* number of instruments)
>>



 ============================================================================

   This should be just about everything (I hope?). You will probably need
   some information about the MOD format and maybe about S3M.

   Have fun!
   << And we sure had! >>

             Fredrik Huss / Mr.H of Triton

   << Jarno Paananen & Petteri Kangaslampi - Guru & Alfred / S2

   If you have questions that you think we might be able to
   answer, have found out more mistakes in this document, or
   know how instrument autovibrato works, e-mail us at:
   Jarno: jpaana@kauhajoki.fi
          _or_
   Petteri: pekangas@sci.fi
   >>
...cut...

--
/ Petteri Kangaslampi -- Sahara Surfers -- C, C++, 80386 ASM \
\         E-MAIL ONLY TO pekangas@sci.fi, PLEASE!            /
