Building a style library

Styles are MIDI loops included on digital keyboards from Yamaha and other manufacturers from which you can build automatic accompaniments. The feature, which makes a good approximation to a backup band, can turn a simple tune into an interesting performance piece. Yamaha-format styles form the core of our program, the Accompaniment Machine. A typical keyboard has 100-200 built-in styles. With the Accompaniment Machine, you have access to thousands of unique styles available as files on the Internet[1]. The program enables application of the vast archive of Yamaha-format styles to any keyboard brand or model.

A continuing project at KBD-Infinity is to build a comprehensive, standardized style library for the Accompaniment Machine. Choosing the best style for a piece can be a challenge because of the sheer volume of material available. The task is made more difficult by the disorganization of available resources. An archive like the PSR Tutorial contains a huge number of style files, but there is a great deal of redundancy for the following reasons:

  • Many style sets have been appropriated from Yamaha keyboards over the years. Thus, there are hundreds of available sets for specific keyboards, all with similar content. Often, the only differences are variations in orchestration (e.g., device-specific XG voices and drumsets).
  • Although there are many original style sets by independent artists, some developers make only minor changes in orchestration, tempo or format and release the styles with new names.
  • Over the decades, Yamaha has added several arcane features to style files and has introduced a break in the format, SFF1 to SFF2 [2]. There is no official documentation of either format.

The result is considerable frustration for users who search large style sets only to continually encounter the same musical content in different wrappers.

We had two goals in building our style libraries:

  • Create a standardized file format that performs well, either on any digital keyboard via the Accompaniment Machine or directly when loaded on a Yamaha keyboard.
  • Significantly reduce corrupt files and redundancy.

To meet the first criterion, we created the PureStyle format described in the previous article. PureStyle files have the following characteristics:

  • A standardized file name that documents the base tempo and time signature.
  • Elimination of unnecessary and confusing information. The file contains only MIDI data to that conforms to the ideal style standard.
ShapeShifter screen shot

Figure 1. ShapeShifter screen shot.

This article discusses ShapeShifter (Fig. 1), a utility we developed for automatic generation of standard libraries with minimal redundancy. Our initial step in building a library is to gather styles from a variety of sources [1] and to sort them into groups (e.g., Latin styles). The result is a directory that may contain hundreds of style files in a variety of formats. Eliminating musically-similar styles by ear from such a large set would be difficult and time consuming. ShapeShifter corrects and analyzes all files in a directory automatically. The user simply navigates to the working directory and clicks the Convert to PureStyle button. In response, the program performs the following operations:

1) Loop through the files and pick out the ones that are not in PureStyle format. For each member of the set, determine if it is in SFF1 or SFF2 format and then read the contents. Delete the file if it is corrupted. Decode any channel redirection or key changes defined in the CASM section and use the information to correct musical entries in the MIDI section to meet the following requirements:

Rule 1. Only upper MIDI channels are used for the style, leaving the lower channels free for melody voices. The channels serve specific functions, with Channel 09h reserved for percussion.

Rule 2. The pitch instruments should play in the key of C and suggest a CMaj7 chord.

Create a standardized name for the style and write a new file in PureStyle format.

2) Optionally, delete the original style file.

3) Loop through the set of PureStyle files in the directory and record the following information in arrays for each file: the name, the byte length, the tempo, and the time-signature numerator and denominator. ShapeShifter also records a quantity called NoteSum, equal to the sum of note values for all NoteOn messages in the MIDI section.

4) Remove duplicate style files. To do this, we need a criterion for duplication. We recognize that two files with different names, byte lengths, tempos or instrument settings may have identical MIDI roots. There is no reason to include both of them, because it is easy to change tempos or to modify instrument assignments within the Accompaniment Machine or Style Customizer. The key to recognizing duplicates is the NoteSum value, equal if the style files share a common MIDI root. ShapeShifter defines a Boolean array NErase with entries initialized to False. The program loops through each member of the style set that has not been previous marked NErase(n) = True. For each style, the program searches the remainder of the array, marking any styles that share the same time signature and NoteSum value as NErase(m) = True. When complete, the program makes another loop, deleting any style files marked for erasure.

Optionally, ShapeShifter keeps a log of all operations. Here’s typical output for processing a raw group of files purchased from the PSR Tutorial site:

Convert non-pure-style files
 Invalid file format: 16B Pop.sty
 Invalid file format: 8 Beat Roc.sty
 8Bt Standa.sty ==> 8BtStanda_107_4-4_ps.sty
 Ballad3.sty ==> Ballad3_104_2-4_ps.sty
 Basic Rock.sty ==> BasicRock_116_4-4_ps.sty
 Blues R&Ro.sty ==> BluesR&Ro_123_4-4_ps.sty
 C Beat1.sty ==> CBeat1_120_4-4_ps.sty
 ...
 Invalid file format: Singer Son.sty
 Thunderer.sty ==> Thunderer_125_4-4_ps.sty
 WmTellOver.STY ==> WmTellOver_120_4-4_ps.sty
 Number of successful conversions: 24
Delete remaining non-pure-style files
 Delete file: 16B Pop.sty
 Delete file: 8 Beat Roc.sty
 ...
 Delete file: WmTellOver.STY
 Number of files deleted: 30
Analysis of pure style file properties
 n  STempo  STsUp  STsDn   SFileSize  SNoteSum    SFileName
 ====================================================================
 0      107     4     4    14499        80037    8BtStanda_107_4-4_ps.sty
 1      104     2     4    11720        62755    Ballad3_104_2-4_ps.sty
 ...
 22      125     4     4    25841       187149    Thunderer_125_4-4_ps.sty
 23      120     4     4    27775       193746    WmTellOver_120_4-4_ps.sty

In this case all files in the directory are unique, so no files are eliminated. For comparison, these are log entries when there are a number of overlapping style files:

Overlap
 n:    1 Tempo: 120  TsUp: 4  TsDn: 4  NoteSum:  67628  FileSize:  5408 Acoustica_120_4-4_ps.sty
 m:   30 Tempo: 120  TsUp: 4  TsDn: 4  NoteSum:  67628  FileSize:  5408 ChristmasBalladLaidBack_120_4-4_ps.sty
 Overlap
 n:    3 Tempo:  94  TsUp: 4  TsDn: 4  NoteSum: 102811  FileSize:  7553 AlleJahreWieder_94_4-4_ps.sty
 m:  121 Tempo:  94  TsUp: 4  TsDn: 4  NoteSum: 102811  FileSize:  7553 ChristmasQuietBallad_94_4-4_ps.sty
 Overlap
 n:    4 Tempo: 120  TsUp: 4  TsDn: 4  NoteSum: 109838  FileSize: 16226 AllIWant4Xmas_120_4-4_ps.sty
 m:  156 Tempo: 120  TsUp: 4  TsDn: 4  NoteSum: 109838  FileSize: 16226 ChristmasTwoFrontTeeth_120_4-4_ps.sty
 Overlap
 n:    5 Tempo:  58  TsUp: 3  TsDn: 4  NoteSum:  74444  FileSize: 11166 AmazingGospel_58_3-4_ps.sty
 m:   73 Tempo:  58  TsUp: 3  TsDn: 4  NoteSum:  74444  FileSize: 11166C hristmasGospel_58_3-4_ps.sty
 ...

In the end, the directory contains a set of style files in the standardized PureStyle format with no corrupted styles and highly reduced redundancy.

Footnotes

[1] See, for instance, the Style section of the Unofficial YAMAHA Keyboard Resource Site.

[2] See, for instance What’s In a Yamaha Style File.

[3] Find out more about KBD-Infinity: Home page.

[4] If you have comments or questions, please contact us at info@kbd-infinity.com.

 

Comments are closed.