Removing measures from MIDI files

Recently while creating a song arrangement, I had an application where I needed to remove measures from a MIDI file. In this case, the file was a section of a Yamaha-format style (IntroB) exported from a style file using Style Master. The overly complex introduction meandered on for eight measures — the last four measures would have been sufficient. So all I had to do was to remove the first four measures. Seemed simple.

The MIDI editors I discussed in a previous article failed at the task. MidiEditor could remove individual notes, but was unable to delete measures, leaving four measures of silence. Speedy MIDI could remove measures, but corrupted the file beyond use. The next step was to search the Internet for a utility. I couldn’t find anything, but I did encounter several posts from people seeking the same function. Accordingly, I decided to add a measure removal capability to MIDI Doctor 2.0.

Figure 1 shows the MIDI Doctor 2.0 interface. New features include a indication of total measures in the file (A), a measure and time counter added to the playback function (B) and new Global settings capabilities (C).

MIDI Doctor 2.0 main window.

Figure 1. MIDI Doctor 2.0 main window.

Figure 2 shows the Global settings dialog. There are three control groups. The Dynamic range group adjusts volume levels of all note messages in a file. A user can use the Play duration group to set a precise temporal length for the MIDI file. The measure removal function is accomplished with the controls of the Make excerpt group. The radio buttons determine whether the excerpt is specified by time or measure number. In the first case, the Start time and End time text fields are active. Their initial values are set to the beginning and end of the file. The user changes the start and end times and clicks Apply to narrow the file duration. Because of MIDI conventions, the excerpt process is not simply a case of saving messages within the time window. The effect of a MIDI message continues indefinitely until a following message counteracts it. Therefore, the state of the synthesizer at any time represents the cumulative effect of preceding messages. Eliminating these messages may result in incorrect instrument voices or volumes. Similarly, deletion of messages after the excerpt may leave hanging notes that play to infinity. MIDI Doctor uses the following process:

Global settings dialog.

Figure 2. Global settings dialog.

  • The absolute times of all messages before TStart are set to t = 0. The velocity values of all NoteOn messages are set to 0.
  • The absolute times of messages between TStart and TEnd are shifted to t = t – TStart.
  • Messages after TEnd are deleted. A message AllNotesOff is appended to prevent hanging notes.

Transfer of the sequence of initial non-sounding messages is limited by the MIDI serial rate. As a result, there may be a slight starting delay for large files. Measures are removed by clicking the By measure radio button, filling in the first and last measures to include in the excerpt and clicking the Apply button. Global operations are performed on MIDI message values stored in memory and determine the duration of the file on save.

New features in StyleMaster

We improved the organization of controls and added new features in the latest version of StyleMaster. Figure 1 shows the current interface. As in all our programs, it is uncluttered and organized for quick reference. The controls of Group A are used to load and save individual style files with specified modifications. The controls of Group B are for bulk modification of all style files in a folder. The Group C controls are used to export styles in a conventional MIDI format and to assemble MIDI file sets into a formatted style. The playback controls of Group D are used to preview styles. Finally, Group E controls are for modifying channel voices (instrumentation). We eliminated the redundant Play/Pause buttons. Now, left-clicking on a section button (e.g. Main A) plays the section and displays the number of measures. Left-click again to stop the section before it reaches the end.

 

Improved StyleMaster interface

Figure 1. Improved StyleMaster interface.

The Global settings button represents a new feature. In response, the program shows the dialog of Fig. 2. The function is to equalize the voice and/or volume settings over all channels. One application is the conversion of a style with multiple instruments to a single instrument version (for example, to create a pure piano accompaniment). The other application is to help you adapt a style for your keyboard. There are 10’s of thousands of styles available on the Internet, each with a specific definition of MIDI voices at different volume levels that probably made a good sound on the original instrument. Many times you might get lucky, but often the style in its original form is poorly matched to your keyboard. In principle, you could correct each voice independently using the commands of Group E on the main screen. In practice, this can be time-consuming and frustrating because of the interactions of sound and volume levels. It can be more effective to start from a simple base and build up.

Global settings dialog

Figure 2. The new Global settings dialog.

The commands of the Voice equalization group perform the following functions if the Voice equalization box is checked:

  1. Set all melody channels to General MIDI with the same GM number. The four top choices often lead to a good stand-alone sound. Alternatively, you can pick a specific GM instrument (Other).
  2. Set Channel 09h (and Channel 08h, if it is used for percussion) to the standard MIDI drum set.
  3. Remove advanced settings like reverb and timbre.

If the Volume equalization box is checked, the volume levels of all channels are set to the specified value. Click the OK button apply the changes and exit the dialog. Figure 3 shows a resulting state of the voice list box after such an operation. StyleMaster also changes the state of the current MIDI device so you can preview the sound and monitor the result of further changes. A subsequent Save style operation records the new voice settings.

Globally modified voices

Figure 3. Globally modified voices.

 

Accompaniment Machine on digital pianos

Although AMac was designed for digital keyboards with a full complement of MIDI voices, the program is quite useful for digital pianos. We tested it recently on the popular Yamaha P125 (Fig. 1). There are two main differences between digital keyboards and pianos:

  • The physical keyboard of the digital piano provides a close simulation of the action of a mechanical piano.
  • There are fewer voices on a digital piano, but they are of high quality.
Yamaha P125 digital piano

Figure 1. Yamaha P125 digital piano.

The P125 has 24 voices in six categories:

  • Piano (grand piano, live grand, ballad grand, bright grand).
  • Electric piano (stage, DX, vintage, synth)
  • Organ (jazz, rock, principal, tutti)
  • CLV/VIB (harpsichord 8′, harpsichord 8’+4′, electric clavichord, vibraphone)
  • Strings (strings, slow strings, choir, synth pad)
  • Bass (acoustic, electric, bass and cymbal, fretless)

The digital piano also has a full set of drum sounds.

Despite the voice limitations, styles rendered by the Accompaniment Machine have a good sound on a digital piano, even with no adjustments. The Yamaha device makes sensible substitutions. In general, the resulting sound is more intimate. A big band style transforms to a combo. Often, the simplified sound is more tasteful and pleasing than the original.

 

 

Tap-Tap-Tap-Redux

In a previous post, I discussed how use the Export/Import features of Style Master in combination with a MIDI editor to remove unwanted content of Yamaha-format style sections (like the downbeat that occurs in most IntroA sections). At that time, I recommended Speedy MIDI as the editor, largely because it had an attractive interface. With further testing, I found that the program had severe faults for the application:

  • It added spurious content in text commands to the modified MIDI file specific to the program (like the display color of tracks)
  • It did inexplicable things, like changing the value of pulses/quarter note.

The second item was a fatal flaw. Unfortunately, it was difficult to find a MIDI editor (free or commercial) that didn’t have at least one serious problem. I have tried ten different editors to perform what should be a simple task. Each one had its own way of corrupting the MIDI file and making unwanted modifications. Finally, I found one that worked, MIDI Editor. Besides preserving the structure of MIDI files, it has full functionality, good online instructions and an attractive interface. Accordingly, I am rewriting the article to feature MIDI Editor.

Most Yamaha-format accompaniment styles for keyboards include multiple introductory sections (IntroA, IntroB, IntroC). The loose convention is that IntroA is relatively simple and starts with a measure of downbeats. This presents a problem if you want to use the introduction in a performance. Furthermore, the downbeat is unnecessary in the Accompaniment Machine — the program includes a visual beat indicator. Here’s a quick way to silence the taps when you’re in the AutoSequence mode of the Accompaniment Machine:

 -4 0 IntroA
 -4 0 StyleVolume 0.00
 -4 0 G
 -3 0 StyleVolume 1.00
  1 0 MainA

The command sequence drops the volume to zero through one measure of the five measure introduction. In performance, you need to remember that there will be a measure of silence after you start the accompaniment.

A alternate solution is to modify the style to eliminate the measure. I’ll go through the steps in this article. We’ll see how to use the Export and Import functions of Style Master.

 

Step 1. Use MIDI File Organizer (or another file management program) to set up a work environment. Figure 1 shows an example. The working directory, C:\TEMP, contains a subdirectory to hold individual sections of the style.

Set up a directory to hold the individual style sections.

Figure 1. Set up a directory to hold the individual style sections.

 

Step 2. Run Style Master and load the style (Figure 2). If you preview IntroA, here’s what it sounds like:

It’s a nice introduction to the song if we can eliminate the downbeat. Click the Export button, navigate to the directory C:\TEMP\SmallHotel and click OK. Style Master breaks the style into the individual MIDI sections (right-hand window of Fig. 1).

Style Master with the sample style loaded

Figure 2. Style Master with the sample style loaded.

Step 3. Run MIDI editor and load and load IntroA.mid. As shown in Fig. 3, the downbeat notes in the percussion channel are easy to identify. Use the mouse to select the first note. Full information on the MIDI message appears at bottom right. Press the Delete key to remove the note and repeat the operation for the other notes in the downbeat. Then save the modified file.

IntroA in MIDI Editor

Figure 3. IntroA in MIDI Editor.

Step 4. Click the Import button in Style Master, choose the directory C:\TEMP\SMALLHOTEL\ and click OK. The section files are reloaded, reformatted if necessary and organized. Now when you preview IntroA, the downbeat is gone. The final step is to save the modified style.

 

AMac: NC and fadeout

In this article I’ll cover two useful techniques for the Autosequence Mode of the Accompaniment Machine: NC and fadeout. The acronym NC often appears in fakebooks. It stands for No chord and designates an interval where the accompaniment stops while the solo continues. In the Autosequence Mode, you can pause the accompaniment and restart it by pressing the Control key. The accompaniment restarts exactly where it left off, and the result may be a jarring or unsynchronized sound. A better approach is the use the StyleVolume command to suppress the accompaniment during the NC section. In this case, the accompaniment maintains a predictable timing.

AMac sequence displayed in ConText

Figure 1. AMac sequence displayed in ConText.

Figure 1 shows how an instance of NC appears in an AMac sequence for the song Love Me Do. The sequence is displayed in the ConText editor using the syntax highlighting definitions that we supply with AMac. The NC starts on the second beat of measure 9. The accompaniment restarts at measure 10. MIDI signals are processed sequentially, so you may need to experiment with the exact positions of the volume changes to avoid sound discontinuities. Here is an example of extreme applications of NC in the song Don’t Get Around Much Anymore.

Many songs of the 60s-80s don’t have an ending, but rather fade away (gradual reduction of the volume to zero). Here’s how to implement the effect in a sequence. Again, the example is Love Me Do with a two measure fading repetition:

Shift 65
1 0 Mark FadeOut Ending
1 0 C
2 0 G 7
2 3 C
Shift 67
1 0 FadeOut 15.0
1 0 C
2 0 G 7
2 3 C
Shift 69
1 0 C
2 0 G 7
2 3 C
Shift 71
1 0 C
2 0 G 7
2 3 C
Shift 73
1 0 C
2 0 G 7
2 3 C
Shift 75
1 0 C
2 0 G 7
2 3 C

The song runs 65 measures before the fade. The Shift command resets the measure number so that the repetition can be constructed in an editor with copy and paste operations. The Mark command is a comment line that documents what is happening. The first repetition is played at full volume, then a 15.0 second fade is triggered. You need to paste enough copies of the section to cover the fade time span. There is no need to worry about adding too many copies. AMac terminates the accompaniment (disregarding addition content) when the volume reaches zero. Here’s the Love Me Do example, which includes several NC sections and a fade at the end.

 

Turbocharging Yamaha-format styles

Styles are great additions to Yamaha keyboards. They make it possible for players with moderate ability to create good sounding songs. On the other hand, standard style files have several limitations. Like MIDI files, they are constrained to support all hardware designed over a span of 40 years. The challenge for improving styles is to make changes in a manner that preserves back-compatibility.

There are 10s of thousands of Yamaha-format style files available on the Internet. Unfortunately, they are far from user friendly. The standard file has an arcane binary format that is difficult to decipher, even for a computer programmer. Many styles are sandbagged so that they perform well only on a specific keyboard. The major drawback is that it is difficult, if not impossible, for a musician to control the content.

Over several years, we have worked to make style files more accessible. One effort is the development of PureStyle, an organized binary style format that conforms to MIDI standards and is compatible with all Yamaha keyboards. The second effort is the development of a program to allow non-technical users to modify the content of style files. The program started with the unimposing name of Style Voice Optimizer. Then, it’s main function was to enable user control over the instrument voices of style MIDI channels. In this way, users could change the sound of styles and optimize voices for their keyboards. Since then, we have added many features.

  • Set baseline tempo. time-signature and volume level.
  • Shift the channel of extra percussion information for compatibility with computer output via soundfonts.
  • Clear tempo changes for compatibility with performance software.
  • Bulk convert styles to PureStyle with user selected options.
  • Export style sections to individual MIDI files for modifications with MIDI editors.
  • Import MIDI file data to create styles.
Figure 1. Style Master setup.

With expanding features, the program name changed to Style Optimizer and finally to Style Master. This article describes a new program feature that removes a major limitation of style files.

Standard styles are cast in concrete. You can choose from a maximum of four main loops, three introductions, three endings and six fill measures, all similar in content. On most instruments, it is impossible to switch styles in the middle of a piece without breaking the flow of the music. In this case, what do you do if a style has nice Main loops but a lame Intro or Ending? Or how can you play a medley? The latest version of Style Master offers a solution. With it, you can assemble a composite style with sections you add from different styles. In other words, the style could include Main sections for four different styles and three independent intro/ending choices.

To be realistic, the creation of a composite style takes some planning and effort. I’ll go through the procedure so you can decide if it’s a feature you can use. I’ll assume you have a basic knowledge of the style format. For the example, I picked two source styles that differed widely in style and instrumentation — one designated KarachiDance (KarachiDance_120_4-4_ps.sty) and the other BachPrelude (BachPrelude_80_4-4_ps.sty).

Step 1. Load the files in Style Master and save them with the options shown in Fig. 1. The files should be saved in a working directory — we’ll call it c:\temp. The program generates a file in PureStyle format. The options have the following effects:

  • General MIDI compliant Only GM commands are included, no XG specifications. This creates a file that may be distributed and then optimized for individual keyboards.
  • Tempo clear. Remove any tempo change messages that may occur in the sections (e.g., ritardandos in Endings). In this case, the sections will play correctly if the baseline tempo of the style is changed.
  • Keyboard compatible. Add dummy information so that the style will work when loaded on a Yamaha keyboard. This directive is not necessary for use in software like the Accompaniment Machine.
  • Percussion channel compress. This directive moves any percussion events from Channel 08h to Channel 09h. This is necessary if you are playing the style on a computer using a soundfont.
  • Include section voices. Check this if you want different instrumentation for different sections. Otherwise, the standard instrument selection is recorded in the SInt section of the file and is applied to all sections. In the example, I’ll illustrate how the sound of style sections can differ markedly.

Step 2. Make two sub-directories in the working directory to hold the MIDI file sets created from the style sections, c:\temp\Karachi and c:\temp\bach. Create a third directory for the combined style, c:\temp\backinkarachi.

File structure for making a combination style

Figure 2. File structure for making a combination style.

Step 3. Reload each source style in Style Master, click Export and choose the appropriate sub-directory. Figure 2 shows the contents of the directories. The program creates one standard MIDI file per section of the source style. (Note: it’s coincidental that the two styles in Fig. 2 have the same set of sections.)

Step 4. Preview the source styles in Style Master to determine which ones you want to keep. Copy these to the combined directory. You may need to change some names to avoid overlap. A map like this is helpful:

BachPrelude_80_4-4_ps.sty
MainA --> MainA
MainB --> MainB
EndingA --> EndingA
IntroA --> IntroA
FillAA --> FillAA
FillCC --> FillAB

KarachiDance_120_4-4_ps.sty
MainA --> MainC
MainC --> MainD
IntroB --> IntroB
IntroC --> IntroC
EndingB --> EndingB
EndingC --> EndingC
FIllAA --> FillBB
FillBB --> FillBA
FillBA --> FillCC
FillDD --> FillDD

Step 5. Click the Import button in Style Master and point to the directory c:\temp\backinkarachi. The program immediately loads the represented sections and builds a style. Many background operations occur to ensure that the different sections play at a consistent tempo.

Step 6. At this point, you can try out the style sections and save the combined PureStyle file. The combination technique has been tested on the Accompaniment Machine, and the resulting file should be compatible with any Yamaha keyboard. If you would like to make a test on your hardware, this zip file contains the original and combined styles: Download style demo.

There are some limitations. Changes of instrumentation made in Style Master affect only the standard voices messages recorded in the SInt section. In creating a combined style, the standard voices are determined when importing the MainA section. Changes made in Style Master do not affect voice messages that occur in individual sections. Therefore, you should make any desired changes to the source style channel voices before exporting them. Finally, the current versions of Style Master and the Accompaniment Machine require that the source styles have the same time signature. Future versions will address this issue.

 

Creating and editing keyboard styles

Styles are the automatic accompaniments on digital keyboards from Yamaha, Roland, Korg and other manufacturers. They can create the sound of a backup group, turning a simple melody into a performance piece. Style files in Yamaha format are the most common. Thousands of styles in every genre are available for download on the Internet.

The content of standard Yamaha-format style files is poorly documented, cumbersome and arcane. One consequence is that the same styles may appear in individual versions for specific keyboards, leading to considerable redundancy. Another problem is that styles for high-end keyboards like Tyros may be sandbagged with hidden information giving a poor sound on ordinary keyboards.

We have taken two steps to make style files an open, standardized medium of information exchange. The first is the creation of the PureStyle format described in this reference. With the elimination of unnecessary information, PureStyle files contain only MIDI messages that follow simple organization rules. The styles are compatible with all Yamaha keyboards and performance software like the Accompaniment Machine. Extensive libraries of PureStyle files are available on our site.

The second step is the addition of new features to our Style Master program. With the software, you can now edit or create Yamaha-format styles using any MIDI editor or digital workstation. The problem we addressed is that most MIDI editors cannot directly edit styles. They make changes to the structure of the file and do not preserve the correct order of marker messages used to identify style sections. For example, many editors divide MIDI channels (instruments) into individual tracks when loading and save files in Type 1 format. Style Master uses the following solution:

  • On export, the program saves each section (e.g., MainA, EndingB,…) of the currently-loaded style as an individual standard MIDI file in a working directory.
  • On import, Style Customizer searches the working directory for section files, converts them to Type 0 format and then loads them in correct order, adding marker messages as needed. The resulting data may then be saved in PureStyle format using any of the filters available in Style Master .

The conversion of a style into an organized set of MIDI files has additional applications. For example, style sections may be imported into sequencers. Section 6 of the Style Master instruction manual gives a complete description of the capability.

 

New features in MIDI Doctor

MIDI Doctor (Fig. 1) is a versatile tool to fix or to customize MIDI files. With it, you can adjust the volume and instrumentation of MIDI channels, transpose the key, modify the tempo, remove channels and perform several other operations. Recently, we’ve added powerful new capabilities. One improvement is a display of the exact temporal position in the file during playback (lower arrow in Fig. 1). This is a useful feature when excerpting sections (discussed below). We’ve also added the feature to MIDI Microscope, MiniMIDI player and MIDI File Organizer.

MIDI Doctor, main window

Figure 1. MIDI Doctor, main window.

A new Global settings button brings up the dialog shown in Fig. 2. There are three operations: 1) adjust the dynamic range, 2) adjust the play duration and 3) make an excerpt. The operations are global in the sense that they change the entire file.

MIDI Doctor, global settings dialog

Figure 2. MIDI Doctor, global settings dialog.

Dynamic range

There are many differences between western art music (i.e., classical) and popular music. One of them is that classical music gets soft and loud while popular music proceeds at a fairly uniform volume level. I was sharply reminded of this when I hoped to relax in the evening by listening to a MIDI file of Rachmaninoff’s Rhapsody on a Theme by Paganini. It was a great file except that the arranger had gone overboard with the dynamics. It ranged from inaudible to painful, and I was constantly clicking the volume control on my headphones.

In response, I added a method to compress or to expand the dynamic range of MIDI files. When the Global settings dialog opens, MIDI Doctor computes the minimum, maximum and average velocity values of all NoteOn messages with non-zero velocity. You can set new values in the boxes. The choices allow you to change the average volume or to compress/expand the low and high volume ranges individually. Click the Apply button to change all velocity values in the file. Be sure to save the modified file with a different name. There are many possible applications of this tool, including converting classical music to easy listening. Here are examples of effect that you can download: Original file with high dynamic range. Modified file with compressed dynamic range.

Play duration

One of the advantages of working with MIDI is that you can change the tempo of a piece without affecting the pitch. With this tool, you can set an exact duration for the file playback. One application — suppose you’re creating a video and need background music that ends exactly at the last fade. When the dialog opens, MIDI Doctor shows the current file duration in minutes and seconds. Simply fill in the desired values, click the Apply button and save the modified file.

Make excerpt

Use this tool is you want to make a demo or to pull out a favorite section. The end result is a modified file that only plays the specified interval. When the dialog opens, the Start and End values are set to the beginning and end of the file. Narrow the range to make an excerpt by changing the times. You can use the time display in the main window to determine values.

You may notice that when you save the excerpted file, it has the same length as the original file. This is because it’s tricky to cut out a piece of a MIDI file. The state of the synthesizer at any time depends on the sequence of all preceding MIDI messages. If you simply cut out a set of messages, you may wind up with a hanging note that plays to infinity. The solution I used in MIDI Doctor was to include all messages in the file, but to adjust those before the excerpt so they play at t = 0 with the velocity of NoteOn messages set to zero. The message time within the excerpt is adjusted to (t – TStart). Messages after the excerpt are all sent at (TEnd – TStart) and NoteOn message have zero velocity.

 

Running AMac on a Windows tablet

In the past, there were limited choices of tablet computers running Microsoft Windows, generally in the price range $700 and up. Now, you can get models for less than $100. A user recently asked whether the Accompaniment Machine will run on a small Windows tablet, so I decided to check it out.

As of April, 2018, there are two low-price choices on Amazon: the Nuvison 8″ at about $70 and the RCA Cambio 10″ at about $100. Besides the larger screen, the RCA device includes a detachable keyboard, a built-in stand and a real USB port to supplement the standard micro port. This was a major decision point for me because of the need to attach a keyboard. True, you can buy an OTG USB hub for the smaller tablet, but it’s not possible to charge the tablet and use the USB functions at the same time.

I purchased the RCA tablet and also a utility kit that included a USB hub, wireless mouse and HDMI cable. Figure 1 shows the RCA tablet (with keyboard detached) running AMac in Windows 10. The hub attached on the left accommodates the mouse adapter, the keyboard connection and a USB stick with MIDI files. Installation of the Yamaha keyboard driver was no more difficult than on any Windows computer.

Accompaniment Machine on a tablet computer

Figure 1. The Accompaniment Machine running on an RCA Cambio tablet driving a Yamaha keyboard.

The Accompaniment Machine is ideally suited to this type of computing environment. It was designed to make efficient use of CPU resources as well as screen area. We have run the program successfully on an old Windows NT HP Netbook, so there are no problems on this relatively powerful machine. The program display fits nicely on the tablet (1280 x 800 resolution). From the beginning, AMac featured control by screen buttons rather than menus. This means that you can invoke all performance functions by single touches without the need to navigate menus or raise the popup keyboard. Alternatively, the program can be controlled with the wireless mouse. The tablet runs all of our software, including Pancho.

The RCA tablet provides extraordinary functionality for the price. It’s a nice choice for a portable performance vehicle, and it wouldn’t be a financial tragedy if it got rained on or lost. In addition to music applications, I plan to use it as a business computer for travel. It can run all my standard applications (e.g., OpenOffice). With the addition of a $20 micro SD card, you can raise the built-in storage to 96 GB.

 

New features in MiniMIDI Player

MiniMIDI Player (Figure 1), a component of the MIDI Utility Pack, is a compact program to play MIDI files. It provides a quick method to preview files and to determine their properties. We recently added some features to enhance its utility.

Screenshot of MiniMIDI Player

Figure 1. Screenshot of MiniMIDI Player

MiniMIDI Player can be used as a standard interactive program in a window. It can also be launched from the command line with the target file as the pass parameter. If MiniMIDI Player is defined as the Windows default program for the extension MID, it will launch when you double-click a MIDI file in Windows Explorer or other file manager. It also responds if you click on a link to MIDI file on the Internet in a browser. By default, in command-line mode the program closes after playing the file. If you want to repeat the play or load another file, click either the Play/Stop or Pause/Resume button. In this case, MiniMIDI Player exits command-line mode and stays active until you click the Exit button.

In addition to the MIDI file type and number of tracks, the program now displays the exact play duration in minutes and seconds including effects of tempo changes. Finally, we reduced the program footprint so it occupies minimal space on your desktop.