Research In Real-Time Music Systems

The audio input/output system (AIOS) plays a central role in real-time music processing systems. It's primary responsibility is to deliver audio data between the audio devices and the client processing system. Although conceptually simple the AIOS is notoriously complex because it must satisfy two sometimes conflicting goals. First it must run at the highest possible frequency to service the constant stream of the smallest possible blocks of data to and from the audio devices. Second it must communicate with multiple asynchronous systems while never blocking itself.

Project Description:

CREL researcher, Kevin Larke, has developed cmas, an AIOS implementation designed to support experimentation with locally distributed real-time music processing systems. cmas has several interesting features.

Multiple Sub-Systems: As shown in the diagram cmas is formed from an array of identical, independent, sub-systems. Each sub-system is implemented with it's own thread, and is assigned an audio device, and digital signal processing (DSP) system interface.

DSP Interface: cmas provides an interface to the DSP systems but does not provide an implementation. This frees the application developer to select a DSP system which best serves the application. The system also provides a bidirectional, thread-safe, communication channel between the external processes (client,network,MIDI) and the DSP systems. This is important because the DSP systems execute in the AIOS sub-system thread while the external processes are considered to be on remote threads, processes, or machines.

Virtual Devices: The system supports both physical and virtual audio devices. Some examples of virtual audio devices are:

• Aggregate Devices: Multiple physical devices may be collected into a single aggregate device. Aggregate devices therefore allow multiple devices to appear to the DSP system as a single device.
• File Devices: File devices can be used to emulate input or output ports and provide or capture data from audio files.
• Non-real-time devices: Non-real-time devices allow the system to be run just as it would in real-time but with the ability to step virtual time in a very controlled way. This is often useful as a debugging aid.

Audio Controls: The system provides a set of audio controls for each audio input and output channel. The controls allow for setting and querying the channels gain, mute state, pass-through state, test tone state. Setting the pass-though allows input audio to be sent directly to the out rather than through the DSP system. Setting the test-tone allows a tone to be generated on an input or output channel.

Diagnostics: The system can automatically generate and send to the client application a stream of diagnostic information which is often useful to evaluate the current state of the system. The stream includes individual input and output audio channel volume information, and the count of messages received from the audio devices and external subsystems.

Programming Interface: Control of the system is implemented through a C library API using function calls or through serialized messages. The system is currently implemented for Apple OS-X and Linux/ALSA.

Future Work:

The system is currently operating as part of a complete experimental audio processing framework. Several improvements, modifications, and evaluations however remain to be done. The next step is to examine the efficiency/latency trade-off as the base operating frequency is increased. This will entail experimenting with the audio period length, and sub-system thread CPU affinity, priority while measuring CPU load. The data generated, and dynamics observed, from these experiments will inform an automatic tuning algorithm to optimize input/output latency.