SuperCollider UGen development in Rust

Earlier this year, I developed a few tools in Rust to allow me to process audio files and implement some algorithms that were just too slow in sclang or Ruby, my two favorite dynamic programming languages. Aside from the clear ulterior motive (procrastination through learning a new programming language), I had the dream of someday being able to easily translate these non-realtime programs into a realtime performance using SuperCollider. There were a few pitfalls along the way, mostly due to learning Rust more or less as I went, but also because of some very specific demands that realtime audio synthesis places upon a programmer.

I'll start by describing the general structure of the project, then move on to general constraints about realtime audio programming in Rust, and finally conclude with how I implemented the SuperCollider UGen itself.

The Structure

The project (at least the part that SuperCollider interfaced with) was fairly simple. It consisted of a Rust library called vox_box, which exposed a number of public, external functions, and a SuperCollider UGen VoxRes. The SC UGen allocated its own memory, and then called the Rust library, with a pointer to the proper buffer, which performed the actual number-crunching. As a result, all of the implementation details exist within the Rust library itself, and furthermore there is no reason that I couldn't use the exact same functions in both the SC usage of VoxBox and the native Rust command line programs that I had written before.

Currently, on my own computer, the Rust library exists as a dylib and is linked when SuperCollider runs. This allows a nice development cycle of recompiling the dynamic library upon small changes to the internal Rust source (i.e., not to the public API), without needing to recompile the actual UGen.

To clarify, the final structure is as follows:

  1. A Rust implementation of a number of methods and algorithms for vocal processing.
  2. A public C API to the Rust library that converts unsafe pointers to safe structs.
  3. A SuperCollider C++ UGen file that uses this C API for processing on buffers.

The Demands

Demand 1: No heap allocations in the loop

Or, to be more clear: no heap allocations within Rust at all. Similar to writing any SuperCollider UGen, the sample to find the next block should always remain free of allocations. However, this means that any Vecs, or anything else allocated on the heap with a size unknown at compile time, must be kept totally out of Rust code. As a result, if any workspace is needed, it must be received from the (unsafe!) C API. This leads to code like this:

#[no_mangle]
/// work must be 3*size+2 for complex floats (meaning 6*size+4 of the buffer)
pub unsafe extern fn vox_box_resonances_mut_f32<'a>(buffer: *const f32, size: size_t, sample_rate: f32, count: &mut c_int, work: *mut Complex<f32>, out: *mut f32) {
    // Input buffer
    let buf: &[f32] = std::slice::from_raw_parts(buffer, size);
    let mut res: &mut [f32] = std::slice::from_raw_parts_mut(out, size);
    // Mutable complex slice
    let mut complex: &mut [Complex<f32>] = std::slice::from_raw_parts_mut(work, size); 
    let mut complex_work: &'a mut [Complex<f32>] = 
      std::slice::from_raw_parts_mut(work.offset(size as isize), size*4 + 2);
    for i in 0..size {
        complex[i] = (&buf[i]).to_complex();
    }
    match complex.find_roots_mut(complex_work) {
        Ok(_) => { },
        Err(x) => { println!("Problem: {:?}", x) }
    };
    ...

and the function that uses it:

let (mut z_roots, mut work) = work.split_at_mut(2*self.len());
let mut z_root_index = 0;
for i in 0..coeff_low {
    z_roots[i] = Complex::<T>::zero();
    z_root_index = z_root_index + 1;
}

let (mut rem, mut work) = work.split_at_mut(coeff_high - coeff_low + 1);
let (mut coeffs, mut work) = work.split_at_mut(coeff_high - coeff_low + 1);
for co in coeff_low..(coeff_high+1) {
    coeffs[co] = self[co];
}

As you can see, in the implementation, the work slice is split into two chunks multiple times, and each time something is shaved off to be used for workspace.

Demand 2: Only operate on slices

This was a major headache for me. I had originally written my code to be generic over Vec types, but in reality I should have chosen slices when possible, and split into separate traits those items that required Vecs as input for some reason. There's a key point here:

A corellary to Demand #1 is that we cannot free anything we have not allocated; therefore, all input must be treated as a non-owned slice. This makes sense, because SuperCollider lets the memory persist until the UGen's destructor is called.

Demand 3: No allocations, seriously

These functions will be called a whole bunch of times, and memory leaks or hidden allocations are not okay. I wrote a very short program for the sole purpose of running Valgrind to test things. The most important aspect of this was the HEAP SUMMARY at the top. I would un-comment and re-comment individual function call lines in the test code, check the heap usage, and if anything budged in the allocations that was an immediate red flag. Here's what Valgrind looks like—not that scary, I swear:

==35025== HEAP SUMMARY:
==35025==     in use at exit: 28,555 bytes in 196 blocks
==35025==   total heap usage: 264 allocs, 68 frees, 39,083 bytes allocated
==35025==
==35025== LEAK SUMMARY:
==35025==    definitely lost: 6,168 bytes in 3 blocks
==35025==    indirectly lost: 2,288 bytes in 6 blocks
==35025==      possibly lost: 4,904 bytes in 46 blocks
==35025==    still reachable: 2,344 bytes in 12 blocks
==35025==         suppressed: 12,851 bytes in 129 blocks
==35025== Rerun with --leak-check=full to see details of leaked memory
==35025==
==35025== For counts of detected and suppressed errors, rerun with: -v
==35025== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I heard somewhere that SC UGen developers should avoid calling external libraries (or STL classes) because they may include hidden allocations using malloc, while SC uses a realtime allocation pool. However, divorcing your algorithm's implementation from the actual SuperCollider UGen allows for writing test programs that verify the number of allocations happening. If the functions you want to call don't allocate any memory, then I don't see why they shouldn't be used. (I will totally, 100%, include corrections on this point if someone can give me a reason.)

Implementation in SuperCollider

Aside from some sort of dumb stuff about copying a ring buffer to a sequential buffer, it's pretty straight-ahead. Basically, the SC UGen just allocates some data and workspace in its constructor, calls a bunch of Rust functions with its next function, and then frees that memory with its destructor. Here's the good stuff:

void VoxRes_next_k(VoxRes *unit, int inNumSamples)
{
  GET_BUF_SHARED;
  for (int i = 0; i < bufFrames; i++) {
    unit->local_buf[i] = bufData[(i + unit->offset) & (bufFrames - 1)];
  }
  vox_box_resample_mut_f32(unit->local_buf, 
    (size_t)bufFrames, unit->resampledFrames, unit->resampled);
  vox_box_autocorrelate_mut_f32(unit->resampled, 
    (size_t)unit->resampledFrames, unit->lpc_size + 2, unit->autocor_buffer);
  vox_box_normalize_f32(unit->autocor_buffer, unit->lpc_size + 2);
  vox_box_lpc_mut_f32(unit->autocor_buffer, 
    unit->lpc_size + 2, unit->lpc_size, unit->lpc, unit->lpc_work); 
  int resonance_count = 0;
  vox_box_resonances_mut_f32(unit->lpc, 
    unit->lpc_size + 1, FULLRATE * unit->resampledFrames / (float)bufFrames, 
    &resonance_count, unit->res_work, unit->res);
  for (int i = 0; (i < resonance_count) && (i < unit->max_resonances); i++) {
    *OUT(i) = unit->res[i];
  }
  unit->offset += (int)BUFRATE;
}

And, even simpler, the SC class file:

VoxRes : MultiOutUGen {
  *kr { 
    arg in, lpcCount = 16;
    ^this.multiNew('control', in, lpcCount)
  }

  init {
    arg ... theInputs;
    inputs = theInputs;
    ^this.initOutputs(4, rate);
  }
}

(For everything, visit GitHub.)

Hope that helped

Please get in touch if you have any corrections, comments, questions, etc! I wrote this library while I was both learning to code Rust and learning what goes into making a SuperCollider UGen, so there are bound to be better ways of doing things.