SDL Audio Streaming: Format Conversion and Resampling with SDL_AudioStream

Real-Time Audio Conversion with SDL_AudioStream

The problem: Your application needs to play audio that doesn't match your device's native audio format—different sample rates, channel counts, or formats. SDL_AudioStream provides real-time conversion between these formats without requiring you to precompute large audio buffers.

Creating and Configuring the Stream

Start by defining both your source format (what you have) and destination format (what the device needs). Use SDL_CreateAudioStream to create a converter:

SDL_AudioSpec srcFormat = {0};
srcFormat.freq = 48000;
srcFormat.format = AUDIO_F32;
srcFormat.channels = 2;

SDL_AudioSpec dstFormat = {0};
dstFormat.freq = 44100;
dstFormat.format = AUDIO_S16;
dstFormat.channels = 1;

size_t srcSize = 4096; // Size of your input buffer
size_t dstSize = SDL_CalculateAudioBufferSize(&dstFormat, srcSize, 1);

SDL_AudioStream *stream = SDL_CreateAudioStream(&srcFormat, &dstFormat);
if (!stream) {
    printf("Failed to create stream: %s\n", SDL_GetError());
    return -1;
}

Processing Audio Through the Stream

In your audio callback, push converted data into the stream and pull the converted output:

void audio_callback(void *userdata, Uint8 *stream_out, int len) {
    Uint8 *converted = (Uint8 *)malloc(len);
    int converted_len = SDL_AudioStreamConvert(stream, converted, len);
    
    if (converted_len > 0) {
        memcpy(stream_out, converted, converted_len);
    } else {
        // Handle underrun - silence or previous buffer
        memset(stream_out, 0, len);
    }
    free(converted);
}

Critical Usage Patterns

1. Flush before destruction: Always call SDL_AudioStreamFlush before SDL_AudioStreamFree to ensure all buffered audio is processed.
2. Monitor buffer levels: Use SDL_AudioStreamAvailableInDownstream to check if you have enough converted data ready.
3. Thread safety: If your audio thread and main thread both access the stream, implement external synchronization.

Common Mistakes and Their Fixes

Memory Leaks from Unclosed Streams

Every SDL_CreateAudioStream must have a matching SDL_AudioStreamFree. Place cleanup in your audio shutdown routine:

// At application shutdown
if (stream) {
    SDL_AudioStreamFlush(stream);  // Process remaining data
    SDL_AudioStreamFree(stream);
}

Buffer Underruns During Playback

If your conversion can't keep up with the callback rate, you'll get silence. Monitor and adjust:

size_t available = SDL_AudioStreamAvailableInDownstream(stream);
if (available < needed_bytes) {
    // Need to feed more input or reduce processing
    printf("Buffer level low: %zu bytes\n", available);
}

Unsupported Format Combinations

Not all format conversions are supported. Check compatibility before creating the stream:

if (SDL_QueueAudio(stream, input_buffer, input_size) < 0) {
    printf("Format not supported: %s\n", SDL_GetError());
    // Fall back to software conversion or reject the stream
}

Practical Verification Steps

1. Test conversion accuracy with known sine wave samples at different frequencies
2. Monitor SDL_AudioStreamAvailableInDownstream during playback to verify buffer levels
3. Use SDL_GetAudioStreamInfo to confirm runtime format matches expectations
4. Profile latency with different buffer sizes to find optimal balance

Limitations and Considerations

• Latency: Larger stream buffers reduce CPU load but increase audio latency—critical for real-time applications
• Format support: SDL may not support all possible format combinations; check documentation for your version
• Thread synchronization: SDL_AudioStream is not thread-safe; you must protect access with mutexes or other synchronization primitives