Implementing Non-Blocking I/O with the Ruby Fiber Scheduler

The Problem: Thread Exhaustion in I/O-Bound Apps

Traditional Ruby concurrency relies on OS threads. Each thread carries a significant memory overhead for its stack and incurs a performance penalty during context switching managed by the operating system. When an application handles thousands of concurrent I/O requests—such as polling multiple APIs or managing many WebSocket connections—the system often hits a memory ceiling or spends more time switching threads than executing code.

The takeaway: For I/O-bound workloads, Ruby's Fiber Scheduler (introduced in 3.0) allows you to handle thousands of concurrent operations on a single OS thread by yielding control during blocking I/O calls, drastically reducing memory consumption.

Requirements for Fiber-Based Concurrency

To move from a thread-per-request model to a fiber-based model, the environment must meet these criteria:

• Ruby Version: Ruby 3.0 or higher.
• I/O Pattern: The workload must be primarily I/O-bound (waiting for network, disk, or timers).
• Scheduler Implementation: Ruby provides the interface for a scheduler, but not a default production implementation. You must use a gem like async or implement a custom Fiber::Scheduler.

The Smallest Suitable Design

The architecture consists of a single OS thread running an event loop. When a Fiber performs a blocking operation (e.g., IO#read or Kernel#sleep), the Ruby VM intercepts this call and hands control back to the Scheduler. The Scheduler registers the I/O event with the OS (using epoll or kqueue) and switches to another ready Fiber.

Implementation Example

Using the async gem is the standard way to implement this design. Run the following in your terminal to install the dependency:

gem install async

Below is a configuration for concurrent HTTP-like requests. Run this script with ruby script.rb:

require 'async'
require 'async/http/internet'

# The Async block sets the Fiber scheduler for the current thread
Async do
  internet = Async::HTTP::Internet.new
  
  urls = [
    'https://www.ruby-lang.org',
    'https://www.google.com',
    'https://www.github.com'
  ]

  urls.each do |url|
    # Each spawn creates a lightweight Fiber
    Async do
      response = internet.get(url)
      puts "Fetched #{url}: #{response.status}
    end
  end
end

Trust and Data Boundaries

Because Fibers in a single scheduler run on a single OS thread, they are bound by the Global VM Lock (GVL). This creates a specific data boundary: you do not need mutexes to protect shared state from simultaneous execution, as only one Fiber is ever executing Ruby code at a time.

However, you must maintain logical atomicity. If a Fiber yields control (via I/O) in the middle of updating a shared hash, another Fiber may read that hash in an inconsistent state. Use a Mutex if the sequence of operations across an I/O boundary must be atomic.

Operational Checks and Failure Modes

The "Blocking the Loop" Failure

The most critical failure mode is CPU starvation. Because Fibers are cooperative, they must yield control. If a Fiber enters a heavy computation loop, it blocks the entire OS thread, preventing the Scheduler from resuming other Fibers.

Diagnostic Check: To verify if a CPU-intensive task is starving your loop, implement a "heartbeat" Fiber:

Async do
  loop do
    puts "Heartbeat: Loop is alive"
    sleep 1
  end
end

If the heartbeat stops printing while your application is processing a request, you have a blocking CPU operation that must be moved to a separate process or a Ractor.

Memory Verification

To verify the efficiency gains, you can compare the memory footprint of 5,000 Threads versus 5,000 Fibers. In a standard Linux environment, 5,000 threads may consume several gigabytes of RAM due to stack allocation, whereas 5,000 Fibers typically consume a few hundred megabytes.

When to Change the Design

The Fiber Scheduler design is no longer suitable under the following conditions:

1. CPU-Bound Workloads: If the application spends more than 20-30% of its time on computation (e.g., JSON parsing of massive payloads, image processing), the GVL becomes the bottleneck. Transition to Ractors (Ruby's actor model) or a multi-process cluster.
2. C-Extensions: If you use gems with C-extensions that perform blocking I/O without releasing the GVL or using the Ruby Fiber API, the scheduler cannot intercept those calls, and the entire thread will freeze.

Rollback and State

Since implementing a Fiber scheduler typically involves wrapping the entry point of your application in an Async block, rolling back involves removing the wrapper and returning to standard synchronous calls or Thread.new. No persistent state is altered by the scheduler itself; it only affects the execution flow of the current process.