Diagnosing and Fixing Errno::EMFILE (Too Many Open Files) in Ruby

The Problem: Resource Exhaustion

When a Ruby application hits the operating system's limit for open file descriptors, it throws Errno::EMFILE: Too many open files. This is rarely a problem of having too many legitimate files open; it is almost always a file descriptor leak, where the application requests a resource (a file, a socket, or a pipe) but fails to release it back to the OS.

The immediate takeaway: Increasing the system limit (ulimit) is a temporary bandage. If you have a leak, the application will eventually crash regardless of how high the limit is set. You must identify the leak source and ensure resources are closed in all execution paths.

Diagnostic Matrix

Use this table to match your application's behavior to the likely cause.

Observation	Likely Cause	Primary Suspect
Linear increase in open files over time	Resource Leak	Unclosed File.open or Net::HTTP connections
Sudden spike during high traffic	Capacity Limit	Too many concurrent TCP connections/sockets
Crash immediately upon startup	Configuration Error	OS soft limit set too low for the app's baseline needs

Step‑by‑Step Investigation

1. Verify the Current Limit

Check the current soft limit for the shell session running your Ruby process. Run this command in the terminal where the app is launched:

ulimit -n

If the result is 1024 (a common default), and your app handles high concurrency, you may be hitting a legitimate limit. However, if you see the error while processing a small number of requests, a leak is more likely.

2. Identify Leaking Descriptors

To see exactly what the Ruby process is holding open, use lsof (List Open Files). You will need the Process ID (PID) of your Ruby application.

Run this command as a user with sufficient permissions (usually the app owner or root):

lsof -p [PID]

What to look for: Look for hundreds of entries pointing to the same file path or many entries labeled TCP or can't identify protocol. If you see a repeating pattern of open files that never disappear, you have found your leak.

3. Audit the Code for Common Leak Patterns

Review your codebase for these two common failure patterns:

Pattern A: Manual File Opening

Avoid this pattern, as an exception raised between opening and closing will leave the file descriptor open:

# RISK: If an error occurs here, the file stays open
f = File.open("data.txt", "r")
process_data(f)
f.close

Pattern B: Unconsumed HTTP Responses

Using Net::HTTP without properly closing the connection or reading the entire body can leave sockets in a CLOSE_WAIT or open state.

The Fixes

Implementing Block‑Based Resource Management

The most effective way to prevent Errno::EMFILE is to use Ruby's block syntax. The block ensures the file is closed automatically when the block terminates, even if an exception is raised.

# SAFE: File is automatically closed at the end of the block
File.open("data.txt", "r") do |f|
  process_data(f)
end

Using Ensure for Non‑Block Resources

If you cannot use a block (e.g., the resource must stay open across different method calls), wrap the closure in an ensure block to guarantee execution.

begin
  @socket = TCPSocket.new('example.com', 80)
  # perform operations
rescue StandardError => e
  log_error(e)
ensure
  @socket&.close
end

Verification and Limitations

To verify the fix, monitor the process descriptors while running a load test. The number of open files should plateau rather than climb indefinitely:

# Run this periodically to monitor the count
lsof -p [PID] | wc -l

Limitations

• Shared Sockets: Be cautious when closing sockets that are shared across threads. Closing a socket that another thread is still using will trigger Errno::EPIPE (Broken pipe).
• External Libraries: If lsof shows leaks in C‑extensions or gems, you may need to update the library or manually trigger garbage collection (GC.start), though the latter is a last resort.

Rollback

If you modified the system ulimit in /etc/security/limits.conf to mitigate the issue, revert the changes to the original values to ensure the OS remains protected from runaway processes.