Finding the cause of a memory leak in Ruby

Question

I've discovered a memory leak in my Rails code - that is to say, I've found what code leaks but not why it leaks. I've reduced it down to a test case that doesn't require Rails: ``` require 'csspool' require 'ruby-mass' def report puts 'Memory ' + `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`.strip.split.map(&:to_i)[1].to_s + 'KB' Mass.print end report # note I do not store the return value here CSSPool::CSS::Document.parse(File.new('/home/jason/big.css')) ObjectSpace.garbage_collect sleep 1 report ``` ruby-mass supposedly lets me see all the objects in memory. CSSPool is a CSS parser based on racc. /home/jason/big.css is a 1.5MB CSS file. This outputs: ``` Memory 9264KB ================================================== Objects within [] namespace ================================================== String: 7261 RubyVM::InstructionSequence: 1151 Array: 562 Class: 313 Regexp: 181 Proc: 111 Encoding: 99 Gem::StubSpecification: 66 Gem::StubSpecification::StubLine: 60 Gem::Version: 60…

Accepted Answer

It looks like you are entering The Lost World here. I don’t think the problem is with c-bindings in `racc` either.

Ruby memory management is both elegant and cumbersome. It stores objects (named `RVALUE`s) in so-called heaps of size of approx 16KB. On a low level, `RVALUE` is a c-struct, containing a `union` of different standard ruby object representations.

So, heaps store `RVALUE` objects, which size is not more than 40 bytes. For such objects as `String`, `Array`, `Hash` etc. this means that small objects can fit in the heap, but as soon as they reach a threshold, an extra memory outside of the Ruby heaps will be allocated.

This extra memory is flexible; is will be freed as soon as an object became GC’ed. That’s why your testcase with `big_string` shows the memory up-down behaviour:

```
def report
  puts 'Memory ' + `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`
          .strip.split.map(&:to_i)[1].to_s + 'KB'
end
report
big_var = " " * 10000000
report
big_var = nil 
report
ObjectSpace.garbage_collect
sleep 1
report
# ⇒ Memory 11788KB
# ⇒ Memory 65188KB
# ⇒ Memory 65188KB
# ⇒ Memory 11788KB
```

But the heaps (see `GC[:heap_length]`) themselves are not released back to OS, once acquired. Look, I’ll make a humdrum change to your testcase:

```
- big_var = " " * 10000000
+ big_var = 1_000_000.times.map(&:to_s)
```

And, voilá:

```
# ⇒ Memory 11788KB
# ⇒ Memory 65188KB
# ⇒ Memory 65188KB
# ⇒ Memory 57448KB
```

The memory is not released back to OS anymore, because each element of the array I introduced suits the `RVALUE` size and is stored in the ruby heap.

If you’ll examine the output of `GC.stat` after the GC was run, you’ll find that `GC[:heap_used]` value is decreased as expected. Ruby now has a lot of empty heaps, ready.

The summing up: I don’t think, the `c` code leaks. I think the problem is within base64 representation of huge image in your `css`. I have no clue, what’s happening inside parser, but it looks like the huge string forces the ruby heap count to increase.

Hope it helps.

Finding the cause of a memory leak in Ruby

Solution

Problem