Ruby: Cannot allocate memory

ruby, ruby-on-rails, vagrant

Solution

When Ruby calls `fork` the OS will make a copy of the entire parent processes address space, even if fork is only being called to `exec` another small process like `ls`. Momentarily, your system needs to be able to allocate a chunk of memory at least the size of the Ruby parent process before collapsing it down to what the child process actually needs.

So rails is generally quite memory hungry. Then if something uses `fork`, you need twice as much memory.

TL;DR Use posix-spawn instead of fork if you are in control of the code. Otherwise give your VM 1024MB or a bit of extra swap space to take up the slack for the `fork` call

Example Ruby Memory Usage with`fork`

Take a random VM, this one has swap space disabled:

$ free -m
             total       used       free     shared    buffers     cached
Mem:          1009        571        438          0          1         35
-/+ buffers/cache:        534        475
Swap:            0          0          0

Look at the `Mem:` row and `free` column. This is around about your size limit for a new process, in my case `438`MiB

My `buffers/cached` have already been flushed for this test so that my `free` memory is at it's limit. You may need to take the `buffers/cache` values into account if they are large. Linux has the ability to evict stale cache when memory is needed by a process.

Use up some memory

Create a ruby process with a string around the size of your free memory. There is some overhead for the `ruby` process so it's not going to exactly match `free`.

$ ruby -e 'mb = 380; a="z"*mb*2**20; puts "=)"'
=)

Then make the string slightly larger:

$ ruby -e 'mb = 385; a="z"*mb*2**20; puts "=)"'
-e:1:in `*': failed to allocate memory (NoMemoryError)
        from -e:1:in `<main>'

Add a `fork` to the ruby process, reducing `mb` until it runs.

$ ruby -e 'mb = 195; a="z"*mb*2**20; fork; puts "=)"'
=)

A slightly larger fork process will produce the `ENOMEM` error:

$ ruby -e 'mb = 200; a="z"*mb*2**20; fork; puts "=)"'
-e:1:in `fork': Cannot allocate memory - fork(2) (Errno::ENOMEM)
        from -e:1:in `<main>'

Running a command with backticks launches that process with a `fork` so has the same outcome:

$ ruby -e 'mb = 200; a="z"*mb*2**20; `ls`'
-e:1:in ``': Cannot allocate memory - ls (Errno::ENOMEM)
        from -e:1:in `<main>'

So there you go, you need about twice the parent processes memory available on the system to fork a new process. MRI Ruby relies heavily on `fork` for it's multi process model, this is due to the design of Ruby which uses a global interpreter lock (GIL) that only allows one thread to execute at a time per ruby process.

I believe Python has a lot less use of `fork` internally. When you do use `os.fork` in Python, the same occurs though:

python -c 'a="c"*420*2**20;'
python -c 'import os; a="c"*200*2**20; os.fork()'

Oracle have a detailed article on the problem and talk about using the alternative of `posix_spawn()`. The article is directed at Solaris but this is a general POSIX Unix issue so applies to Linux (if not most Unices).

There is also a Ruby implementation of `posix-spawn` which you could use if you are in control of the code. This module doesn't replace anything in Rails, so it won't help you here unless you replaced the calls to `fork` yourself.

Problem

I am in the process of development of a Ruby on Rails application. I am a newbie to Ruby/Rails. I use Ruby 2.2.0 and Rails 4.2. When I run a command like: ``` rails g migration SomeMigrationName ``` it fails with the ``` Cannot allocate memory - fork(2) (Errno::ENOMEM) ``` I use Macbook Pro mid 2014 with OS X 10.10 on board and Vagrant/Virtualbox to run a virtual machine (Ubuntu 14.04) for Rails development. Here is my Vagrant file: ``` Vagrant.configure(2) do |config| config.vm.box = "ubuntu/trusty64" config.vm.network "forwarded_port", guest: 3000, host: 3000 config.vm.synced_folder "dev", "/home/vagrant/dev" config.vm.synced_folder "opt", "/opt" config.vm.provider "virtualbox" do |vb| vb.memory = "512" end end ``` I've read that such an error occurs when RAM is out of limit, but I use same config (Vagrant file) for the another dev environment which runs several Python/Tornado apps, MongoDB and Redis and it all works fine. Do I need to increase vb.memory value or it's a Ruby bug?

Original source