Ruby - Redis based mutex with expiration implementation
locking, multithreading, mutex, redis, ruby
Solution
If you're using Redis 2.6+, you can do this much more simply with the Lua scripting engine. The Redis documentation says:
A Redis script is transactional by definition, so everything you can do with a Redis transaction, you can also do with a script, and usually the script will be both simpler and faster.
Implementing it is trivial:
LUA_ACQUIRE = "return redis.call('setnx', KEYS[1], 1) == 1 and redis.call('expire', KEYS[1], KEYS[2]) and 1 or 0"
def lock(key, timeout = 3600)
if redis.eval(LUA_ACQUIRE, key, timeout) == 1
begin
yield
ensure
redis.del key
end
end
end
Usage:
lock("somejob") { do_exclusive_job }
Problem
I'm trying to implement a memory based, multi process shared mutex, which supports timeout, using Redis. I need the mutex to be non-blocking, meaning that I just need to be able to know if I was able to fetch the mutex or not, and if not - simply continue with execution of fallback code. something along these lines: ``` if lock('my_lock_key', timeout: 1.minute) # Do some job else # exit end ``` An un-expiring mutex could be implemented using redis's `setnx mutex 1`: ``` if redis.setnx('#{mutex}', '1') # Do some job redis.delete('#{mutex}') else # exit end ``` But what if I need a mutex with a timeout mechanism (In order to avoid a situation where the ruby code fails before the `redis.delete` command, resulting the mutex being locked forever, for example, but not for this reason only). Doing something like this obviously doesn't work: ``` redis.multi do redis.setnx('#{mutex}', '1') redis.expire('#{mutex}', key_timeout) end ``` since I'm re-setting an expiration to the mutex EVEN if I wasn't able to set the mutex (`setnx` returns 0). Naturally, I would've expected to have something like `setnxex` which atomically sets a key's value with an expiration time, but only if the key does not exist already. Unfortunately, Redis does not support this as far as I know. I did however, find `renamenx key otherkey`, which lets you rename a key to some other key, only if the other key does not already exist. I came up with something like this (for demonstration purposes, I wrote it down monolithically, and didn't break it down to methods): ``` result = redis.multi do dummy_key = "mutex:dummy:#{Time.now.to_f}#{key}" redis.setex dummy_key, key_timeout, 0 redis.renamenx dummy_key, key end if result.length > 1 && result.second == 1 # do some job redis.delete key else # exit end ``` Here, i'm setting an expiration for a dummy key, and try to rename it to the real key (in one transaction). If the `renamenx` operation fails, then we weren't able to obtain the mutex, but no harm done: the dummy key will expire (it can be optionally deleted immediately by adding one line of code) and the real key's expiration time will remain intact. If the `renamenx` operation succeeded, then we were able to obtain the mutex, and the mutex will get the desired expiration time. Can anyone see any flaw with the above solution? Is there a more standard solution for this problem? I would really hate using an external gem in order to solve this problem...