Rails - Invalid Authenticity Token After Deploy
deployment, ruby, ruby-on-rails
Solution
ANSWER: After extensive work by EngineYard (they're awesome!) they were able to diagnose the issue. The root cause of this issue is a bug with mongrel clusters. Mongrel doesn't seem to see the first post request after being started. EngineYard did extensive work to diagnose this:
There doesn't appear to be anything in your code causing the issue and I have found people outside of our environment that have experienced the bug as well (http://www.thought-scope.com/2009/07/mongrelcluster-rails-23x-bad-post.html). I suppose a lot of people don't see it because the first request to a site generally isn't a post or they chalk it up to flukes.
[There is a potential workaround using CURL.] The curl work around would do a simple GET request to each of your mongrels on the server to prime them so to speak. You could do this with capistrano, but that won't work if you deploy via the dashboard. You can find a short section on deploy hooks we have built into the infrastructure here: https://cloud-support.engineyard.com/faqs/overview/getting-started-with-engine-yard-cloud
Adding a simple run curl http://localhost:500x > /dev/null should work (where x is the port you have 5000-50005 on your current setup).
We have addressed the issue by switching our stack from Mongrel to Passenger, but apparently, a fix for Mongrel is in the works. Hopefully, this helps someone who sees this same strange issue.
Problem
We're using EngineYard Cloud to deploy our Ruby on Rails application. We are running Rails v2.3.3. EngineYard Cloud deploys to AWS instances in a manner similar to Capistrano. After each deploy, we're running into Invalid Authenticity Token errors. Specifically, any user that has previously visited our application and then visits after the deploy and then tries to submit a form gets an invalid authenticity token error. This error persists until they reset their cookies for the site. After they reset their cookies, the site works as expected with no errors. We are using ActiveRecord's session store and sessions are being saved to the database. This is the error we are seeing: ActionController::InvalidAuthenticityToken /usr/lib/ruby/gems/1.8/gems/actionpack-2.3.3/lib/action_controller/request_forgery_protection.rb:79:in `verify_authenticity_token' The session object is nil after the deploy, however, the session data still persists in the database and the session ID cookie still exists: Session: - session id: nil - data: nil We haven't been able to explain this one. Any thoughts on what could be the root cause? Thanks for any suggestions! EDIT: Just to update on this, we've been able to isolate an example of the error. 1) User loads form 2) Code is updated on server 3) User submits form ** Invalid Authenticity Token error occurs It seems that when the environment changes, Rails is unable to handle this with the authenticity token. We've tried several steps to resolve: - Resetting the session - Deleting the session cookie (both in JavaScript and Rails) - Wiping the session table in the database after deploying code Nothing works. The only thing that works is having the user clear their cookies client-side. (We've been Googling (even tried Binging!) for answers, but no dice. This seems to be a similar related issue: http://railsforum.com/viewtopic.php?id=21479) Also: initially we thought this was isolated to our deployment to EngineYard, but we've also been able to reproduce it on our development server that we deploy to via Capistrano. Any thoughts would be gratefully accepted. Thanks!