Elasticsearch out of sync when overwhelmed on HTTP at test suite

elasticsearch, rspec, ruby, ruby-on-rails

Solution

Your test is confused - it's testing assignment, pagination, and (implicitly) parameter passing. Break it out:

Parameters

let(:tire) { double('tire', :search => :sentinel) }

it 'passes the correct parameters to Companies.tire.search' do
  expected_params = ... # Some transformation, if any, of params
  Companies.stub(:tire).with(tire)
  get :index, params

  expect(tire).to have_received(:search).with(expected_params)
end

Assignment

We are only concerned that the code is taking one value and assigning it to something else, the value is irrelevant.

it 'assigns the search results to companies' do
  Companies.stub(:tire).with(tire)
  get :index, params

  expect(assigns[:companies]).to eq :sentinel
end

Pagination

This is the tricky bit. You don't own the ES API, so you shouldn't stub it, but you also can't use a live instance of ES because you can't trust it to be reliable in all testing scenarios, it's just an HTTP API after all (this is the fundamental issue you're having). Gary Bernhardt tackled this issue in one of his excellent screencasts - you simply have to fake out the HTTP calls. Using VCR:

VCR.use_cassette :tire_companies_search do
  get :index, params
  search_result_length = assigns[:companies].length

  expect(search_result_length).to eq 2
end

Run this once successfully then forever more use the cassette (which is simply a YAML file of the response). Your tests are no longer dependent on APIs you don't control. If ES or your pagination gem update their code, simply re-record the cassette when you know the API is up and working. There really isn't any other option without making your tests extremely brittle or stubbing things you shouldn't stub.

Note that although we have stubbed `tire` above - and we don't own it - it's ok in these cases because the return values are entirely irrelevant to the test.

Problem

I have a Rails app with an Rspec test suite which has some feature/controller tests depending on ElasticSearch. When we test the "search" feature around the system (and other features depending on ES) we use a real ES, it works perfectly at development environment when we're running single spec files. When the suite runs at our CI server it gets weird, because sometimes ES won't keep in sync fast enough for the tests to run successfully. I have searched for some way to run ES in "syncronous mode", or to wait until ES is ready but haven't found anything so far. I've seen some workarounds using Ruby `sleep` but it feels unacceptable to me. How can I guarantee ES synchronicity to run my tests? How do you deal with ES on your test suite? Here's one of my tests: ``` context "given params page or per_page is set", :elasticsearch do let(:params) { {query: "Resultados", page: 1, per_page: 2} } before(:each) do 3.times do |n| Factory(:company, account: user.account, name: "Resultados Digitais #{n}") end sync_companies_index # this is a helper method available to all specs end it "paginates the results properly" do get :index, params expect(assigns[:companies].length).to eq 2 end end ``` Here's my RSpec configure block and ES helper methods: ``` RSpec.configure do |config| config.around :each do |example| if example.metadata[:elasticsearch] Lead.tire.index.delete # delete the index for a clean environment Company.tire.index.delete # delete the index for a clean environment example.run else FakeWeb.register_uri :any, %r(#{Tire::Configuration.url}), body: '{}' example.run FakeWeb.clean_registry end end end def sync_companies_index sync_index_of Company end def sync_leads_index sync_index_of Lead end def sync_index_of(klass) mapping = MultiJson.encode(klass.tire.mapping_to_hash, :pretty => Tire::Configuration.pretty) klass.tire.index.create(:mappings => klass.tire.mapping_to_hash, :settings => klass.tire.settings) "#{klass}::#{klass}Index".constantize.rebuild_index klass.index.refresh end ``` Thanks for any help!

Original source