Dynamic RSpec spec generation

dynamic, rspec, ruby, specifications

Solution

This doesn't work, because ... RSpec doesn't ... 'see' any specs at compile-time ...

I was wondering why, and did some research. If you examine the following code and the output, you can draw some conclusions :

- there is no compile time in Ruby

- the body of a class/module is immediately evaluated and executed

- so is the `describe` method

- the body of a `describe` block is immediately processed by RSpec

- RSpec stores `before` blocks and `it` blocks for later execution

- when the parsing of a file is terminated, RSpec starts reporting the `describe` strings and executing the stored `before/it` example blocks

The following code demonstrates this.

module SO
    puts '>>>>> Ruby sees module SO and executes/evaluates its body'
    cases = [1,2,3]

    describe "SO Cases" do
        puts "in module SO, RSpec sees describe Cases self=#{self}"
        before(:each) do
            puts '  in before(:all)'
        end

        cases.each do |case_|
            puts "    in each loop with case_=#{case_}"
            it "Case #{case_}" do
                puts "        check spec for case #{case_}"
            end
        end
    end
end

module M # to avoid "warning: class variable access from toplevel"
    puts '>>>>> Ruby sees module M and executes/evaluates its body'
    describe "M Cases" do
        puts "in module M, RSpec sees describe Cases self=#{self}, ancestors :"
        ancestors.each {|a| puts "    #{a}"}
        print 'self.methods.grep(/^it/) : '; p self.methods.grep(/^it/).sort
        before(:all) do
            puts "  in before(:all) self=#{self}"
            @@cases = [1,2,3]
            puts "  ... now cases=#{@@cases}"
            File.open('generated_cases.rb', 'w') do |fgen|
                fgen.puts 'puts "\n*** inside generated_cases.rb ***"'
                fgen.puts 'module GenSpecs'
                fgen.puts "puts '>>>>> Ruby sees module GenSpecs and executes/evaluates its body'"
                fgen.puts '    describe "GenSpecs Cases" do'
                fgen.puts '        puts "in module GenSpecs, RSpec sees describe Cases"'
                @@cases.each do |case_|
                    puts "    in each loop with case_=#{case_}"
                    fgen.puts <<-IT_SPECS
        it "Case #{case_}" do
            puts "    some spec for case_=#{case_}"
        end
        IT_SPECS
                end
                fgen.puts '    end'
                fgen.puts 'end # module GenSpecs'
                fgen.puts "puts '>>>>> end of ruby file generated_cases.rb <<<<<'"
            end
            puts 'file generated_cases.rb has been closed'
            require_relative 'generated_cases'
        end

        it 'has loaded Cases' do
            @@cases.should_not be_empty
        end
    end
end
puts '>>>>> end of ruby file t2_spec.rb <<<<<'

Execution :

$ ruby -v
ruby 1.9.2p320 (2012-04-20 revision 35421) [x86_64-darwin12.2.0]
$ rspec --version
2.12.2
$ rspec --format nested t2_spec.rb 
>>>>> Ruby sees module SO and executes/evaluates its body
in module SO, RSpec sees describe Cases self=#<Class:0x007fcaf49a6e80>
    in each loop with case_=1
    in each loop with case_=2
    in each loop with case_=3
>>>>> Ruby sees module M and executes/evaluates its body
in module M, RSpec sees describe Cases self=#<Class:0x007fcaf2852e28>, ancestors :
    #<Class:0x007fcaf2852e28>
    RSpec::Core::ExampleGroup
    ...
self.methods.grep(/^it/) : [:it, :it_behaves_like, :it_should_behave_like, :its]
>>>>> end of ruby file t2_spec.rb <<<<<

SO Cases
  in before(:all)
        check spec for case 1
  Case 1
  in before(:all)
        check spec for case 2
  Case 2
  in before(:all)
        check spec for case 3
  Case 3

M Cases
  in before(:all) self=#<RSpec::Core::ExampleGroup::Nested_2:0x007fcaf2836ca0>
  ... now cases=[1, 2, 3]
    in each loop with case_=1
    in each loop with case_=2
    in each loop with case_=3
file generated_cases.rb has been closed

*** inside generated_cases.rb ***
>>>>> Ruby sees module GenSpecs and executes/evaluates its body
in module GenSpecs, RSpec sees describe Cases
>>>>> end of ruby file generated_cases.rb <<<<<
  has loaded Cases

GenSpecs Cases
    some spec for case_=1
  Case 1
    some spec for case_=2
  Case 2
    some spec for case_=3
  Case 3

Finished in 0.01699 seconds
7 examples, 0 failures

Creating a file and requiring it this way is for the demonstration and may not work in your case. I would recommend doing it in two phases : in the first you read the spreadsheet and create a file.rb with a `describe` and several `it` examples. In the second phase you launch Ruby to process the generated file.

Problem

I'd like to generate a series of specs dynamically based on an external data source. Specifically, I have a Google Spreadsheet that's downloaded, and each row in the spreadsheet should be used for its own test case: ``` describe "Cases" do before(:all) do # Download spreadsheet and # populate cases in DB end Cases.each do |case| it "Case #{case.num}" do # spec end end end ``` This doesn't work, because, for starters with RSpec, doesn't (as I understand it) 'see' any specs at compile-time, so `before(:all)` is never executed. If I were to place an empty `it` block it'd get the `before(:all)` to execute, but then I'd run into the problem that `Cases.each` is evaluated before anything else, which is empty because it hasn't been populated yet by the `before(:all)` block. In short, I'm confused, and my understanding of RSpec appears very limited. I'd like to fetch data, use that data to set up a bunch of specs, and then have them run by RSpec. This would (might?) work if Cases were an array set up before-hand (outside the describe block), but I need it to be set up at runtime. Is what I want possible to do in RSpec?

Original source