How can I handle large files in Ruby?

file-io, ruby

Solution

You should try to catch exception to check if the problem is really on the read block or not.

Just so you know I already made a script with kinda the same syntax to search real big file of ~8GB without problem.

export = File.new("resultsfinal.txt","w+")

File.open("bibrec2.dat").each do |line|
  begin
    line.scan(/[a]{1}[1234567890xX]{10}\W/) do |x|
      export.puts x
    end
    line.scan(/[a]{1}[1234567890xX]{13}/) do |x|
      export.puts x
    end
  rescue
    puts "Problem while adding the result"
  end
end

Problem

I'm pretty new to programming, so be gentle. I'm trying to extract IBSN numbers from a library database .dat file. I have written code that works, but it is only searching through about half of the 180MB file. How can I adjust it to search the whole file? Or how can I write a program the will split the dat file into manageable chunks? edit: Here's my code: ``` export = File.new("resultsfinal.txt","w+") File.open("bibrec2.dat").each do |line| line.scan(/[a]{1}[1234567890xX]{10}\W/) do |x| export.puts x end line.scan(/[a]{1}[1234567890xX]{13}/) do |x| export.puts x end end ```

Original source