Getting big decimals back from a yaml-serialized field in the database with Ruby on Rails

bigdecimal, ruby-on-rails, serialization

Solution

The ActiveSupport core extension code you mentioned is "already" fixed in the master branch (the commit is about an year old and undoes an implementation as old as Rails 2.1.0), but since Rails 3.2 only gets security updates, your app might be stuck with the old implementation.

I think you'll have three options:

- Port your Rails app to Rails 4.

- Backport Psych's `BigDecimal#to_yaml` implementation (monkey patch the monkey patch).

- Switch to Syck as YAML engine.

Each option has its own disadvantage:

Porting to Rails 4 seems to me the best alternative, if you have the time to do so (the commit mentioned above is available in Rails since v4.0.0.beta1). Since it's not released yet, you'd have to work with the beta. I don't suspect any grand changes to come, though some GSoC ideas read as if they still could make it into the 4.0 release...

Monkey patching the ActiveSupport monkey patch should be fairly less complex. Though I did not find the original implementation of `BigDecimal#to_yaml`, a somewhat related question led to this commit. I guess I'll leave it to you (or other StackOverflow users) how to backport that particular method.

As quick'n'dirty workaround, you could simply use Syck as YAML engine. In the same question, user rampion posted this piece of code (which you could place in an initializer file):

YAML::ENGINE.yamler = 'syck'

class BigDecimal
  def to_yaml(opts={})
    YAML::quick_emit(object_id, opts) do |out|
      out.scalar("tag:induktiv.at,2007:BigDecimal", self.to_s)
    end
  end
end

YAML.add_domain_type("induktiv.at,2007", "BigDecimal") do |type, val|
  BigDecimal.new(val)
end

The main disadvantage here (besides the unavailability of Syck on Ruby 2.0.0) is, that you can't read normal BigDecimal dumps within your Rails context, and everybody who want's to read your YAML dumps, needs the same kind of loader:

BigDecimal.new('43.21').to_yaml
#=> "--- !induktiv.at,2007/BigDecimal 43.21\n"

(Changing the tag to `"tag:ruby/object:BigDecimal"` won't help either, since it yields `!ruby/object/BigDecimal`...)

Update – things I learned so far

The odd behaviour seems to date back to times of Rails 1.2 (you might also say Feb'2007), as per this blog entry.

Modifying the `config/application.rb` in this way did not help:

require File.expand_path('../boot', __FILE__)

# (a)

%w[yaml psych bigdecimal].each {|lib| require lib }
class BigDecimal
  # backup old method definitions
  @@old_to_yaml = instance_method :to_yaml
  @@old_to_s    = instance_method :to_s
end

require 'rails/all'

# (b)

class BigDecimal
  # restore the old behavior
  define_method :to_yaml do |opts={}|
    @@old_to_yaml.bind(self).(opts)
  end
  define_method :to_s do |format='E'|
    @@old_to_s.bind(self).(format)
  end
end

# (c)

At different points (here a, b and c) a `BigDecimal.new("42.21").to_yaml` has yielded some interesting output:

# (a) => "--- !ruby/object:BigDecimal 18:0.4221E2\n...\n"
# (b) => "--- 42.21\n...\n"
# (c) => "--- 0.4221E2\n...\n"

where a is the default behaviour, b is caused by the ActiveSupport Core Extension and c should have been the same result as a. Maybe I'm missing something...

On carefully rereading you question, I had this idea: Why not serialize in another format, like, JSON? Add another column to your database and migrate over time like this:

class Person < ActiveRecord::Base
  # the old serialized field
  serialize :preferences

  # the new one. once fully migrated, drop old preferences column
  # rename this to preferences and remove the getter/setter methods below
  serialize :pref_migration, JSON

  def preferences
    if pref_migration.blank?
      pref_migration = super
      save! # maybe don't use bang here
    end
    pref_migration
  end

  def preferences=(*data)
    pref_migration = *data
  end
end

Problem

Using Ruby on Rails I have a couple of fields that are serialized (arrays or hashes mostly). Some of those contain `BigDecimal`s. It is very important that those big decimals remain big decimals, but Rails is turning them into floats. How do I get `BigDecimal`s back? Looking into this issue I found that serializing a big decimal in plain Ruby, without Rails, works as expected: ``` BigDecimal.new("42.42").to_yaml => "--- !ruby/object:BigDecimal 18:0.4242E2\n...\n" ``` but in a Rails console it doesn't: ``` BigDecimal.new("42.42").to_yaml => "--- 42.42\n" ``` That number is the string representation of the big decimal, so it's all-right. But when I read it back it is read as a float, so even if I convert it to `BigDecimal` (something I don't want to do as it's error prone), it is possible I'll lose precision, which isn't acceptable for my app. I tracked down the culprit to `activesupport-3.2.11/lib/active_support/core_ext/big_decimal/conversions.rb` which overrides the following method in BigDecimal: ``` YAML_TAG = 'tag:yaml.org,2002:float' YAML_MAPPING = { 'Infinity' => '.Inf', '-Infinity' => '-.Inf', 'NaN' => '.NaN' } # This emits the number without any scientific notation. # This is better than self.to_f.to_s since it doesn't lose precision. # # Note that reconstituting YAML floats to native floats may lose precision. def to_yaml(opts = {}) return super if defined?(YAML::ENGINE) && !YAML::ENGINE.syck? YAML.quick_emit(nil, opts) do |out| string = to_s out.scalar(YAML_TAG, YAML_MAPPING[string] || string, :plain) end end ``` Why would they do that? And more importantly, how do I work-around it?

Original source