Understanding attributes in AWS DynamoDB with Ruby

amazon-dynamodb, amazon-web-services, ruby

Solution

I've finally figured out most of what I needed to understand the data model of DynamoDB and using the Ruby SDK.

Below is my example code, which hopefully will help someone else, and I've got a fully fleshed out example here: https://gist.github.com/Integralist/9f9f2215e001b15ac492#file-3-dynamodb-irb-session-rb

# https://github.com/BBC-News/alephant-harness can automate the below set-up when using Spurious
# API Documentation http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations.html
# Ruby SDK API Documentation http://docs.aws.amazon.com/AWSRubySDK/latest/frames.html#!AWS/DynamoDB/Client/V20120810.html

require "aws-sdk"
require "dotenv"
require "spurious/ruby/awssdk/helper"

Spurious::Ruby::Awssdk::Helper.configure
# => <AWS::Core::Configuration>

Dotenv.load(
  File.join(
    File.dirname(__FILE__), "config", "development", "env.yaml"
  )
)
# => {"AWS_REGION"=>"eu-west-1", "AWS_ACCESS_KEY_ID"=>"development_access", "AWS_SECRET_ACCESS_KEY"=>"development_secret", "DYNAMO_LU"=>"development_lookup", "DYNAMO_SQ"=>"development_sequence", "SQS_QUEUE"=>"development_queue", "S3_BUCKET"=>"development_bucket"}

dyn = AWS::DynamoDB::Client.new :api_version => "2012-08-10"
dyn = AWS::DynamoDB::Client::V20120810.new
# => #<AWS::DynamoDB::Client::V20120810>

dyn.create_table({
  # This section requires us to define our primary key 
  # Which will be called "item_id" and it must be a numerical value
  :attribute_definitions => [
    { :attribute_name => "item_id", :attribute_type => "N" }
  ],
  :table_name => "my_books",
  # The primary key will be a simple Hash key (not a Hash/Range which requires both key types to be provided)
  # The attributes defined above must be included in the :key_schema Array
  :key_schema => [
    { :attribute_name => "item_id", :key_type => "HASH" }
  ],
  :provisioned_throughput => {
    :read_capacity_units  => 10,
    :write_capacity_units => 10
  }
})
# => {:table_description=>{:attribute_definitions=>[{:attribute_name=>"item_id", :attribute_type=>"N"}], :table_name=>"my_books", :key_schema=>[{:attribute_name=>"item_id", :key_type=>"HASH"}], :table_status=>"ACTIVE", :creation_date_time=>2014-11-24 16:59:47 +0000, :provisioned_throughput=>{:number_of_decreases_today=>0, :read_capacity_units=>10, :write_capacity_units=>10}, :table_size_bytes=>0, :item_count=>0}}

dyn.list_tables
# => {:table_names=>["my_books"]}

dyn.scan :table_name => "my_books"
# => {:member=>[], :count=>0, :scanned_count=>0}

dyn.put_item(
  :table_name => "my_books",
  :item => {
    "item_id" => { "N" => "1" }, # oddly this needs to be a String and not a strict Integer?
    "item_title" => { "S" => "My Book Title"},
    "item_released" => { "B" => "false" }
  }
)
# Note: if you use an "item_id" that already exists, then the item will be updated.
#       Unless you use the "expected" conditional feature

dyn.put_item(
  :table_name => "my_books",
  :item => {
    "item_id" => { "N" => "1" }, # oddly this needs to be a String and not a strict Integer?
    "item_title" => { "S" => "My Book Title"},
    "item_released" => { "B" => "false" }
  },
  # The :expected key specifies the conditions of our "put" operation.
  # If "item_id" isn't NULL (i.e. it exists) then our condition has failed.
  # This means we only write the value when the key "item_id" hasn't been set.
  :expected => {
    "item_id" => { :comparison_operator => "NULL" }
  }
)
# AWS::DynamoDB::Errors::ConditionalCheckFailedException: The conditional check failed

dyn.scan :table_name => "my_books"
# => {:member=>[{"item_id"=>{:n=>"1"}, "item_title"=>{:s=>"My Book Title"}, "item_released"=>{:b=>"false"}}], :count=>1, :scanned_count=>1}

dyn.query :table_name => "my_books", :consistent_read => true, :key_conditions => {
  "item_id" => {
    :comparison_operator => "EQ",
    :attribute_value_list => [{ "n" => "1" }]
  },
  "item_title" => {
    :comparison_operator => "EQ",
    :attribute_value_list => [{ "s" => "My Book Title" }]
  }
}
# => {:member=>[{"item_id"=>{:n=>"1"}, "item_title"=>{:s=>"My Book Title"}, "item_released"=>{:b=>"false"}}], :count=>1, :scanned_count=>1}

dyn.query :table_name => "my_books", 
  :consistent_read => true, 
  :select => "SPECIFIC_ATTRIBUTES",
  :attributes_to_get => ["item_title"],
  :key_conditions => {
  "item_id" => {
    :comparison_operator => "EQ",
    :attribute_value_list => [{ "n" => "1" }]
  },
  "item_title" => {
    :comparison_operator => "EQ",
    :attribute_value_list => [{ "s" => "My Book Title" }]
  }
}
# => {:member=>[{"item_title"=>{:s=>"My Book Title"}}], :count=>1, :scanned_count=>1}

dyn.delete_item(
  :table_name => "my_books",
  :key => {
    "item_id" => { "n" => "1" }
  }
)
# => {:member=>[], :count=>0, :scanned_count=>0}

Problem

I can't seem to wrap my head around the AWS Ruby SDK documentation for DynamoDB (or more specifically the concepts of the DynamoDB data model). Specifically I've been reading: http://docs.aws.amazon.com/AWSRubySDK/latest/frames.html#!AWS/DynamoDB.html Note: I have read through the Data Model documentation as well and it's still not sinking in; I'm hoping a proper example in Ruby with clear up my confusion In the following code snippet, I create a table called "my_books" which has a primary_key called "item_id" and it's a Hash key (not a Hash/Range combination)... ``` dyn = AWS::DynamoDB::Client::V20120810.new # => #<AWS::DynamoDB::Client::V20120810> dyn.create_table({ :attribute_definitions => [ { :attribute_name => "item_id", :attribute_type => "N" } ], :table_name => "my_books", :key_schema => [ { :attribute_name => "item_id", :key_type => "HASH" }, ], :provisioned_throughput => { :read_capacity_units => 10, :write_capacity_units => 10 } }) # => {:table_description=>{:attribute_definitions=>[{:attribute_name=>"item_id", :attribute_type=>"N"}], :table_name=>"my_books", :key_schema=>[{:attribute_name=>"item_id", :key_type=>"HASH"}], :table_status=>"ACTIVE", :creation_date_time=>2014-11-24 16:59:47 +0000, :provisioned_throughput=>{:number_of_decreases_today=>0, :read_capacity_units=>10, :write_capacity_units=>10}, :table_size_bytes=>0, :item_count=>0}} dyn.list_tables # => {:table_names=>["my_books"]} dyn.scan :table_name => "my_books" # => {:member=>[], :count=>0, :scanned_count=>0} ``` I then try and populate the table with a new item. My understanding is that I should specify the numerical value for `item_id` (which is the primary key) and then I could specify other attributes for the new item/record/document I'm adding to the table... ``` dyn.put_item( :table_name => "my_books", :item => { "item_id" => 1, "item_title" => "My Book Title", "item_released" => false } ) ``` But that last command returns the following error: ``` expected hash value for value at key item_id of option item ``` So although I don't quite understand what the hash will be made of, I try doing that: ``` dyn.put_item( :table_name => "my_books", :item => { "item_id" => { "N" => 1 }, "item_title" => "My Book Title", "item_released" => false } ) ``` But this now returns the following error... ``` expected string value for key N of value at key item_id of option item ``` I've tried different variations, but can't seem to figure out how this works? EDIT/UPDATE: as suggested by Uri Agassi - I changed the value from `1` to `"1"`. I'm not really sure why this has to be quoted as I've defined the type to be a number and not a string, but OK let's just accept this and move on.

Original source