Pagination in Amazon DynamoDB using Boto

Question

How do I paginate my results from DynamoDB using the Boto python library? From the Boto API documentation, I can't figure out if it even has support for pagination, although the DynamoDB API does have pagination support.

Accepted Answer

Boto does have support for "pagination" like behavior using a combination of "ExclusiveStartKey" and "Limit". For example, to paginate `Scan`.

Here is an example that should parse a whole table by chunks of 10

```
esk = None

while True:
    # load this batch
    scan_generator = MyTable.scan(max_results=10, exclusive_start_key=esk)

# do something usefull
    for item in scan_generator:
        pass  # do something usefull
    # are we done yet ?
    else:
        break;

# Load the last keys
    esk = scan_generator.kwargs['exclusive_start_key'].values()
```

EDIT:

As pointed out by @garnaat, it is possible that I misunderstood your actual goal. The above suggestion allows you to provide pagination like SO does for questions for example. No more than 15 per pages.

If you just need a way to load the whole result set produced by a given `Scan`, Boto is a great library and already abstracts this for you with no need for black magic like in my answer. In this case, you should follow what he (@garnaat) advises. Btw, he is the author of Boto and, as such, a great reference for Boto related questions :)

Pagination in Amazon DynamoDB using Boto

Solution

Problem

Related problems