How to download all data in a Google BigQuery dataset?

Question

Is there an easy way to directly download all the data contained in a certain dataset on Google BigQuery? I'm actually downloading "as csv", making one query after another, but it doesn't allow me to get more than 15k rows, and rows i need to download are over 5M. Thank you

Accepted Answer

You can run BigQuery extraction jobs using the Web UI, the command line tool, or the BigQuery API. The data can be extracted

For example, using the command line tool:

First install and auth using these instructions: https://developers.google.com/bigquery/bq-command-line-tool-quickstart

Then make sure you have an available Google Cloud Storage bucket (see Google Cloud Console for this purpose).

Then, run the following command:

```
bq extract my_dataset.my_table gs://mybucket/myfilename.csv
```

More on extracting data via API here: https://developers.google.com/bigquery/exporting-data-from-bigquery

How to download all data in a Google BigQuery dataset?

Solution

Problem