How to access a file on Amazon S3 from the Command Line?
amazon-s3, amazon-web-services, homebrew, macos, terminal
Solution
The AWS Command Line Interface (CLI) is a unified tool to manage AWS services, including accessing data stored in Amazon S3.
The AWS Command Line Interface is available for Windows, Mac and Linux.
If the bucket owner has granted public permissions for `ListBucket`, then you can list the contents of the bucket, eg:
aws s3 ls s3://bml-data
If the bucket owner has granted public permissions for `GetObject`, then you can copy an object:
aws s3 cp s3://bml-data/churn-bigml-80.csv churn-bigml-80.csv
Both of these commands works successfully for me.
See also:
- AWS Command Line Interface Documentation
Problem
Question: Is there a simple way to access a data file stored on Amazon S3 directly from the command line? Motivation: I'm loosely following an online tutorial where the author links to the following URL: ``` s3://bml-data/churn-bigml-80.csv ``` It is a simple csv file, but I can't open it using my web browser, or with `curl`. The tutorial opens it with BigML, but I want to download the data for myself. Some googling tells me that there are a number of python and Scala libraries designed for S3 access ... but it would be really nice to open or download the file more directly. I use Mac and am a big fan of homebrew, so the perfect solution (for me) would work on this system. Bonus Question: Is there any good way to see the contents of an Amazon E3 bucket (that I don't own)? The nature of the file (80% of a particular data-set) makes me suspect that there may be a `churn-bigml-20.csv` file hiding somewhere out there. My automatic approach would be to try and curl / open the expected file ... the solution to the first question will allow me to check this hunch but in an ugly way. If anyone knows of a way to remotely explore the contents of a specific S3 bucket, then that would be very useful. Again, exploring google and SO tells me that there are libraries for this, but a more direct approach would be useful.