s3.getObject().createReadStream() : How to catch the error?
amazon-s3, amazon-web-services, javascript, node.js, stream
Solution
If you'd like to catch the `NoSuchKey` error thrown by `createReadStream` you have 2 options:
- Check if key exists before reading it.
- Catch error from stream
First:
s3.getObjectMetadata(key)
.promise()
.then(() => {
// This will not throw error anymore
s3.getObject().createReadStream();
})
.catch(error => {
if (error.statusCode === 404) {
// Catching NoSuchKey
}
});
The only case when you won't catch error if file was deleted in a split second, between parsing response from `getObjectMetadata` and running `createReadStream`
Second:
s3.getObject().createReadStream().on('error', error => {
// Catching NoSuchKey & StreamContentLengthMismatch
});
This is a more generic approach and will catch all other errors, like network problems.
Problem
I am trying to write a program to get a zip file from s3, unzip it, then upload it to S3. But I found two exceptions that I can not catch. 1. `StreamContentLengthMismatch: Stream content length mismatch. Received 980323883 of 5770104761 bytes.` This occurs irregularly. 2. `NoSuchKey: The specified key does not exist.` This happens when I input the wrong key. When these two exceptions occur, this program crashes. I'd like to catch and handle these two exceptions correctly. I want to prevent a crash. ``` const unzipUpload = () => { return new Promise((resolve, reject) => { let rStream = s3.getObject({Bucket: 'bucket', Key: 'hoge/hoge.zip'}) .createReadStream() .pipe(unzip.Parse()) .on('entry', function (entry) { if(entry.path.match(/__MACOSX/) == null){ // pause if(currentFileCount - uploadedFileCount > 10) rStream.pause() currentFileCount += 1 var fileName = entry.path; let up = entry.pipe(uploadFromStream(s3,fileName)) up.on('uploaded', e => { uploadedFileCount += 1 console.log(currentFileCount, uploadedFileCount) //resume if(currentFileCount - uploadedFileCount <= 10) rStream.resume() if(uploadedFileCount === allFileCount) resolve() entry.autodrain() }).on('error', e => { reject() }) } }).on('error', e => { console.log("unzip error") reject() }).on('finish', e => { allFileCount = currentFileCount }) rStream.on('error', e=> { console.log(e) reject(e) }) }) } function uploadFromStream(s3,fileName) { var pass = new stream.PassThrough(); var params = {Bucket: "bucket", Key: "hoge/unzip/" + fileName, Body: pass}; let request = s3.upload(params, function(err, data) { if(err) pass.emit('error') if(!err) pass.emit('uploaded') }) request.on('httpUploadProgress', progress => { console.log(progress) }) return pass } ``` This is the library I use when unzipping. https://github.com/mhr3/unzip-stream Help me!!