Reading text from Amazon s3 stream
amazon-s3, c#, readfile, readline, streamreader
Solution
There is a similar performance bottleneck on HTTPWebResponse in .NET, which is probably what the AmazonS3 class they made is wrapping.
It's caused by the object taking a long time to resolve proxy settings, there are a few potential solutions listed here, but the easiest option might be to add the following to your app.config file:
<system.net>
<defaultProxy enabled="false">
<proxy/>
<bypasslist/>
<module/>
</defaultProxy>
</system.net>
Alternatively you could replace the call here:
AWSClientFactory.CreateAmazonS3Client(amazonSettings.AccessKey, amazonSettings.SecretAccessKey)
with a call to an overload that accepts a third parameter of 'AmazonS3Config', where you can specify a null proxy via 'AmazonS3Config.ProxyHost = null' - which should effectively be the same as the above configuration change for only that request.
Problem
I use the following code to read text file from Amazon S3, and processing it line by line. This code works but the problem is it is slow. ``` GetObjectRequest getObjRequest = new GetObjectRequest() .WithBucketName(amazonSettings.BucketName) .WithKey(_fileKey); using (AmazonS3 client = AWSClientFactory.CreateAmazonS3Client( amazonSettings.AccessKey, amazonSettings.SecretAccessKey)) using (GetObjectResponse getObjRespone = client.GetObject(getObjRequest)) using (Stream amazonStream = getObjRespone.ResponseStream) { StreamReader amazonStreamReader = new StreamReader(amazonStream); tempGsContact = new GSContact(); while ((_fileLine = amazonStreamReader.ReadLine()) != null) { if (_fileLine.Equals("END:VCARD")) { // Make process 1 } else if (!_fileLine.Equals(string.Empty)) { //Make process 2 } } } ``` The question is: can I get more sufficient way to reduce the time cost?