Big files uploading (WebException: The connection was closed unexpectedly)

UPDATED See post #3 below. There is a need to upload a file to the web automatically (without browser). Host - Mini File Host v1.2 (if this does matter). Didn't find specific api in documentation, so at first i sniffed browser requests in Firebug as follows : ``` Params : do Value : verify POST /upload.php?do=verify HTTP/1.1 Host: webfile.ukrwest.net User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; ru; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 4.0.20506) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: ru,en-us;q=0.7,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://filehoster.awardspace.com/index.php Content-Type: multipart/form-data; boundary=---------------------------27368237179714 Content-Length: 445 -----------------------------27368237179714 Content-Disposition: form-data; name="upfile"; filename="Test.file" Content-Type: application/octet-stream 12345678901011121314151617sample text -----------------------------27368237179714 Content-Disposition: form-data; name="descr" -----------------------------27368237179714 Content-Disposition: form-data; name="pprotect" -----------------------------27368237179714-- ``` Here we can see parameter, headers, content type and chunks of information (1 - file name and type, 2 - file contents, 3 - additional params - description and password, not necessarily applied). So i've created a class that emulates such a behaviour step by step : HttpWebRequest on the url, apply needed parameters to request, form request strings with StringBuilder and convert them to byte arrays, read a file using FileStream, putting all that stuff to MemoryStream and then writing it to request (took major part of a code from an article at CodeProject where it uploads a file to Rapidshare host). Neat and tidy, but... It doesn't seem to work :(. As result it returns initial upload page, not a result page with links i could parse and present to a user... Here are main methods of an Uploader class : ``` // Step 1 - request creation private HttpWebRequest GetWebrequest(string boundary) { Uri uri = new Uri("http://filehoster.awardspace.com/index.php?do=verify"); System.Net.HttpWebRequest httpWebRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(uri); httpWebRequest.CookieContainer = _cookies; httpWebRequest.ContentType = "multipart/form-data; boundary=" + boundary; httpWebRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.2; ru; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 4.0.20506)"; httpWebRequest.Referer = "http://filehoster.awardspace.com/index.php"; httpWebRequest.Method = "POST"; httpWebRequest.KeepAlive = true; httpWebRequest.Timeout = -1; //httpWebRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; httpWebRequest.Headers.Add("Accept-Charset", "windows-1251,utf-8;q=0.7,*;q=0.7"); httpWebRequest.Headers.Add("Accept-Encoding", "gzip,deflate"); httpWebRequest.Headers.Add("Accept-Language", "ru,en-us;q=0.7,en;q=0.3"); //httpWebRequest.AllowAutoRedirect = true; //httpWebRequest.ProtocolVersion = new Version(1,1); //httpWebRequest.SendChunked = true; //httpWebRequest.Headers.Add("Cache-Control", "no-cache"); //httpWebRequest.ServicePoint.Expect100Continue = false; return httpWebRequest; } // Step 2 - first message part (before file contents) private string GetRequestMessage(string boundary, string FName, string description, string password) { System.Text.StringBuilder stringBuilder = new StringBuilder(); stringBuilder.Append("--"); stringBuilder.Append(boundary); stringBuilder.Append("\r\n"); stringBuilder.Append("Content-Disposition: form-data; name=\""); stringBuilder.Append("upfile"); stringBuilder.Append("\"; filename=\""); stringBuilder.Append(FName); stringBuilder.Append("\""); stringBuilder.Append("\r\n"); stringBuilder.Append("Content-Type: application/octet-stream"); stringBuilder.Append("\r\n"); return stringBuilder.ToString(); } // Step 4 - additional request parameters. Step 3 - reading file is in method below private string GetRequestMessageEnd(string boundary) { System.Text.StringBuilder stringBuilder = new StringBuilder(); stringBuilder.Append(boundary); stringBuilder.Append("\r\n"); stringBuilder.Append("Content-Disposition: form-data; name=\"descr\""); stringBuilder.Append("\r\n"); stringBuilder.Append("\r\n"); stringBuilder.Append("Default description"); stringBuilder.Append("\r\n"); stringBuilder.Append(boundary); stringBuilder.Append("\r\n"); stringBuilder.Append("Content-Disposition: form-data; name=\"pprotect\""); stringBuilder.Append("\r\n"); stringBuilder.Append("\r\n"); stringBuilder.Append(""); stringBuilder.Append("\r\n"); stringBuilder.Append(boundary); stringBuilder.Append("--"); //stringBuilder.Append("\r\n"); //stringBuilder.Append(boundary); //stringBuilder.Append("\r\n"); return stringBuilder.ToString(); } // Main method public string ProcessUpload(string FilePath, string description, string password) { // Chosen file information FileSystemInfo _file = new FileInfo(FilePath); // Random boundary generation DateTime dateTime2 = DateTime.Now; long l2 = dateTime2.Ticks; string _generatedBoundary = "----------" + l2.ToString("x"); // Web request creation System.Net.HttpWebRequest httpWebRequest = GetWebrequest(_generatedBoundary); // Main app block - form and send request using (System.IO.FileStream fileStream = new FileStream(_file.FullName, FileMode.Open, FileAccess.Read, FileShare.Read)) { byte[] bArr1 = Encoding.ASCII.GetBytes("\r\n--" + _generatedBoundary + "\r\n"); // Generating pre-content post message string firstPostMessagePart = GetRequestMessage(_generatedBoundary, _file.Name, description, password); // Writing first part of request byte[] bArr2 = Encoding.UTF8.GetBytes(firstPostMessagePart); Stream memStream = new MemoryStream(); memStream.Write(bArr1, 0, bArr1.Length); memStream.Write(bArr2, 0, bArr2.Length); // Writing file byte[] buffer = new byte[1024]; int bytesRead = 0; while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0) { memStream.Write(buffer, 0, bytesRead); } // Generating end of a post message string secondPostMessagePart = GetRequestMessageEnd(_generatedBoundary); byte[] bArr3 = Encoding.UTF8.GetBytes(secondPostMessagePart); memStream.Write(bArr3, 0, bArr3.Length); // Preparing to send httpWebRequest.ContentLength = memStream.Length; fileStream.Close(); Stream requestStream = httpWebRequest.GetRequestStream(); memStream.Position = 0; byte[] tempBuffer = new byte[memStream.Length]; memStream.Read(tempBuffer, 0, tempBuffer.Length); memStream.Close(); // Sending request requestStream.Write(tempBuffer, 0, tempBuffer.Length); requestStream.Close(); } // Delay (?) System.Threading.Thread.Sleep(5000); // Getting response string strResponse = ""; using (Stream stream = httpWebRequest.GetResponse().GetResponseStream()) using (StreamReader streamReader = new StreamReader(stream/*, Encoding.GetEncoding(1251)*/)) { strResponse = streamReader.ReadToEnd(); } return strResponse; } ``` Plays with ProtocolVersion (1.0, 1.1), AllowAutoRedirect (true/false), even known ServicePoint.Expect100Continue (false) didn't fix an issue. Even a 5sec timeout before getting response (thought in case of a big file it doesn't uploads so quick) didn't help. Content type "octet-stream" was chosen by purpose to upload any file (could use some switch for most popular jpg/zip/rar/doc etc., but that one seems universal). Boundary is generated randomly from timer ticks, not a big deal. What else? :/ I could give up and forget this, but i feel i'm pretty close to solve and then forget about it :P. In case you need the whole application to run and debug - here it is (70kb, zipped C# 2.0 VS2k8 solution, no ads, no viruses) : @Mediafire @FileQube @FileDropper