Download file progressively using TIdHttp
delphi, download, http, idhttp, indy
Solution
Variant #1 is the simpliest, and is how Indy is meant to be used.
Regarding the disk allocation issue, you can derive a new class from `TFileStream` and override its `SetSize()` method to do nothing. `TIdHTTP` will still attempt to pre-allocate the file when appropriate, but it will not actually allocate any disk space. Writing to `TFileStream` will grow the file as needed.
Regarding status reporting, `TIdHTTP` has `OnWork...` events for that purpose. The `AWorkCountMax` parameter of the `OnWorkBegin` will be the actual file size if known (the response is not chunked), or 0 if not known. The `AWorkCount` parameter of the `OnWork` event will be the cumulative number of bytes that have been transferred so far. If the file size is known, you can display the total percentage by simply dividing the `AWorkCount` by the `AWorkCountMax` and multiplying by 100, otherwise just display the `AWorkCount` value by itself. If you want to display the speed of the transfer, you can calculate that from the difference of `AWorkCount` values and the time intervals between multiple `OnWork` events.
Try this:
type
TNoPresizeFileStream = class(TFileStream)
procedure
procedure SetSize(const NewSize: Int64); override;
end;
procedure TNoPresizeFileStream.SetSize(const NewSize: Int64);
begin
end;
.
type
TSomeClass = class(TSomething)
...
TotalBytes: In64;
LastWorkCount: Int64;
LastTicks: LongWord;
procedure Download;
procedure HttpWorkBegin(ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
procedure HttpWork(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
procedure HttpWorkEnd(ASender: TObject; AWorkMode: TWorkMode);
...
end;
procedure TSomeClass.Download;
var
Buffer: TNoPresizeFileStream;
HttpClient: TIdHttp;
begin
Buffer := TNoPresizeFileStream.Create('somefile.exe', fmCreate or fmShareDenyWrite);
try
HttpClient := TIdHttp.Create(nil);
try
HttpClient.OnWorkBegin := HttpWorkBegin;
HttpClient.OnWork := HttpWork;
HttpClient.OnWorkEnd := HttpWorkEnd;
HttpClient.Get('http://somewhere.com/somefile.exe', Buffer); // wait until it is done
finally
HttpClient.Free;
end;
finally
Buffer.Free;
end;
end;
procedure TSomeClass.HttpWorkBegin(ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
begin
if AWorkMode <> wmRead then Exit;
// initialize the status UI as needed...
//
// If TIdHTTP is running in the main thread, update your UI
// components directly as needed and then call the Form's
// Update() method to perform a repaint, or Application.ProcessMessages()
// to process other UI operations, like button presses (for
// cancelling the download, for instance).
//
// If TIdHTTP is running in a worker thread, use the TIdNotify
// or TIdSync class to update the UI components as needed, and
// let the OS dispatch repaints and other messages normally...
TotalBytes := AWorkCountMax;
LastWorkCount := 0;
LastTicks := Ticks;
end;
procedure TSomeClass.HttpWork(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
var
PercentDone: Integer;
ElapsedMS: LongWord;
BytesTransferred: Int64;
BytesPerSec: Int64;
begin
if AWorkMode <> wmRead then Exit;
ElapsedMS := GetTickDiff(LastTicks, Ticks);
if ElapsedMS = 0 then ElapsedMS := 1; // avoid EDivByZero error
if TotalBytes > 0 then
PercentDone := (Double(AWorkCount) / TotalBytes) * 100.0;
else
PercentDone := 0.0;
BytesTransferred := AWorkCount - LastWorkCount;
// using just BytesTransferred and ElapsedMS, you can calculate
// all kinds of speed stats - b/kb/mb/gm per sec/min/hr/day ...
BytesPerSec := (Double(BytesTransferred) * 1000) / ElapsedMS;
// update the status UI as needed...
LastWorkCount := AWorkCount;
LastTicks := Ticks;
end;
procedure TSomeClass.HttpWorkEnd(ASender: TObject; AWorkMode: TWorkMode);
begin
if AWorkMode <> wmRead then Exit;
// finalize the status UI as needed...
end;
Problem
I want to implement a simple http downloader using TIdHttp (Indy10). I found two kind of code examples from the internet. Unfortunately none of them satisfy me 100%. Here is the code and I want some advise. Variant 1 ``` var Buffer: TFileStream; HttpClient: TIdHttp; begin Buffer := TFileStream.Create('somefile.exe', fmCreate or fmShareDenyWrite); try HttpClient := TIdHttp.Create(nil); try HttpClient.Get('http://somewhere.com/somefile.exe', Buffer); // wait until it is done finally HttpClient.Free; end; finally Buffer.Free; end; end; ``` The code is compact and very easy to understand. The problem is that it allocates disk space when downloading begins. Another problem is that we cannot show the download progress in GUI directly, unless the code is executed in a background thread (alternatively we can bind HttpClient.OnWork event). Variant 2: ``` const RECV_BUFFER_SIZE = 32768; var HttpClient: TIdHttp; FileSize: Int64; Buffer: TMemoryStream; begin HttpClient := TIdHttp.Create(nil); try HttpClient.Head('http://somewhere.com/somefile.exe'); FileSize := HttpClient.Response.ContentLength; Buffer := TMemoryStream.Create; try while Buffer.Size < FileSize do begin HttpClient.Request.ContentRangeStart := Buffer.Size; if Buffer.Size + RECV_BUFFER_SIZE < FileSize then HttpClient.Request.ContentRangeEnd := Buffer.Size + RECV_BUFFER_SIZE - 1 else HttpClient.Request.ContentRangeEnd := FileSize; HttpClient.Get(HttpClient.URL.URI, Buffer); // wait until it is done Buffer.SaveToFile('somefile.exe'); end; finally Buffer.Free; end; finally HttpClient.Free; end; end; ``` First we query the file size from the server and then we download file contents in pieces. Retrieved file contents will be save to disk when they are received completely. The potential problem is we have to send multiple GET requests to the server. I am not sure if some servers (such as megaupload) might limit the number of requests within particular time period. My expectations - The downloader should send only one GET-request to the server. - The disk space must not be allocated when the download begins. Any hints are appreciated.