Multithreaded HTTP Download

Sebastiaan Koppe via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Sat Nov 28 04:50:19 PST 2015


On Saturday, 28 November 2015 at 10:46:11 UTC, tcak wrote:
> The only case that would make sense is if the server limits the 
> upload speed of each TCP socket. Unless you are in this 
> position, I do not expect to see any difference by opening 
> multiple sockets and requesting different parts of same file.

I used to live in China for some time, and access to 
international servers was dog-slow. It also was very 
unpredictable. Sometimes a connection would be ok (80 kb/s) then 
suddenly drop to < 3 kb/s. Other times it wouldn't start at all.

Having seen that behavior I build a multi-part downloader that 
opened around 40 connections and restarted a connection if it 
went below 1Kb/10sec.

Worked very well. With it I got up to 1Mb/sec. Which is pretty 
fast considering I used to stare at 9Kb/s download dialogs.

I really hated all those programs that needed to download 
something and expected the connection to be flawless and the 
throughput to be infinite.


More information about the Digitalmars-d-learn mailing list