Why are concurrent downloads faster
May 1
Many download managers like this, this and this support downloading a file over multiple parallel connections, one per thread.
The concept is that each connection will download one part of the file separately.
For example if there are 5 connections, then the first connection is going to download the first 0-20% portion of the file, second connection will download 20-40% portion and so on..
Similarly, on the server side, there will be 5 threads, where one thread will be reading 20% of the file in parallel.
But, I thought that trying to concurrently read a single file with multiple threads will actually make the download significantly slower, since the read head of the mechanical disk will have to do more seeks than before.
Even if we assume that the disk controller queuing mechanism is intelligent enough to batch all the 5 multipart requests to a single file together in one sequential read, it does not give us any advantage over doing the read in one just one thread and then serving the file over just 1 http connection.
So how are parallel downloads to a file faster?
1 answer
Accepted answer · original discussion
May 1
My understanding is that downloading different file parts in parallel is only useful when the bottleneck is the network connection: either the upload bandwidth of the server from which you download, or the bandwidth of the network between the server and you. When these links are saturated, the available bandwidth will be divided up across the connections, and in some cases it could be divided evenly across connections. So if you have 5 connections open then you would be getting a larger share of the bandwidth than if you only have one.
Of course, this doesn't work if the server and network are sharing the bandwidth in a more clever way, e.g., by allocating the share between client IPs instead of connections.
When the bottleneck is disk IO on the server or client, then indeed this strategy will not help, and could even harm performance because the reads and writes will be less sequential. Also, when the bottleneck is the available bandwidth between your ISP and modem (which I'd say is maybe the most common case), then parallel downloads should neither harm nor help.
0 question comments
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.