I have a gaming laptop and a phone, both of which have USB 3.2 Gen 2 and Gen 1 ports. I also use a USB 4 Type-C cable. Now, recently, I have downloaded music files of over 300GB. If I transfer them one by one, it takes a lot of time. Today, I compressed the folder to a single zip file, and the transfer finished in less than 20m. Why is that so?
for each file, the computer has to ask the phone about the file, wait for the phone to process the file request, and respond. Then it can start transferring the file. With a single file, it can copy everything in one go without stopping.
You’re basically being bottlenecked by your phone’s CPU, and by high latency in a single-threaded task.
No, is not the phone’s CPU. I guarantee you that can process several gigabytes a second.
It’s the overhead of all that handshaking and confirmation coming over USB, and having to be verified with the sometimes slow phone storage chips or worse on a microSD.
It’s IO overhead and waiting all over the place, not a CPU bottleneck.
Why does this overhead not exist when I’m sending files over USB to an external HDD or flash drive?
I have an external HDD array connected via USB 3.2 and it handles file transfers same as a SATA drive. There’s no handshaking beyond the initial negotiation of the USB connection, certainly not on a per-file basis.
That’s using eSATA for a protocol and not USB in all likelihood. Also a lot of the handshaking and confirmation come from the USB driver itself. You won’t see anything fancy, it’ll just be annoyingly slow.
Interesting, so eSATA protocol can be used over a USB physical link?
Only ones that support it. It has a different plug and pins, there is just a port that can take both.
If you’re plugging in with a normal USB cable, it could be doing a few other tricks to not use the normal USB way of transferring files, or just a fast enough device (eg: 3.2g2+) to not notice the overhead without a side by side.
Most of the ‘tricks’ are just usually having a driver or controller chip open up a raw data pipe (the universal serial part of USB) and exposing the drive as … anything other than removable storage. If the OS doesn’t see the drive as removable, it won’t do most of the crazy overhead stuff of confirming and validating each transfer.
Also if it’s USB4, it can expose the device as just a pcie device (I don’t remember if it’s native to 4 or has to be an overlap with Thunderbolt but eh), or a DisplayPort data stream. That’s how those newfangled docks connect to the host while keeping speed up while the dock has every plug under the sun; they’re all devices down stream of the mimic’d pcie endpoint, which gets to blast data over raw.
This external drive I have is just USB mass storage. It’s using the USB protocol and shows as removable external storage, same as a flash drive.
Removable storage is not the issue here. There is no significant overhead introduced by having removable storage with USB mass storage protocol.
OP is forced into using MTP which does suck and adds overhead. This is not the “normal” way of transferring to me, that would be USB mass storage.