When running
rsync -Paz /home/sbird "/run/media/sbird/My Passport/sbird"
As told by someone, I run into a ran out of storage error midway. Why is this? My disk usage is about 385 GiB for my home folder, and there is at least 800 GiB of space in the external SSD (which already has stuff like photos and documents). Does rsync make doubly copies of it or something? That would be kind of silly. Or is it some other issue?
Note that the SSD is from a reputable brand (Western Digital) so it is unlikely that it is reporting a fake amount of storage.
Personally, I have no more tips that those that have already been presented in this comment section. What I would do now to find out what’s going on is the age-old divide-and-conquer debugging technique:
Using rsync or a file manager (yours is Dolphin), only copy a few top-level directories at a time to your external drive. Note the directories you are about to move before each transfer. After each transfer check if the sizes of the directories on your internal drive (roughly) match those on your external drive (They will probably differ a little bit). You can also use your file manager for that.
If all went fine for the first batch, proceed to the next until you find one where the sizes differ significantly. Then delete that offending batch from the external drive. Divide the offending batch into smaller batches (select fewer directories if you tried transferring multiple; or descend into a single directory and copy its subdirectories piecewise like you did before).
In the end you should have a single directory or file which you have identified as problematic. That can then be investigated further.
YIKES, I found that .local is around 30GB on my system ssd, over 50GB in the external SSD. Much of that is due to Steam and Kdenlive. I can try uninstalling Steam…
Something interesting that I found: according to dolphin, many folders have many GB extra (e.g. 52GB vs 66GB for documents folder which is kind of crazy) while filelight records 52GB vs 112GB for documents folder, which if true, is kind of insane. Using du -sh records 53G vs 136G (they’re the same when using --apparent-size, weird. Specifically for Godot directory, it’s 3.8GB vs 41 GB!!!)!!! Files like videos and games seem to be about the same size, while Godot projects with git are much bigger. Weird.
These differences really are insane. Maybe someone more knowledgeable can comment on why different tools differ so wildly in the total size they report.
I have never used BTRFS, so I must resort to forwarding googled results like this one.
Could you try
compsize ~? If thePerccolumn is much lower than 100% or theDisk Usagecolumn is much lower than theUncompressedcolumn, then you have some BTRFS-specific file-size reduction on your hands, which your external exFAT naturally can’t replicate.percentage of total is 83% (292G vs uncompressed 349G apparently)
It’s good you found some pathological examples, but I’m at the end of my rope here.
You can use these examples and the other information you gathered so far and ask specifically how these size discrepancies can be explained and maybe mitigated. I suggest more specialized communities for this such as !linux@lemmy.ml, !linux@programming.dev, !linux@lemmy.world, !linux4noobs@programming.dev, !linux4noobs@lemmy.world, !linuxquestions@lemmy.zip.
I have cross posted to a Linux community. Thank you so much for all your help :DDDD
I’m assuming that Filelight count file size differently, and I will be trusting the result from dolphin more
using -H with rsync doesn’t seem to do anything unfortunately…
With Dolphin Godot directory is 1GB vs 5GB. Why is there a difference between filelight, dolphin, and du -sh? So weird
it looks like much of the extra Godot bulk is in .git and .Godot directories
Something about never knowing the time when one has two clocks
Oh that’s actually a good idea. Thanks person! I will report back soon