It’s good you found some pathological examples, but I’m at the end of my rope here.
You can use these examples and the other information you gathered so far and ask specifically how these size discrepancies can be explained and maybe mitigated. I suggest more specialized communities for this such as !linux@lemmy.ml, !linux@programming.dev, !linux@lemmy.world, !linux4noobs@programming.dev, !linux4noobs@lemmy.world, !linuxquestions@lemmy.zip.
But I can critisize it for having only one sharp edge instead of 2. Or for being too short or too long. Or for having a handle that’s not shaped well for my hand. (That last metaphor is probably the correct one for the sentiment I’m going for.)
These differences really are insane. Maybe someone more knowledgeable can comment on why different tools differ so wildly in the total size they report.
I have never used BTRFS, so I must resort to forwarding googled results like this one.
Could you try compsize ~? If the Perc column is much lower than 100% or the Disk Usage column is much lower than the Uncompressed column, then you have some BTRFS-specific file-size reduction on your hands, which your external exFAT naturally can’t replicate.
Error: Too many unprocessed floats.
No; since every user defines the perfect program differently. Which should be the default behaviour(s)?
du --count-links only counts hard-linked files multiple types. I assumed you had a symlink loop that rsync would have tried to unwrap.
For instance:
$ ls -l
foo -> ./bar
bar -> ./foo
If you tried to rsync that, you’d end up with the directories foo, bar, foo/bar, bar/foo, foo/bar/foo, bar/foo/bar, foo/bar/foo/bar, ad infinitum, in the target directory.
Personally, I have no more tips that those that have already been presented in this comment section. What I would do now to find out what’s going on is the age-old divide-and-conquer debugging technique:
Using rsync or a file manager (yours is Dolphin), only copy a few top-level directories at a time to your external drive. Note the directories you are about to move before each transfer. After each transfer check if the sizes of the directories on your internal drive (roughly) match those on your external drive (They will probably differ a little bit). You can also use your file manager for that.
If all went fine for the first batch, proceed to the next until you find one where the sizes differ significantly. Then delete that offending batch from the external drive. Divide the offending batch into smaller batches (select fewer directories if you tried transferring multiple; or descend into a single directory and copy its subdirectories piecewise like you did before).
In the end you should have a single directory or file which you have identified as problematic. That can then be investigated further.
I’d say you can trust that.
I’m sorry. I was stupid. If you had duplicates due to a file system loop or symlinks, they would all be under different names. So you wouldn’t be able to find them with this method.
You checked 385GiB of files by hand? Is that size made up by a few humongously large files?
I suggest using uniq to check if you have duplicate files in there. (uniq’s input must be sorted first). If you still have the output file from the previous step, and it’s called rsync-output.txt, do sort rsync-output.txt | uniq -dc. This will print the duplicates and the number of their occurrences.
Idk if rsync traverses symlinks and filesystems by default,
From the man page:
Beginning with rsync 3.0.0, rsync always sends these implied directories as real directories in the file list, even if a path element is really a symlink on the sending side. This prevents some really unexpected behaviors when copying the full path of a file that you didn’t realize had a symlink in its path.
That means, if you’re transferring the file ~/foo/bar/file.txt, where ~/foo/bar/ is a symlink to ~/foo/baz, the baz directory will essentially be duplicated and end up as the real directory /SSD/foo/bar and /SSD/foo/baz.
Let’s back up and check your assumptions: How did you check that the disk usage of your home folder is 385GiB and that there are 780GiB of free disk space on your external drive?
You can store the output of rsync in a file by using rsync ALL_THE_OPTIONS_YOU_USED > rsync-output.txt. This creates a file called rsync-output.txt in your current directory which you can inspect later.
This, however means that you won’t see the output right away. You can also use rsync ALL_THE_OPTIONS_YOU_USED | tee rsync-output.txt, which will both create the file and display the output on your terminal while it is being produced.
What does that mean?
Imagine your hard drive like a giant cupboard of drawers. Each drawer can only have one label, so you must only ever store one “thing” in one drawer, otherwise you wouldn’t be able to label the thing accurately and end up not knowing what went where.
If you have giant drawers (a large block size), but only tiny things (small files) to store, you end up wasting a lot of space in the drawer. It could fit a desktop computer, but you’re only putting in a phone. This problem is called “internal fragmentation” and causes files to take up way more space than it would seem they need.
–––––
However, in your case, the target block size is actually smaller, so this is not the issue you’re facing.
Could it be you have lots of tiny files and/or a rather large-ish block size on your SSD?
You can check the block size with sudo blockdev --getbsz /dev/$THE_DEVICE.
I imagine that similar regulations exist in the EU. For instance, no copy machine will let you copy a Euro bill.
You can, but you shouldn’t. You don’t know what else relies on Math.random. That’s why there’s the wrapper function. That you can override in unit tests without worrying about other, unrelated code.
“You ain’t gonna need it”. It means: Build the thing you need now and don’t try to predict what you’ll need in 3 years. You ain’t gonna need it anyway.
If I understand correctly, you want to check the current domain (eg. w3schools) against api.w3schools and www.youtube, and return true for the first and false for the second (or the other way around)
Then technically it’s possible without string splitting:
const href = 'retrieved-from-the-anchor-element';
(new URL(href, location.href).hostname).endsWith(location.hostname)
In a little town in the Netherlands life was good. The planning committee actually had smart people who made sure to plan the town according to the people’s needs. Kosher butchers, for instance, were placed near Jewish community centers. They could do that because the town had kept records on who lived where, including the people’s religion. It really was a utopia.
Then the nazis invaded, got their hands on those registries, and with utmost efficiency cleared the town of all jews.
I don’t know if this story is true. I read it (probably much better worded) a few years ago. But it honestly doesn’t matter if it’s true.