What if an OS stored a checksum of every file on your computer so your web browser could know if you downloaded a duplicate
What if an OS stored a checksum of every file on your computer so your web browser could know if you downloaded a...
Brayden Rodriguez
Other urls found in this thread:
bench.cr.yp.to
twitter.com
Michael Foster
what if you weren't rarted
Evan Brooks
Why would your OS do this when your Web Browser can easily record checksums of all files in your download locations?
What a fucking awful idea to force the kernel to even do that for a web browser of all things.
Adrian Brooks
you'd probably save some bandwidth then
Tyler Adams
what if you have hundreds of thousands if not millions of files and that this is a huge waste of time
Noah Collins
Because what if I move files around
what if it would save me organization time
Hudson Rodriguez
Sounds like you're a disorganized cunt to begin with. Have you considered taking the time to organize your files so you don't have a problem as retarded as repeatedly downloading the same file because you don't know where you put it the last time?
Cameron Ramirez
useful for deduplication, some filesystems already store checksums along with files.
What if?
>muh extra 100mb of space taken up for 10tb of files
Nolan Lopez
>download 40 GB game
>spend even more time hashing all the files
Carson Sanchez
>checksum
why not just use the file size in bytes. i'm sure it's unique enough as a primary key
Luke Mitchell
edge cases kind of screw it up. While it's probably reasonably accurate for large files, small files are likely to have lots of collisions. It's also easily exploitable and shouldn't be used for trusting outside sources.
Jack Wood
unironically this
no use case for this shit
Aiden Foster
... I'm guessing you don't know much about log n algorithms and data structures. Protip, it's not hard.
Michael Ramirez
>hashing speed is slower than download speed
bench.cr.yp.to
nigger
Ayden Sanchez
If it's so easy then why don't you just fucking do it yourself, submit a patch and call it a day?
Jordan Moore
>small files are likely to have lots of collisions
right but they're small so who cares. but if you do, run a diff on collision to resolve it
Asher Hill
Why is this even a problem to begin with? There are use cases where you want to download the same file. What happens when you mount one or several filesystem over a network or from an external drive? The answer is this is a nonproblem.
Nolan Foster
What is ZFS deduplication?
Henry Scott
You can't because there's absolutely no way to calculate a hash before you download the file.
It'd be nice if it was built into http but it isn't.
As others said, you can use post-download dedup or use file length as an unique id in the cases the server gives you that value (not always).
Austin Anderson
btrfs has out-of-band deduplication as well
Jonathan Fisher
oh it already exists hmmmm
Isaiah Morales
yes, but it's possibly different to how you're thinking
it won't prevent or delete the new file, but just make the new file take up no additional disk space
both files are 'there', they just internally share the same physical space on disk
Brody Martinez
That's an even better idea than what I was thinking
Hudson Nelson
ZFS has optional deduplication that works similarly* to this but it takes a lot of memory, slows down the FS, and doesn't save much space in the grand scheme of things. A lot of the cloud storage and backup services do dedup behind the scenes because that's the scale where it starts to make sense.