Dear Jow Forums

Dear Jow Forums,

My father likes taking thousands of photos, transfer them from the camera to his laptop, and then make backup upon backup upon backup of the same goddamn files scattered throughout his absolute mess of a folder structure.
His system drive is an SSD, that local storage space comes at a premium, so I've granted him access to an area of my spinny rust RAID1 array, where he can then upload his shit over network.

Now for obvious reasons I'd like to avoid my precious storage wasted by him sending tens of thousands of files which all have several duplicates at different file paths on the same physical hard drive.

Can anyone recommend a free piece of software that can sift through a given set of folders, look for duplicate files, and present the results to me for final action? Attributes such as file name, size, and date modified would be enough for what I need.

Attached: 1497307733658.jpg (621x502, 23K)

Other urls found in this thread:

alldup.de/en_download_alldup.php
twitter.com/NSFWRedditGif

Bash

Uh... so he keeps everything on one SSD? Did you explain the concept of a backup to him?

that sounds like a job for a 6 line bash script, yeah

I recommend rsync

Fdupes

In an easily stealable portable device, yes. He's a sixty-some year old boomer, you know how it goes.

>use an image size optimizer for each file
>sort by file size to find dupes
solves 2 problems easily

Dont ZFS and BTRFS have offline dedup?

So you'd fuck up all the original files to get rid of duplicates?

Bravo.

>what is lossles compression

>I've granted him access to an area of my spinny rust RAID1 array
don't. tell him he needs to get his own home NAS or cloud space.

make a script that compares md5 and transfers those that are unique. idk.

Ultimately if he can't manage to clean his shit up he'll just be paying for new higher capacity drives when the volume runs full.
The last thing I need is another separate NAS to manage (if he got one for himself that would for sure be up to me, too), which is why I offered some space on my already established hardware.

Thanks for all the suggestions so far

Depends on what format the files are in. Even if PNG is lossless you still lose data when converting RAW to PNG.

It actually turns out CCleaner (the non-pozzed version from before Avast took over) has a nifty little duplicate finder tool that quickly spits out a detailed list of directories which hold identical files based on a number of conditions you define in the search.
Thanks guys!

dupeguru my nigger

Which version?

Checked that one out too, seems good

5.32.6129

Alldup portable is a pretty useful program to find duplicate images. alldup.de/en_download_alldup.php

buy some oldass harddisks and some ancient PC from craigslist (cost you less than

Visipics my dude

For the future, he should use Adebis Photo Sorter
Very simple software that copy photo (jpeg and raw) in YYYY/MM/DD folder structure (or other).
Its a proprietary abandonware but I like it.
Personally, I use this + Nextcloud server for backup. I have like 600GB of photos.

If he doesn't change the files themselves, only the name, can't you just MD5 them then delete duplicates?
Should be a pretty simple python or bash script, no?

This RAID1 array is a headless computer?

borgbackup is the answer, pic related is what i call with cron

Attached: longscrot.png (1130x1500, 198K)

The fuck? What is your question?

>Now for obvious reasons I'd like to avoid my precious storage wasted by him sending tens of thousands of files which all have several duplicates at different file paths on the same physical hard drive.
How about you simply create a TrueCrypt/Vera Crypt container of whatever space you want to give him and simply mount the container and give him the access to this file?

If you create a container of 1TiB or 500MiB or whatever he can not get over this limit.
I think there are other ways to create limited space for your father.
Something with VDI files or some shit.

Why the fuck are you even giving him access to your server did he threaten you to give him your space or what?
>user you ungrateful fuck you horde all your HDDs for yourself give your father some
I buy him a external HDD and call it a day.

Personally i did have pretty illiterate father I have gotten away with taking his money to "buy him 8TB HDDs" and Buying the 8TB HDD using it in my computer and taking a old 8 GB USB stick i used and giving this to my father.
He was happy and it all checked out he did have his blazing 8 whatever space it was enough for him for the next years.

And he payed for the 8 TB out of his own pocket.

Attached: 0a88949d0c42fdf21adf6b2b304c1daaacbb28be.jpg (880x660, 147K)