Data Hoarding General /dhg/

Wait, there are people who actually use golang?

will add this in the pastebin. thanks user

yh lol. hipster hoarders.

Attached: 1021171811387_17_5028287941_acb85e2f99_o.jpg (1821x2767, 1020K)

Gross, cool pic though

Attached: 176bd74509f06498341a43a538646762.gif (294x289, 147K)

That is cool dude, didn't see some of those tools

Copy these somewhere github.com/mayfrost/guides/blob/master/ALTERNATIVES.md#file-utilities
That list is not dedicated to data hoarding but many tools are excellent for the job

Attached: apucomputer.png (804x799, 10K)

I am really into 5.6 megahertz DSD. 4tb mirrored on separate HDD. Just filled them up time for a massive updgrade.

I dont bother with movies and tv. Its a waste IMHO as they will allways be seeded somewhere. Hi-res music, my photos and memes and books are whats most valuable to me.

Attached: b8fca928-58ce-4559-bf67-d474fee07b3e.png (1831x950, 209K)

Attached: 1021171901339_59_15151103053_2c75fe2fd6_o.jpg (1773x2697, 589K)

Never going to happen. Koreans don't care about copyright except in Korea.

being a pleb that doesnt mind low quality movies is nice

>tv shows, anime
>~2200 movies
>tomes of games I will play eventually™
>all well under 10TB

Attached: disks.png (817x156, 8K)

updated mah man. thx for that.

Attached: 1021171907537_59_4400492866_1125a591ac_o.jpg (1329x2000, 588K)

>5.6 megahertz DSD
snakeoil victim spotted

digits dont lie

I've got a terabyte of shit sitting on two 3.5 HDDs that I can't access cause I have a laptop and I'm saving for a desktop.
plus a 1 terabyte hdd full of random documents and stuff, i call it my backup but it's constantly plugged in so it's probably gonna fail soon
one of the hdds is full of pirated movies, I don't pay for anything, even when it's more convenient to do so. the other is full of pictures from Jow Forums, and pirated music
desu I'm not sure I own anything legal

>Ten TB of torrented porn
help me lads

There's also Anti-Twin which works really well on Windows. On linux it hits its limits in the wine emulation.
It has a nicer interface than dupeguru imho

Nothing about backups or ensuring integrity of your data?

Pls share

Anybody got a decent guide/suggestion for babbys first box?
Been using an old PC as a headless debian file/torrent server but it uses too much electricity (costs me >$10/month just to have it on). Looking for something with a lower power draw and smaller size.
Want to use it for storage + streaming to a TV and maybe playing some emulated games on.

nice (((datamining))) thread

Attached: 1539488531321.png (1920x1080, 563K)

>5.6 megahertz DSD
I download these and immediately convert to 24/96 FLAC.

Unlimited storage Google Drive that I pay about £6 a month for. Keeps me covered.

I'm currently converting all my FLACs to wavpack, saves about 5%

Would if I had better net. (70kB/s up)

How about this motherboard and case?
supermicro.com/products/motherboard/atom/A2SDi-4C-HLN4F.cfm
fractal-design.com/home/product/cases/node-series/node-304-black

i've been using deezloader to download record label catalogs in flac, so far have about 1TB so far including stuff i havn't tagged and organized
not that much overall but it's stuff i've ripped myself and for the most part isn't available anywhere else
i have 5-6TB of other music, but i'm probably going to stick with this method for a while, other than large collection torrents like audio4u

Whew that's a bit of a pricey boy. I was thinking of something like an Odroid XU4

>Use your old case,
>Get an efficient 300/350watt psu
>Some low power pentium chip like a G4560 or something cheaper
>idk for motherboard maybe if you can score a cheap supermicro somewhere
>4gb ECC

should be like 40-80watts

give me some. i'll add them

I hope you’re happy. You’ve gone done broke my heart.
>notsokimochii.dat

Anyone using CEPH? Please share your experience

My current box already averages ~60watts. Electricity is expensive as fuck here.

Attached: file.png (311x101, 4K)

>Odroid XU4
Are you saying you want to build a storage server that runs all its IO through a single USB port?

Fug, didn't realise it had that same problem as rpis.

>1TB of FLAC music
I'm impressed. I have 19000 files of music, 20% FLAC and it comes to around 350GB.
I'd be interested in DSD if it were more available.

fucking hell, that's some autism right there

I doubt a human could tell the difference.

simple foobar2000 search-and-convert-to-wavpack turned into "shuffle files around the HDDs because the spinning disk is almost full", but some space savings were made.

old 1TiB Samsung ran at 100% during the copy and convert, where as 6TiB Toshiba just sat there, chilling. fug, I need another Toshiba.

rofl, the savings were way smaller than I predicted. pic related. maybe I should've used tighter compression on WavPack, since this was 100% bound by HDD performance.

Attached: wavpack.jpg (747x403, 155K)

should've also copied the replaygain tags to the converted files, since they don't need to be updated when going lossless to lossless. another hour wasted.

what about the odroid hc2?

I have a bunch of content on an 8tb drive in my PC that I want to move to a device that I would keep running 24/7 so I could access it from my phone and other devices.

My question is if I should build a quiet, small form factor PC, or just buy a Synology NAS or something? I would like to be able to stream movies and I've heard that transcoding could be an issue with a NAS.

Attached: archive.png (1920x1080, 199K)

I've been downloading a bunch of technical ebooks; I've amassed somewhere around 5K give or take. When I started, I kept the filenames simple but as my collection grew, I realized I need a better filename to organize everything. Right now the pattern I'm using is "Publisher_Title_Edition" so an example would be "PrenticeHall_C_Programming_Language_The_2nd." This makes it good for locating files based on publishers, but was curious if there's a better, more descriptive pattern I could use. Was thinking of adding author's last name and year published somewhere in the name but I don't know how to arrange the name.

Build a server yourself, can easily be done for sub $300 and is much more versatile than a prebuilt NAS plus you will have more room for expansion down the line.

thread theme: youtu.be/xb_PyKuI7II

use calibre and tag it all

maybe this is the place to ask...

are there any drives larger than 2TB that come in the form factor of 2.5" with a z-height of 9.5mm or smaller?

is this a theoretical maximum based on platter size? i havent seen anything beyond 2TB in this form factor in a long time, and it doesnt look like anything is on the horizon.

current technology is plateauing, see: anandtech.com/show/11925/western-digital-stuns-storage-industry-with-mamr-breakthrough-for-nextgen-hdds/2

Thanks, I'll look into it.

good one. it fits.

Attached: 1021171841406_54_7518834546_c14debf4fa_o.jpg (1948x2849, 1021K)

there isn't. i've looked into it. adding to it, if you increase the block size more than 64k, linux has issues with the hard drive too.
so, need more parallel hardware in HDD itself which is still under research.

Attached: 1021171841406_61_7518831434_f3c2cc4d75_o.jpg (1765x2435, 1.1M)

got it, thanks.

raid's not a backup, so what's a good backup solution? a NAS with a few drives and periodic copying of data I wish to keep onto them?
as for right now I have
>1+ of gaymen and other related, with misc stuff on the same drive
>700 GB of porn
>300 GB of music, some lossy but the vast majority being MP3 320kbps, FLAC, and other lossless formats
>400 GB of pictures, video, and video editing material
>1TB of movies, television shows, cartoons, and anime

>Second NAS at a different location that mirrors the data on your first array
>Cold storage
>Cloud i.e gdrive
>Backup service i.e backblaze
>Or just leaving it in RAID6 because frankly >2 disk failures/ransomware/fires are quite unlikely and fuck the extra cost
For your small amount of data I'd just mirror it on some cheap ass cloud service like gsuite or something. Or you could buy a second HDD and cold store it. 3tb HDDs go on sale for $50ish and 4tbs sometimes go for $80 on the best sales. Black Friday coming up soon too.

>what's a good backup solution?
At least 3-2-1 for irreplaceable data

>Or just leaving it in RAID6 because frankly >2 disk failures/ransomware/fires are quite unlikely and fuck the extra cost
This doesn't protect you against software or user failure, which is exactly why people say RAID is not a backup.

- 3.3 TB of anime
- 200 GB of fetish porn
- 75 GB of Music (FLAC)
- Maybe another few 100 GB of non-essential or random stuff

RAID is for uptime. Keeps your data available in the event that one of your drives decide to neck itself.

A back-up solution would be to mirror all the shit you want to back-up in externals. A good solution is 3-2-1. I usually settle with just 3-2 (it'd be 2-2 in this case, right?)

I'm not claiming raid is a backup, I am saying that it's "good enough" if you are trying to protect a large amount of replaceable data and you don't want to spend more money on a proper solution.

Yeah, like i said, its uptime.
I'm not going to back up TBs of anime and porn, but it's going to take a little more than a dead drive for me to lose it.

and remember: your movies, porn and downloaded music is not irreplaceable

2x 512Gb 960 Pro's in RAID 0 for primary and WD HDDs for the rest

Attached: speccy2.jpg (824x576, 113K)

>your movies, porn and downloaded music is not irreplaceable
where you think we are. maybe no porn
but, music and movies, yeah nah gonna have to disagree with you there.

(some) music is irreplaceable, movies definitely aren't

>build a quiet, small form factor PC, or just buy a Synology NAS
depends on how much you're willing to spend and how many drives you're going to use.

well, all files are now converted. total space savings were a whopping three gigabytes for about 20hrs of cpu time. tightening the compression on wavpack resulted to additional 0.5% file savings for 100% increase in encoding speed. however, this was worth it, because now the files have proper padding for tags and I don't have to rewrite the whole file when the tags are updated.

/blog, thanks for reading

there is a lost media wiki and theres a lot of movies.

once there is commercial digital disc release of a movie, chances of it becoming a lost movie will reduce to almost nil. about all lost movies are lost because the reels were lost in a fire or otherwise destroyed.

good one. how's the seek speed in the files themselves while playing?
any noticeable speed change while moving through the audio?

in this age of censoring??? nah. the past year and how the internet monopolies have gone about censoring and memory holing videos and content, the need for archival only increased in me honestly speaking.
any movie which maybe "offensive" can be wiped from the internet.
not taking those chances man.

seeking is instantaneous, decoding speed is 1300x realtime - about the same as flac

redpill me on wavpack a little.
im used to FLAC for the past couple of years and most of my stuff is in FLAC.
hesitant to move to other ones due to lack of support in some softwares etc.
so, is it worth it and how's the support in other softwares? i feel wavpack is not yet mainstream yet.

When I jizz in tissue and keep them around, am I data hoarding then?

data rot you fucking idiot. put it in a freezer.

Attached: 1021171841406_35_7333098304_585aa1da6f_o.jpg (1634x2200, 820K)

There's no TRIM command on RAID.

Ironwolf vs RED?

it's open source and very much viewed as alternative codec, that's my reason for using it. it's both faster to decode and encode, and wv compresses better than flac, but that's a bonus. for ultimate non-mainstream audio compression experience, but hybrid wavpack into matroska container with chapter marks.

next up: i'm re-encoding my tom's audio kompressor range rips and putting them into matroska. single album, single file; very minimalist approach to audio archiving.

that's so interesting

Attached: supreme.jpg (600x750, 26K)

t. public tracker leeching yify watcher

what are you using? any tools?
i'm gonna add them to the /dhg/ pastebin.
just put links or scripts which are online which you use. anything may help.

Attached: 1021171907537_69_4646652384_816130ac8b.jpg (500x352, 91K)

first post in the thread and it's a bait post.
good one.

Attached: 0425193923794_042_4809497063_030030cbf9_o.jpg (2561x1707, 1.39M)

foobar2000's all I'm using. it does literally anything you would want to with audio management.

for example, this search would return all range rips with embedded cue sheets where TAK codec was used: %codec% IS TAK AND %__cue_embedded% IS yes. then it's a simple matter of re-encoding the files with wavpack encoder that comes with foobar2000.

thanks man. good stuff i'll try it out on a few ones and see.

Attached: 1021172823589_51_14315809208_3f2c67e2d0_o.jpg (3978x1900, 1020K)

>delete all porn from 3x8tb
>next month
>buy one more hdd for recover porn

Attached: 1540116609715.png (444x386, 135K)

Seeding is for sissies

If its not avc then its not high quality

>hevc

I used to hoard movies, music, even software but some time ago I found a hdd from about 10 years ago and some old stuff on it, I realized I will never watch those movies again because watching a movie second time is completely pointless. Since then I deleted most of my files. There are better things to do with your life than hoard

Give link to tool discography flac, right now

I thought hevc is mostly used for compression but the actual quality is comparable to x264