Data Hoarding General - /dhg/

>talk about what you hoard anons
>talk about programs and how you annotate and arrange your data
Making a better arrangement would help other anons in being much efficient.

Made a little pastebin
pastebin.com/raw/znXyf01z
which others can reuse if im not there to create a new thread once the thread reaches 300 replies.
Peace and keep hoarding faggots
*Add this pastebin in the OP always.

Previous threads :
warosu.org/g/thread/68129051
warosu.org/g/thread/68151091

Attached: dhg.jpg (1440x1080, 1.28M)

Other urls found in this thread:

matroska.org/technical/specs/codecid/index.html
scottlinux.com/2016/08/28/gpu-passthrough-with-kvm-and-debian-linux/
digitalocean.com/community/tutorials/initial-server-setup-with-debian-9
openvpn.net/vpn-server-resources/how-to-connect-to-access-server-from-a-linux-computer/
github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS
freebsd.org/doc/handbook/zfs.html
twitter.com/SFWRedditGifs

I've been archiving lossless music for a while in FLAC and whatnot.
But I still don't know what format, codec and resolution I should download for movies.
There's so many re-releases, remasters, codecs, resolutions, cameras. It's quite a mess.
What kind of stuff should I look for when downloading movies for archival purposes?

matroska for videos is serving me well.
check this out user
matroska.org/technical/specs/codecid/index.html

Fuck, I'm thinking of setting up a server to do so, and also to backup my data and run onlyoffice. Is a computer built from used old parts enough or some more hardware is needed?

RAM is not that important but, a good processor is important.
preferably one with a higher frequency.

Attached: 1021171907537_07_4236440304_f3138719f6_o.jpg (1904x2586, 1.14M)

also check this out for hardware -

Unless you are running FreeNas
>8 GB of RAM is the absolute minimum requirement. 1 GB per terabyte of storage is a standard starting point for calculating additional RAM needs, although actual needs vary.

Also, guys is it possible to run Debian for example and then have some kind of RAID setup on that?

yes, depends on the file system though and some OS just have better support and features streamlining for those filesystems like freebsd, ghostbsd for hammer, zfs etc.
But, if you're very comfortable with debian then sure go with it but if you're unsure with debian i suggest using those specific OSes which correlate with your needs.

Attached: 1021171817971_30_5335151012_47f8fae0eb_o.jpg (1132x2064, 705K)

The easiest is probably plain old md. It's reliable, pretty mature, easy to work with.
I got about 800 MB/s with 4 cheap 2TB 7200 RPM Seagate disks in RAID10 Far2 and 64MB chunks, which is pretty close to the maximum the disks can do. Just don't try to run a Windows VM off the disk, the retarded IO patterns will kill performance.
If you feel like experimenting a bit you can try BTRFS, which can guarantee data integrity by way of checksums. If you use a recent enough kernel (4.14 as absolute minimum) then it's pretty stable.
Or there's ZFS, which sort-of works on Linux, but I wouldn't bet my data on it. Better try something like Illumos or FreeBSD if you want it.

Thank you.
I am quite comfortable with Debian as I use it on my main laptop and my crappy laptop server. Are there any tools or technologies that I can check for Debian file server? I also plan to use onlyoffice, maybe nextcloud, also openvpn and maybe a web server too on the same machine. Do you have any recommendations how to set that up? I feel like that would need some kind of virtualization, right?

then check up on gpu passthrough to use cross platform applications.
and since you're talking about webserver and openvpn, check out fail2ban for protection first then setup a server (some local folder) through that vpn.

i have this saved for gpu passthrough in my bookmarks lel but it's old. so you may need to read through the latest docs.
scottlinux.com/2016/08/28/gpu-passthrough-with-kvm-and-debian-linux/

check out digitalocean of vpn stuff, they write some good stuff on debian for beginners and moderates. they also have some good starter materials for arch and ubuntu too.

here's a starting points -
digitalocean.com/community/tutorials/initial-server-setup-with-debian-9
openvpn.net/vpn-server-resources/how-to-connect-to-access-server-from-a-linux-computer/

Attached: 1021171832663_21_6199457799_9e90f7ff47_o.jpg (1281x981, 341K)

Thanks.
Regarding the starting points and fail2ban I have played with those a bit and already have them installed.
Should I create a different vm for every service I run?

yes, most of the time it's always better to compartmentalize your services so you can manage them better with scripts and you can do it automatically too.
one thing to note is when you do that, the system cannot identify between duplicate sessions of same service, for example consider you opened office once and then started office again, it would start another session.
so, i didnt take care of handling this specific issue but if you have time, you can do some exception handling to not have more than 1 session of that particular service.

Attached: 1021171907537_09_4235666335_4d292bd147_o.jpg (2634x1900, 1.52M)

I hoard pdfs and anime images. I've got about 250GB worth of anime images (some of which includes manga), and 400GB+ of PDFs.

have any pdfs on survival?
if so, post a magnet or mega or anything.

Hoarding TV-Series in .mkv 1080p. Have a bunch of 8TB disks already filled up to the max. Buy more anytime soon.
Mostly loaded from share-online

why .mkv?

not him, but probably because it's a universal container that can hold anything
very popular with pirates for the same reason

Because it plays on literally any device, can contain meta data, and if you push it trough MKVtoolNix, you can name the sound files after their codec, bitrate and if it's 2.0 or 5.1 sound.
I do the same with the video file and name them aftercodec, resolution, fps and bitrate
It even can hold chapters. So what do you wish more?

what other format (container) should i use instead?

Oppaitime has sitewide freeleech for a week. I've downloaded so much hentai.

some good NTR links?

Damn user, post some of your favorite PDFs.

Go fuck yourself

You can setup ZFS on Debian which should suffice all RAID setups you might want to have. Avoid hardware RAID at all costs. Here's a guide if you also want your root partition on ZFS on Debian:
github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS
If you want to learn how to use ZFS and set it up, I'd recommend taking a look at FreeBSD's guide on ZFS:
freebsd.org/doc/handbook/zfs.html

Just download the Do/k/ument.

already have it. still scouring for more.

Good luck. Living is your best survival resource. Ave Nex Alea.

Any topic you're specifically interested in? There is a torrent with 150GB of medical books and papers in it, and there are terabytes of free PDFs on pubmed.

Interested in the med link?

True.
Saluto Nex Alea.

Is there a way to rip youtube comments in an automated fashion?

There is also an 85GB font collection there.

Not him but thanks. That pic tho.

Depends on how much space you have. The most obvious choice would be downloading blu-ray isos or mkv remuxes because they are both in original blu-ray quality. mkvs are more convenient imo, but they don't contain the extra shit from the blu-ray, only the movie.