/hsg/ Home Server General

a $5/mo digitalocean VPS should be plenty if you just need a place to run some scripts on a schedule. But it's more fun to run your own server, and you'll eventually want to host more services as well.

Should I use docker or KVM to manage my servers in my server?

Has anybody who has taken the linux+ have a recommendation for a good pdf? can only seem to find downloads for materials on the 101/102 exam and the current edition is the 103/104 wasn't sure if there was better material out there.

host an ftp server
unironically opendirectories subr*ddit
depends i guess
my server averages maybe 70w but i dont pay more than €0.08/kWh so i think its worth it because that allows me to keep everything accessible at home
if you just want to do cronjobs, look into some dirt cheap vps in eastern europe, e.g. hostmaze.com/openvz-vps/ starting at $1.5 a month, unmetered bandwidth, cant say anything about performance though
i dont see any reason why it wouldnt be enough, i think any (headless) gnu/linux distro should do the trick honestly
im using docker, its neat, but i've been meaning to look into systemd-nspawn since i use lesbian, because that also looks really neat
whats the "linux+"? i've got a bunch of pdf:s from humblebundle that an user shared a while ago that might be related

Attached: 1550604169876.jpg (620x877, 237K)

/hsg/ is still going - awesome!

>i dont see any reason why it wouldnt be enough, i think any (headless) gnu/linux distro should do the trick honestly

Good to know, thanks!

this sounds cool actually. is it going to be a server that hosts its own wifi network for filesharing, or will it live on an existing (internet-connected) wifi network?
helping out the local gardening co-op or storing agricultural data for the apocalypse?

How's your current setup? Any changes?

Intranet only, for anyone within range of the antenna.

>helping out the local gardening co-op or storing agricultural data for the apocalypse?

Helping whoever is brave enough to connect and finds the subjects useful.

Attached: farm stuff.png (652x425, 29K)

kinda - mostly all neglected since I've been working on other things desu, lot of graphic design work.

Just pulled the pi cluster from the rack and planning out a re-install for shits'n'giggles

Run a tor relay.

Attached: tor.png (1916x534, 1.4M)

Other than being a bit resource heavy, how much safer would setting up a VM and running the server from inside the VM? Like VirtualBox for instance. Data to be accessed on that guest machine (server) would be via a shard folder through the VM to the host machine. The shared folder would be separated completely and the VM could be setup, cloned, and restored from that clone should anything happen to the guest machine. Then people online could access whatever you have setup. Wait, is there a name for this already?

>CELERON N3060
>eMMC
>41 Watts on idle
There's no fucking way that's true. The real power consumption is probably in the single digits with the screen turned off. It's practically free.

Its just nicer to run a hypervisor user then you can spin up multiple machines and turn them off when you want. Try new services etc

I wouldn't use virtualbox for server virtualization, try proxmox instead (it's kvm based).

Just use the cloud already
Home servers are outdated boomer tech

Cloud is server user. That's what people ITT are making. They are making their own little clouds.

What is piratebox?

Attached: piratebox[1].jpg (620x229, 156K)

tehehe lads i just accidentally my entire mariadb, time to restore from today's backups

Attached: 1553967687216.jpg (492x330, 43K)

>FarmBox

Attached: farmbox.png (596x425, 341K)

Buddy just passed me an r310
The qnap has a dead drive bay and was replaced so I copped it for free

The ac66u is a wireless bridge from my garage. I mainly use the qnap for backups so speed doesn't matter over the wireless bridge. Although I've had some very good performance.

Attached: IMG_20190408_160923.jpg (3264x2448, 2.15M)

I would like access too please. need some of those titles.
I offer this grey bune for your farm. I call her Nasim the Grey.

Attached: 6d705e94-4fe2-401f-bd0c-82c87c7dd5fb..jpg (1024x676, 34K)

I was looking to ditch the perc6/i that was in my server when I bought it. Max 2TB drives, lol. I wanted to keep using the drive backplane, but it had just a single SFF-8087 connector on it. So I bought a four port sata card and a mini-sas to four sata breakout cable, both from China.

The cable arrived in the post today, but the setup didn't work. Turns out the pins in the sas-sata breakout cables are exactly mirrored compared to proper sata. Who knew, right? So I had to flip the connectors on the sata card to the other side of the board, and then it worked.

My home server is nearing its final form for now, all I need is a pwm fan that runs fast enough on idle so it doesn't trigger the rpm alarm. Perhaps one of the industrial Noctuas.

Attached: IMG_3077.jpg (2272x1704, 484K)

>Nasim the Grey.

The grey breeze?

>I would like access too please. need some of those titles.

Check:

Attached: Homegrowmen Association.jpg (512x512, 293K)

I plan on hosting my data on VMs but I'm afraid of the data corrupting for some reason, what is the best backup data workflow?

>Copy the entire vmdk weekly and compress and store the data

I cant see this being feasible because of scaling, I eventually want like 20TB of shit

>Create snapshots weekly and compress

Better for storage, not sure how efficient it is

>rsync the media to another drive weekly

Again, cant scale very well

>Use RAID and mirror the psychical disk (ESXI datastore)

Might do this anyways and actually use my H710 for RAID 10

What's the best way to back up the VM data? The VMs themselves I can back up using raid, I dont want to use a third party cloud solution and I understand the risks of not having Geo-replication.

How do I prevent my VM data (media stored on the VM) from being corrupted anyways?

Attached: 1469992545070.jpg (682x1024, 307K)

When Nasim the Grey ROCKETS past you, all you'll feel a gentile breeze

planning to set up a home server for anime storage and torrenting
why and how should i visualize

>I plan on hosting my data on VMs but I'm afraid of the data corrupting for some reason, what is the best backup data workflow?

>>Copy the entire vmdk weekly and compress and store the data
>I cant see this being feasible because of scaling, I eventually want like 20TB of shit
Too storage intensive

>>Create snapshots weekly and compress
>Better for storage, not sure how efficient it is
Too storage intensive, but better.

>>rsync the media to another drive weekly
>Again, cant scale very well
Rsyncing the VM files is a shit idea.

>>Use RAID and mirror the psychical disk (ESXI datastore)
>Might do this anyways and actually use my H710 for RAID 10
>What's the best way to back up the VM data? The VMs themselves I can back up using raid, I dont want to use a third party cloud solution and I understand the risks of not having Geo-replication.
RAID is not a backup. It protects against a specific kind of hardware failure (losing individual disks). It does not protect against: Corruption, catastrophic damage (losing the whole server), accidental/intentional/malicious file damage.

>How do I prevent my VM data (media stored on the VM) from being corrupted anyways?
Use ECC RAM in your hypervisor. Set up your VMs to use an iSCSI datastore on a NAS with ECC using ZFS (or another resilient filesystem). This is a pretty standard entry level 'enterprise' setup.

Another standard method is to setup backup scripts for each application you're hosting (dump the database, copy all the data files, tarball and encrypt them all) and then scp/rsync the files to some central storage (preferably another computer acting as a NAS) and send the encrypted backups to some cloud storage (B2, S3, whatever). Run this script on a cron job and leave it alone. Many of the services you'll be running already support backing up like this (some directly to cloud storage). Don't worry about backing up OSes, OS is easy, data is hard.

Don't store your data inside a VM if you have that much, use dedicated storage hardware and connect to the VM with NFS or SMB.
I'm doing this and I only have an 8tb array. If you get a prebuilt NAS like QNAP, it will have filesystem snapshots built in.
The only extra concern with this method is making sure the mount happens before launching data services, so services don't freak out about missing data.

You mean... virtualize? Generally there is no reason to, just run this thing on BSD/Linux natively and you're fine.

What MAY be interesting is to containerize with docker-compose or something, it can abstract away some complexity if you want "the usual" from a software package with annoying dependencies and upgrade cycles (like quite a few have).

For example using the obvious images from here:
linuxserver.io/our-images/

> what is the best backup data workflow?
Borgbackup or syncthing (staggered versioning) is quite fine for use with "various data" from within or outside a VM.

> I cant see this being feasible because of scaling, I eventually want like 20TB of shit
Yea, you don't want the big pile of data in some vmdk? Even though you could still run borg on that from the outside or inside, it's pretty gay.

Keep the changing bulk data separate from the VM. Same as is done everywhere with docker etc, your bulk data isn't in the container images, it's elsewhere.

What are the best HD drives for home NAS servers these days? I had a rack of 3TB WD Red drives a while ago, but I need to refresh everything soon.

Thanks for the help guys, it seems like the consensus is to copy the data to a whole different server, this is pretty annoying though because I dont have another server and I dont want to use a 3rd party cloud

I'm not worried about the datastore being corrupted, but just the vmdk themselves, might do a rcopy to a different VM via NFS or something

im looking for a good router that can more than max out my 200Mbs internet. my current router is old shit, and tops at 30 with wifi. straight through to modem is 200

I'd like to keep it under $50.

get a router that support 802.11n

Well, I myself didn't say storage necessarily on a different server. I'd just not place it inside in the VM machine image because vmdk files are not all that great to backup.

Shouldn't be too much of a problem, get some Chinese router with gbe ports, openwrt/padawan/dd-wrt support and 1200-ish bg / ac wlan. They should be decent enough to run faster than what you have.

I think one popular way to go about it is to break open and extract the 8-10TB USB3 external drives and just operate them in erasure coded redundant storage (RAID, RAIDZ, Snapraid, Ceph ecpool, ...).

Because somehow these drives are usually quite a bit cheaper than the 8-10TB drives that you buy separately.

Let's say I rediscovered an old fileserver that has terabytes of movies, TV, ebooks, etc. on it.
What utilities are available to me to detect and catalogue the media in this collection? Ideally I could view the assumed title/series with the source filename and correct errors.

Plex seems like it's smart enough to match & download metadata for Movies/TV/Music, but is there an equivalent for books/comics? What do you all use to keep track of what media is in your store?

Thinking about ZFS, couldnt I just back up a ZFS snapshot and back that up to a different storage and schedule it?

Got an old Dell 3020mt running FreeNAS and Plex on it rn. Wondering if I should just use Debian and smb instead?

Attached: 1534317188725.gif (1920x1080, 1.13M)

Hellos lads, newbie with proxmox here. I've been using Debian for 2 or 3 years so I decided to give it a try.

Is there some basics/must I should do to the hypervisor? Like restrict logins or something. I'm really newbie to hypervisors, the most I've used was LXC without LXD.

I dunno, it should be fairly secure since it's basically Debian. I don't forward any ports from the outside into the host itself, only to the appropriate VMs.

If your server has IPMI, you can edit /etc/default/pve-ha-manager to enable the hardware watchdog

Unless you have a subscription, comment out the line in /etc/apt/sources.list.d/pve-enterprise.list and add instead:
deb download.proxmox.com/debian stretch pve-no-subscription
After that do the usual apt things to upgrade

My usual additional things to install are screen, htop, iotop, ncdu and cryptsetup. I forget if net-tools is installed by default, but I want it to have ifconfig

The container templates are under your local storage, Content -> Templates

By now I'm trying to add a public key for my local network, so I can use my OS terminal instead of the webui. After that, I will edit ssh options to only accept ssh from local network and check which ports are on use.

Another newbie questioon, I'm assuming the 1st node I've created is my real machine, and everything will be under it, right?

Quick question about zfs, could I make a snapshot, send it to another machine, recover from the snapshot on a different machine?

>scenario
Something happened to my VM, the whole thing is fucked and I cant stand it back up, but I have been sending snapshots to external storage.

Could I stand up another VM, move the zfs snapshot to the new VM and restore? Would it essentially be the VM as before in terms of data?

>I'm assuming the 1st node I've created is my real machine, and everything will be under it, right?
What do you mean by 1st node? The server itself?
But yeah the VMs and containers will be under the server.

scap this, it wouldnt work, I would have to send the snapshot from the external, back to the VM, then roll back

This is honestly fine, desu since I mostly just care about the media it will have so in the event that the VM is fucked, I should be able to roll back

So how about this back up plan?

>Actually use my H710 and use RAID 10 for the datastores (yes I am aware than RAID are not backups)

>on the VMs themselves, I'm using a NFS to send files from one VM to another for media, on the nfs server, periodically create ZFS snapshots and send them to external storage or a different drive on same server

>in the event that the NFS media files are fucked, copy the snapshot from the external store and roll back to when the files arent fucked

This is a good workflow right?

Two questions though, how do I prevent or lower chances of my data from being corrupted?

After the successful roll back, since the file system is in the same state as it was before, am I at a risk of getting corrupted again?

Attached: gina_carano_carano_vs_cyborg_weigh_ins_XwQaxR6.sized.jpg (400x600, 101K)

Sorry, I realised this user answered my question regarding corruption prevention

I was under the impression the (plex)client does the transcoding, but the client doesn't support the codec/isn't powerful enough, the server will do the transcoding if you have a plex pass
been a while since i checked this out though

comfy pic

Sorry for not replying, I've been messing around with things. I've managed to do the things I said I will do, and I was trying to create a VM, but here comes my 1st question:

>I have 5 disks, 4 HDD, 1 SSD.
>Installed proxmox on SSD
>Left some of it without format so I can put VM there
>I've searched but I can't add any HDD or expand SDD

How exactly storage works under proxmox? My idea was to put VMs into the SSD and then adding them HDD to use, but seems like I'm unable to.

I'm not really pro enough to explain how the storage works, buuuut

I'd give the whole ssd to proxmox. It creates a LVM system and allocates space to the VMs using logical volumes. There's no need for you to partition anything when you create a VM.
The place where you would add more storage is under Datacenter, in the Storage option. I'm not sure how that works however, I only ever added directories (for backups etc) using that. Go to the proxmox wiki for more info.

My data disks (where my porn and everything is) aren't in the GUI at all, I decrypt and mount them from the command line (under /srv) and have set up bind mounts to the containers, so the file server VM can access the folders.

So, if I got it right, from your explanation and what I've messed around:
>I only can play around with the disks through LVM volumes and such.
For me is a hassle but I will deal with it. I create storage, and when I create a VM I assign storate to them from where I created it, I guess. I wanted to do this to improve a little the security so I can have some machines with firewall and some others without it.

Should I save the disks actual data? I was planning to assign them to the VM but I don't know if proxmox will fuck them up.

>I only can play around with the disks through LVM volumes and such.
Well yes sort of, but there's no need to do any of that manually because proxmox just does it automagically for you. It's kind of the point of it, I think. It allows you to tell proxmox that I want this VM to have 20GB now instead of 10, and it just does it on the fly. You can have different types of storage and pick what goes where, like backups in one place and disk images in another etc.

>Should I save the disks actual data? I was planning to assign them to the VM but I don't know if proxmox will fuck them up.
This question I don't understand.

Sorry for my english.
Right now, I'm migrating, so the 4HDD I said I have, have data inside. They're mostly backed up (torrents for example, are expendable) so I don't know if the data inside will be safe if I create a new LVM group to use them.

>so I don't know if the data inside will be safe if I create a new LVM group to use them.
No it won't be safe. If you create anything on top of the old data, it will be gone.

You could mount them manually and add that directory into proxmox.

I've tried that already and I did it again because you suggested it and it doesn't appear on disks. I got to Datacenter -> Node -> Disks and it won't show up.
It appears as mounted, tho, but when you try anything it just says "No disks unused"

Good terminal for windows?

I need to SSH into my linux servers from there.

Kitty works for me, ymmv

Windows comes with ssh now and unicode shit works with powershell iirc. You can also just use an actual terminal with WSL.

putty-url

I use ConEmu + Putty.

mintty, alternatively cmd.exe w/ WSL

oh you don't even need WSL on win10, openssh is now included (with functioning ssh-agent even)

Disks under the node just shows the physical drives connected to that host. Directories don't appear there.

Here, local and local-zfs are the storage pool that proxmox created, while varmistus is a directory that actually resides on a btrfs raid1 volume that's on the two 2TB drives.

But again, unless you want to use the directory for proxmox-related activities like backing up VMs or storing disk images, I'd just do a bind mount to the VM that needs access to it.

Attached: Kuvakaappaus 2019-04-09 12-25-42.png (1112x324, 64K)

The Linux+ CompTIA cert, I’m trying to assess my knowledge and btfo my brainlet coworkers

>But again, unless you want to use the directory for proxmox-related activities like backing up VMs or storing disk images, I'd just do a bind mount to the VM that needs access to it.
I guess I will end up doing that. I went proxmox mostly for isolation, I want to have 2 open boxes so I have to learn first about VM privileges since I know LXC has unprivileged containers but for same reason they can't acess to the HDD directly. Unless (I believe) I do some LVM volume.

>build home server thingy
>suddenly want to gaymen
>too poor to buy another PC
>gaymen on the same machine
Now anything server is squeeze into one core.

Plex server does the transcoding, and it does it on free tier. The only thing paying unlocks is saving remotely for offline viewing on mobile devices, Automatic upload of photos from your phone (no more uploading to botnet googlel or applel).

Hows it working out? I use a Dell Precision workstation as my gaymen rig. Xeon, ECC ram, nothing wrong with that :D

Had some issues with the gayming VM, and I only have 4 cores and 8GB of RAM so it's pretty slow anyway.
Otherwise it's fine. IPFS takes WAY too much memory by default though and after a day of running it would prevent the VM from starting.

>Too storage intensive
this is literally what every backup program does you tard

>Too storage intensive, but better.
snapshots arent a backup you tard

>Rsyncing the VM files is a shit idea.
nigger have you never heard of things like Veeam Backup & REPLICATION or VMware Site Recovery Manager?

>Use ECC RAM in your hypervisor. Set up your VMs to use an iSCSI datastore on a NAS with ECC using ZFS (or another resilient filesystem). This is a pretty standard entry level 'enterprise' setup.
No its not you tard. Because you poorfags will still be using 10^14 URE disks.

You want to know the answer? Tape. Get you some 10^17 URE rates.

WD Reds are trash 10^14 URE disks

Enjoy disks randomly dropping off the bus during periods of high usage you tard.

Royal TS if you have a bunch of machines you need to manage

>CompTIA
>brainlet coworkers
you're the brainlet if you're going for CompTIA certs

>Enjoy disks randomly dropping off the bus during periods of high usage you tard.
That is not a thing.

This actually wouldn't even happen if you used the enclosures without breaking them open and extracting the drives first, UASP is common on USB3 drives.

>This is a good workflow right?
I myself don't actually like ZFS' performance or snapshot features - if I wanted block level backups (incremental or full), I'd use LVM2 with ext4, xfs, or even btrfs.

> After the successful roll back, since the file system is in the same state as it was before, am I at a risk of getting corrupted again?
Yes, obviously you'd have to go to the last uncorrupted state and not corrupt it this time?

What would be the best way to turn my desktop into a home media server? Like for personal plex and gaming (i guess?). Right now it has a 6700k and 16 giggly bytes or ram.

Right click media folder
Share it
Don't turn the PC off

>snapshots arent a backup you tard
What the actual fuck?

Do you even know how backups work?

>using windows
No but really though I do want to host plex, home media server, and an ftp server. Atm i just have a web server

>My usual additional things to install are screen...
lol why?

cygwin

thanks, I got an openbox router that would have been $80, for $20. thank god for those discontinued colors.

No problem. Though it's not just discontinued colours and open boxes.

The Chinese ~$25-40 routers (with massive specs like >=128MB RAM, decent chips/antenna and open sauce firmware support) just definitely pushed down prices.

zfs question: mirrors or raidz?

Discuss.

It's 41W when batteries are empty or you are at heavy load, doubt it uses more than 15W when idle

Yeah, I'm saying the same thing.

you're gay if you don't run your own iron

install pfsense/opnsense on any reasonably modern system (with aes-ni) and 2 network interfaces

just don't expose the host to the outside. set up a vpn if you need to access the PVE interface remotely.

Attached: hsg-20190402-s.jpg (3024x2016, 1.46M)

oops old pic, since been updated with a LACK coffee table

Attached: hsg-20190409-ss.jpg (3400x1526, 1.7M)

Any opinions on consumer NAS brands?
I need some 4+ bay box for my tech-illiterate father, he will only use SMB, no fancy upnp transcoding, cloud desktop or web services. Top priority is reliability and recoverability.
Are there any alternatives to HP microservers in the same price and hardware feature range? According to support Asustor is designed to run any OS from external storage but their hardware is very limited.

> Any opinions on consumer NAS brands?
They work, but for >2 drives you generally just want to avoid and DIY.

> I need some 4+ bay box for my tech-illiterate father
What stops you from assembling a simple box for him if he's not going to understand the web interface of a NAS anyhow?

It doesn't take that much time - you can use a very nice case and all that and still end up cheaper.

>reliability
BTW you're not gonna get more than the usual computer here unless you go full enterprise rack.

BUT if you DIY the NAS as mini ITX box or whatever you can replace parts. If you don't, it'll mostly be impossible to find replacements, you'll replace it as whole unit.

> recoverability
Works fine, use Linux mdadm RAID or something.