/hsg/ Home Server General

Home server thread
new thread since the old one died. do better


NAS is how most people get into this. It’s nice have a /comfy/ home for all your data. Streaming your movies/shows around the house and to friends is good feels. Repurpose an old desktop, buy a SBC, or go with cheap used enterprise gear. Lots of options and theres even a flowchart. Ask.

/hsg/ is about learning and expanding your horizons. Know all about NAS? Learn virtualization. Spun up some VMs? Learn about networking by standing up a pfsense box and configuring some vlans. Theres always more to learn and chances to grow. Think you’re godtier already? Setup openstack and report back.

>What software should I run?
install gentoo. Or whatever flavor of *nix is best for the job or most comfy for you. Emby to replace netflix, nextcloud to replace googlel, ampache to replace spotify, the list goes on and on. Look at the awesome selfhosted list and ask.

>Datahoarding ok here?
YES - you are in good company. Shuck those easystores and flash IT mode on your H310. All datahoarding talk welcome.

>Do I need a rack and all that noisey enterprise gear?
No. An old laptop or rpi can be a server if you want.

>Links
github.com/Kickball/awesome-selfhosted
old.reddit.com/r/datahoarder
labgopher.com
reddit.com/r/homelab/wiki/index
wiki.debian.org/FreedomBox/Features

Attached: laine.jpg (1600x900, 450K)

Other urls found in this thread:

linuxserver.io/our-images
filebot.net/
youtube.com/watch?v=qSpfWP-Fgjc
libreelec.tv/downloads_new/
pakronics.com.au/products/rock960-board-based-on-the-rk3399-soc-4gb-ram-version-ss102110159
twitter.com/NSFWRedditGif

Pi cluster is alive again - add docker etc. tomorrow.

Attached: Untitled-1.jpg (2186x1956, 1001K)

I struggled with that issue as well, though at first with Plex and then with Emby instead of Kodi. Ultimately the conclusion I had to accept was that shit downloaded from all kinds of places and all kinds of release groups ultimately comes in too many sorts of naming schemes and I'd never get seamless, correct identification which "just worked".

After realizing that, the conclusion is to obviously rename things in such a way that any program can identify them and place them in nicely named and organized directories. The problem with that is of course if you still want to seed torrents for some of those things, so what I did instead was make symlinks in a different folder. Everything is downloaded into my generic torrents folder and then symlinked with different names in a clean, nicely organized directory for Emby to scan. It works very well and everything is picked up seamlessly.

In order to make the symlinks easier to create, I wrote a script which creates them on a per-season basis, so I just run that script whenever I download a season of something and it creates nicely named symlinks. For shows which are currently airing I made another script and configured rtorrent to call it after it finishes downloading a torrent. That script matches the name of the downloaded file against a list of rules and then symlinks it in accordance to that rule, so airing shows in my case are automatically downloaded via RSS and then symlinked into the Emby library where they pop up automatically, ready for viewing.

Any good recommendations for de centralizing my pdf library? I see you can do hosting for ebook formats but I'm unsure of pdf.

what are you looking for? if it's search and a nice centralized ui to access them from, I'd rec ambar

I can do that with SAMBA, ultimately the ability to view over a web interface without having to download would be nice.

I'm also wondering if I can find a system that let's me scan isbn and automatically look up the information and input it into a database for my physical books.

What I ordered this week
>8 core dell server
>8x raspberry pi cluster
>48tb NAS
>router and switch
>pc and laptop
>network lab package
>hardware, cables, etc.

Very nice. Although I don't get the rpis, why bother running 8 of them instead of 2 rock64pro or 1 x86_64?

Just for fun, I want to build a network-centric application that uses distributed computing. Also to practice automation.

Calibre Web?

I really would have suggested to avoid the RPis, but since you already ordered... eh, have "fun" with the little suckers.

How to keep my Orange PI from burning down the house for being 5000 degrees while I'm away from my base? How to boot it up when I'm not there?

wake on lan/magic packets

What's bad about rpi? It's cheaper than Rock64pro.

Maybe.

Slow chip that additionally throttles horribly particularly when passively cooled, 1GB vs 4GB RAM, crappy ethernet controller that runs on a slow USB controller (so if you run storage on a RPi, you can generally expect to get less than 30MB/s), and some more flaws.

It’s not for serious development, just getting started out. But if you know more about cluster platforms I’m listening.

How do I make my qbittorent save file to my nas? It seems like I'm having permission problem. Anyone here who did something like this?

Solved it. I needed to set uid when mountin samba.

What network lab equipment did you get fampai?

Attached: keith_secola-2.jpg (322x398, 20K)

Ok so I have an old r610 that has Windows Server 2016 on it that literally only does DHCP and Plex. I want to really experiment around with it some more, but I'm kind of at a loss of what to do. Any ideas? I've already got old enterprise grade networking gear so that's already taken care of.

Attached: 1530028396945.gif (500x280, 1009K)

Maybe some of these are interesting?
linuxserver.io/our-images

Or perhaps you can automate something like backups. Personally, I'd do none of this on Windows however.

use filebot to automatically rename and organize your downloads
filebot.net/

you can run it as a daemon that checks your download directory and it'll do everything automatically

me in my house

Who here have ordered Rock64? How long does standard shipping takes?

Same server here! 2.5 in or 3.5?
I'd suggest setting up a baremetal hypervisor with either hyper-v core, proxmox, or esxi.

I have esxi running with 2 lans via pfsense and several Windows/Linux vm's for dedicated purposes: one is daily driver I RDP into from some old Pentium laptop, another runs plex, and another is used for torrenting. I also run pi hole on this lan.

On the other lan, I run a docker host for a nextcloud, bitwarden, and docuwiki.

I think I might put my nas to rest and use the r610 to host open media vault.

So yeah...you can do a lot.

I don't want to rename and move my downloads, I want to keep them with their original names for seeding. That's the whole point of using symlinks.

2.5 as I had a lot of 2.5 inch HDD's laying around when i was in the market. I messed around with esxi but stopped at some point. I'll have to look into getting some vm's spun up again.

Attached: Psg1_dmg.png (1024x1024, 348K)

what is the purpose of flashing to IT mode, I have been running h200 HBA's for years and have never noticed any diff between stock and IT flashed as far as ZFS is concerned.

sup fags

Attached: hsg.png (571x684, 23K)

what is worng with the font
also shitty performance/watt CPU and why have 16gb of RAM if you only use 460mb?

>why have 16gb of RAM if you only use 460mb?
It is better to have it and not need it than it is to need it and not have it.

Attached: Walken-Cowbell.jpg (390x292, 21K)

uhm not at such high levels, you are wasting electricity and fucking with RAM prices, i could get behind you having 4gb and using 400mb but having 16gb is just selfish.

>what is worng with the font
no subpixel antialiasing, light hinting

WHY IS THE HOME SERVER GENERAL DYING NOOO DONT LET THIS HAPPEN, anyways i put together a low power pc to host a website but ended up with no money to pay for the domain so now i host a Minecraft server where i play with my friend, pretty comfy desu

thanks thats not a bad idea, i might go for a symlink scheme too

Any ideas for scanning ISBN, automatically scraping the info into a databases and then viewing it?

anyways, the low power pc specs are:
MOBO: Mini ITX asrock, 1gbit eth, SODIMM

RAM, 1 PCI port, soldered CPU, p good for my use case.

CPU: Celeron J3355, awesome and just 10w TDP, so it can be passive cooled to have a 0db server, can host up to 20 players in Spigot, maybe a little more.

RAM: 4GB kinda overkill for that cpu, counting the OS RAM usage i could hos up to 30 players with this ammount, but the CPU is a limiting factor, still i built this PC with the plan to host a website and that uses way more ram and less CPU power.

HDD: some pos 320Gb samsung HDD i took out of an old PC, want to upgrade to a 120gb SSD + 2tb HDD and take the PSU fan out for an almost silent PC.

meant to be together:
MOBO: Mini ITX asrock, 1gbit eth, SODIMM
RAM, 1 PCI port, soldered CPU, p good for my use case.*

His system is an OpenBSD with 4 hours uptime. That is typical RAM usage for that OS. 108 days later it may have over half of that cached & buffered depending on what is being done. At 1.9GGHz, user is not exactly ripping a hole in the ozone layer... The A88XM-E is a DDR3 board, and DDR3 is easily scavenged anyway. The host may have been built as a mid to upper level gaming PC to begin with.

I am posting from:
'SOPine-A64-Rev C / SOPine Baseboard Model A Rev B
Installed: Wed Nov 21 01:35:50 2018
XXXXXXXX 4.20.0-1-ARCH aarch64 GNU/Linux
-----memory----------
free buff cache
1162380 37368 7002840
because technically, I feel the same way you do.

i get what you say but still im all for "use what you need" my server uses 3.2/3.6 gb of ram so i have 4gb uwu

it's so you can use JBOD rather than a bunch of raid0's

I have 96GB of ram on a r710

4x rock64pro would be a lot better and i hope the NAS is 24tb x2 in raid 1 or 96tb / 2 in raid 1.

how do you do this? i would like to have a pi cluster but there isnt a lot of information around, which programs? tutorials? pls sir help me

cool(? i never said its bad to have a lot of ram but its selfish to have a lot and not use it.

>selfish to have a lot and not use it
i assume im entitled to borrow your car whenever i please as long as you arent currently using it then? fuck off faggot.

what, i never said i could use your ram, but yeah producing a car or letting a car running 24/7 with no purpose is selfish.

Attached: flat,1000x1000,075,f.u1.jpg (1000x717, 123K)

>literally putting politics in a conversation about moral values

im an anarco capitalist btw

If I have a media PC running GNU/Linux, that is directly connected to speakers and to my LAN, what music program can I use to make that PC play music while being controlled by android phone or another PC in the LAN?

I read OP's post, and I don't think ampache helps me fulfill this task...

try hosting a local website in the server with your music with music controls and when you want to change music or control volume just enter the website and do it.

poke around a little harder, there is actually quite a bit of info about pi clustering - search for stuff like brambles, dockers swarm etc.

youtube.com/watch?v=qSpfWP-Fgjc

Attached: Untitled-1.jpg (977x996, 345K)

Or you know, run pymedusa, check anime on the show's settings page, use hardlinks and if needed use jackett if you have many sources.

You still have to babysit it to make sure it's matching things correctly.
Filebot has a symlinks option

>I really would have suggested to avoid the RPis, but since you already ordered... eh, have "fun" with the little suckers.
Sentences like this one are the reason why you don't have any friends.

>>I really would have suggested to avoid the RPis, but since you already ordered... eh, have "fun" with the little suckers.
>Sentences like this one are the reason why you don't have any friends.
"You didn't ask me, but I know better than you, so have a preemptive 'I told you so' anyway!"

anything new recently in terms of distributed storage or is cephs/gluster still the way to go?

Attached: Untitled-2.jpg (1080x1920, 293K)

good morning /hsg/
just installed a fan and a cheap lian li pt-fn07 fan controller to cool my disks because i noticed they were going slightly warmer than my comfort during snapraid diff and sync (about 50-56C) hopefully it will be cooler next time
also company bonus and tax returns are coming soon so i will be able to afford more drives

Attached: 2019-04-14-121647_1342x669_scrot.png (593x267, 33K)

What is a good, hopefully inexpensive, option for a DIY Kodi media player box?

I see a lot of android based ones but I want to do a proper x86_64 Linux - should I go mini-ATX or try some kind of SoC board (Intel Atom or something)?

I'd like hardware acceleration for media codecs, I have an ancient laptop I was thinking of using but it only has 128mb VRAM and can't play modern videos without a lot of stuttering.

i would honestly suggest you look at libreelec supported devices
libreelec.tv/downloads_new/
cpu:s like rockchip rk3399/rk3328/etc are really good for media and will likely both perform equally and draw much less power than an atom based system, rk3399 for example has a native 10-bit pipeline so 10-bit anime is fully supported

I like the idea of pi clusters, docker, kubernetes, but I have no clue what I'd do with that. What do you use it for?

>i hope the NAS is 24tb x2 in raid 1 or 96tb / 2 in raid 1
What would be the point of that? RAID 1 is still not a backup and RAID 6 is cheaper for practically the same result.

I use tvnamer.

I really wouldn't do that because qbittorrent will throw a bunch of errors whenever the share is not mounted and you will have to force recheck everything. Just install qbittorrent-nox on your NAS and use the web-UI.

I'll look around at rockchip options, this pakronics.com.au/products/rock960-board-based-on-the-rk3399-soc-4gb-ram-version-ss102110159 seems reasonable but it's out of stock atm.

Might do a comfy homeserver.
What's a good case that can accomodate a lot of hdds inside?

im using a define r5
fractal design has good accessories for adding more hdd racks

Ceph is still the biggest thing.

Gluster... no thanks. I'd prefer Lizard/Moose/Seaweed FS and such even if I wanted sonething other than Ceph.

I don't know what you'd actually use pi clusters for.

But docker/kubernetes is mostly used to make running software (many times?) easier.

If you want x86_64 and easy get a Chinese HTCP box that runs Windows for $100-150 off some store, they generally have pentium atom and such.

yeah, gluster is not playing nice at all - no idea how it wants dns handled, seems very picky

I'll have a squiz at ceph after dinner

Attached: Untitled-3.jpg (878x1116, 505K)

I'm not here to make friends, but to discuss technology. That includes people pointing out when something sucks.

Feel free to return to your social network circle jerk.

Ceph is very powerful, but you'll quite possibly have trouble with it being complex and not yet very mature/stable.

Even if you only use the more basic features and orchestrate (rook.io), you likely still have a whole lot of issues.

aye, from what I was reading it seems more suited for use with kubernates than swarm - i'll give it a look see all the same.

As far as I can tell it works the same with docker swarm and on bare metal.

It's probably just that many ceph deployments are larger and preferred to use k8s or have admins that are more familiar with k8s. Docker swarm/compose was pretty damn useless for handling a lot of servers in the past.

any ddr3 ecc ram recommendations to buy from aliexpress? all of them have some bad reviews so decided to ask here

Why would you buy from ali instead of ebay when the latter is cheaper?

god aye, swarm was terrible at the beginning, good bit better now with proper service handling and stack deployment.

I'm almost done configuring Gladys and honestly, it's not as bad as i expected. Now i just need to buy a few chinkshit devices to use its IoT features.

Are you guys brainlette/noob friendly here?
Ordered 3 2 TB drives for 70 bucks that'll be here tomorrow. I need guidance uwu

Attached: Screenshot_2019-04-11_19-19-58.png (1280x1024, 297K)

What guidance do you need?

Generally you just hook them up, run mdadm+LVM2 or snapraid or whatever else you want to arrange the storage, and then proceed to put your data on the drives / host stuff off the drives as usual.

Hello!

I'm interested in getting a NAS for home use. I mostly torrent files (run several instances of bittorent) and stream on 2-3 devices thru Plex locally. My current desktop does the job well, but I find it to be power inefficient.

That said I'm very much interested in the many options on the market. I do not have knowledge of OS/ Firmware/ FileSistems so I had a look at different pre-built NAS systems, many beign around 2000 US Dollars, which I found surprising to say the least.

The steep price drove me away from the pre-built option and came here to seek advice on how to approach this situation before I dwell in to deaper reading. My budget being anywhere from 500 to 1500, Jow Forums could you suggest what I should get or what should I learn before making a purchase?

The networking side I'm kinda stupid with, its like 5 windows PC's on the network, so I looked into Samba and that seems easy enough. The existing issue is that there's no network between existing computers (I've tried everything) I'm fairly certain it comes from using routers as switches and can be fixed by putting network switches in front of the routers.

Also I've never really done a raid set up before, let alone on a *nix system. Is setting it up in the RAID BIOS good enough or do I need to do some work on the OS side as well?

> The existing issue is that there's no network between existing computers (I've tried everything)
Odd. You mean you can't even ping ip addresses of other machines and so on when pinging is generally allowed?

If so, sure, you might want to fix your AP/routers/switch settings. Maybe it's all intentionally isolated (routing only between the internet and the individual machine) by configuration on these.

> Also I've never really done a raid set up before, let alone on a *nix system. Is setting it up in the RAID BIOS good enough or do I need to do some work on the OS side as well?
Don't use the BIOS at all, these are often terrible implementations.

You do it all on the OS side. mdadm is really easy. Basically like:
mdadm --create --verbose /dev/md0 --level=5 /dev/sdx /dev/sdy1 /dev/sdz
(You can use whole drives or partitions on them - I mixed them, you probably just want the whole drives)

It's my family and wife's computers that are all windows PC's.
>modem/wifi in main house(2 windows PC's)
>direct from modem to secondary house and splits at router 1(3 windows PC's, 1 *nix)
>secondary houses directly connects to garage splits Ethernet at router 2 (server location and a place where a few windows PC's get set up)

I'm not entirely convinced they're not able to see each other, when I was fucking around with trying to fix this a few months back I could see in the WIN10 home group my wifes login picture on my wind10 system. We're connected via router 2, but no file share, no PC on the network it's self. All sharing options have been fixed and configured correctly too.

That's why I'm going to try the switch idea.

I have a PCI sata raid card, I was referring to the bios on that (it gives me a prompt to enter set up, in setup I have a host of options from format to set up.)

> That's why I'm going to try the switch idea.
Sure. I guess if you're not good with network diagnostics, just putting a switch in between is cheap and probably easy.

> I have a PCI sata raid card, I was referring to the bios on that (it gives me a prompt to enter set up, in setup I have a host of options from format to set up.)
Yes, don't use that. Use it as SATA controller.

Only very few cards are even "okay" at what they do and even then it won't allow you to use other ports on your mainboard and so on, nor will you know that if it breaks you can easily get a replacement. Mdadm is pretty much always better.

Is the sauce of the pic Blame? I sweartogod I've seen that somewhere

Serial Experiments Lain - that pic is kinda Blame-esque tho


Fucking systemd, After= means mount it AFTER the network is live!

Attached: Untitled-1.jpg (1920x1080, 939K)

Yeah, that's a good idea too.

the font is fine. luxi mono

the CPU is also fine but I could probably increase the clock speed a bit. more concerned with low power than muh fancy graphics

the RAM is also fine and cost like $10 more than 8 GB. I never need to upgrade it. jumps +1GB or so when running firefox but meh

the host was specifically built as a low energy footprint machine that packs a reasonable punch. I was originally gonna build it into a sun sparc case with ~200W power supply but I nixed that idea and bought a case with 10 HDD bays.

currently has 1 SSD for the OS, and 2x 4 TB drives for home and backup, respectively. adding a 3rd soon to store genome sequences

it's only 4h uptime cuz I'm at home and was doing some work/upgrades that required reboots. usually I'll reboot it every 3 months minimum

this machine mostly does networked backups and streams music FYI. all my public facing stuff (web, email,dav, etc) is hosted on a VPS in scandinavia cuz
>muh privacy

also comcast are faggots and they block inbound ports 25/80 for unknown reasons (probably dipshit customers in botnets or wanting to charge $300/mo for "business" class with the same shit uplink)

Why would someone pick FreeNAS over running Ubuntu/Debian/etc with mdadm/btrfs/etc configuration or vice versa? Doing research

Attached: forklift-accident.jpg (4608x2592, 1.29M)

I use OMV because it just werks.

FreeBSD has better ZFS implementation

Okay, so when it detects all the drives just ignore it and don't make an array in the card's BIOs?

Here's a map of my network. idk if it helps at all

Attached: Untitled.png (965x745, 29K)

>Okay, so when it detects all the drives just ignore it and don't make an array in the card's BIOs?
Yes. Just let them be individual SATA drives.

> Here's a map of my network. idk if it helps at all
Kinda nice. But yea, the obvious thing to try would be to change settings or throw your own firmware on the orange AP routers. First try to make it so that devices on one router can ping each other by IP, then make sure that doing the same with hostnames and full samba works, then get the two separate routers into the same network including subnet (probably run only one of these as dhcp server/forwarder for both?) until all your machines are connecting normally in your intranet.

buy a new ryzen platfrom in 1-2 months once it comes out, it will handle torrenting and plex transcoding just fine and you'll have resources for more VMs and containers to play with

weirdly enough FreeBSD will be switching to the ZOL implementation soon though

Thanks, I'll be sure to set the drives in mdadm and not fiddle with the card crap.

Would it be beneficial to set each computers IP statically or would it cause more head aches? on the AP routers, what do you mean when you say changing the firmware? Like an OEM update or like a libreboot thing, but for routers? Would I be just logging into the router and playing with the settings to get them to look into the same network/subnet? I'm sorry if my questions are retard tier, networking is my weakest category, sorta why I wanted a project like this. You've been a massive whitepill when I was feeling a little over my head thanks again!

Attached: 1550705979873.png (303x566, 163K)

You could do this on an Odroid XU4Q/HC1/HC2/N2 or Rock64/Rock64Pro if you don't need many drives on that machine.

Of course no realtime video transcoding to fancy formats on these, but they will stream. And usually it's actually better (more power efficient, nicer) just having a few playback devices that can play back current audio/video rather than introducing MONSTER machines that can realtime transcode in good quality... which also won't be power efficient, BTW.