Is hosting an image board really that expensive? It's just text and images or a webms

Is hosting an image board really that expensive? It's just text and images or a webms.

Attached: 1514637496490.jpg (900x900, 58K)

Other urls found in this thread:

blog.Jow
github.com/eggbertx/gochan/
twitter.com/SFWRedditVideos

Not really, the administration always sheds crocodile tears and cooks the books because the alternative involves convincing sperglord nerds that it's OK to make moot or the current admin rich with your shitposts and crippling Jow Forums addiction.

Behind the scenes the website is making tons of money.

blog.Jow Forums.org/post/152411853897/Jow Forums-bandwidth-was-3pb-last-month-Jow Forums-doesnt

>t. Ban images and webm's

frog.jpg

onions

does Jow Forums use their own servers? or do they use a datacenter sever?

No. Imgur ran without proper funding for years and they have to deal with long term data retention, as well as basically serving up content for all of reddit (which has WAY more traffic than here)
Jow Forums was run off of a couple mac minis for the longest fucking time, and even an increase of traffic ten-fold wouldn't require much more than two or three decent servers.

Literally false, that statistic includes Cloudflare precaching and buffering. $0.03/GB is also ludicrously expensive, nobody is paying that for hosting services.
It's also laughable to imply that a website that has no data permanence and has a limit of 4MB per possible upload file (except on /hr/ I guess) could even remotely approach 3 *petabytes*. Do you understand how astronomically large that is? The only people to actually approach that amount are sites like Youtube and Netflix, who only started cracking that number a few years ago.

Let me put this into perspective:
3 Petabytes is 22890000000 MB, approximately. Which, at a 4MB limit per file, comes out to 5,722,500,000.
Let's round that to 5,722,000,000 to account for text posts and page serving, maybe thumbnails. I'm being generous with that, too.
That's nearly 6 BILLION file downloads in the span of a month to achieve 3PB. Jow Forums is popular, it's not THAT popular.

probably aws

it always depends on the scale
a site like Jow Forums that's used by millions every day is always going to be expensive

>Jow Forums was run off of a couple mac minis for the longest fucking time, and even an increase of traffic ten-fold wouldn't require much more than two or three decent servers.
which one is it?
do they still use mac mini servers or do the use datacenter severs
>a site like Jow Forums that's used by millions every day is always going to be expensive

it's mostly lurkers that find about Jow Forums for the first time.

>do they still use mac mini servers or do the use datacenter severs
They run everything through Cloudflare now, so it's fairly likely they're just relying on them to be a CDN instead of bothering to host their own equipment.

>They run everything through Cloudflare now, so it's fairly likely they're just relying on them to be a CDN instead of bothering to host their own equipment.

so they save bandwidth by doing that?

they get convenience since you can scale and maintain a site way easier in a CDN rather than hosting completely on your own

isn't cloud fare for blocking ddos attacks or some shit. Don't they redirect the ddos traffic somewhere else so Jow Forums doesn't get it?

In a sense. Cloudflare does a lot of really neat caching and rerouting techniques. It doesn't make any sense to have to query the host every single time a user requests a file, especially when you don't have that many of them. The bandwidth is still used because you're still sending data around, but think of how Netflix works. They have lots of data centers so they get to use more of the cheap bandwidth (local loops) and less of the expensive bandwidth (cross-country BGP lines).

Cloudflare is notable for their robust anti-DDOS capabilities, though their techniques aren't particularly new. Blackhole rerouting, script injection checks (Ever seen that "CHECKING WITH CLOUDFLARE, PLEASE WAIT UP TO 5 SECONDS" thing on some webpages?), and I can't recall the exact name for it but there's a network routing technique that let's you reuse the same IP to point to multiple different servers. Because networking is a "shortest path first" 99.999999% of the time, it's relatively stable to have multiple datacenters hosting the same IP address, so if you're in LA and try to go to Jow Forums you just get directed to the closest physical data center (in the networks eyes anyway), and if you're in NYC and go to Jow Forums you get the same content but at a data center on the east coast instead.

Is their way of doing this mostly software based? Do they have special software that let them do this kind of shit? Why can't i do this for my own website instead relying on some gay company?

>Jow Forums was run off of a couple mac minis for the longest fucking time, and even an increase of traffic ten-fold wouldn't require much more than two or three decent servers.
Until Jow Forums gets into the news and normalfags everywhere flood here that even those mac minis wouldn't be able to handle it

Because you don't have multiple datacenters located around the world with peering agreements to tier 1 bandwidth providers.

>Is their way of doing this mostly software based?
In the sense that networking is done through software configurations and memcaching takes place in stateless memory banks, yes. It's not like they just run some program on a generic server somewhere though.

>Do they have special software that let them do this kind of shit?
Short answer: Yes and no. Same answer as above, really.

>Why can't i do this for my own website
Get multiple datacenters with terabit data pipes and spread them around the world and you can.

That already happened dummy. Besides, Jow Forums only gets about 22 million visitors per month and the average visit is only a couple minutes. If you threw a few more mac minis at it they'd still have enough processing power to cope with it.
The main reason Jow Forums uses cloudflare (and this change happened well before gookmoot took over) is because it was getting DDoS'd every other fuckin day.

That being said you can perform your own DDoS mitigation if you want, blackhole routing can be done on just about anything and is pretty effective. The main concern there becomes processing power over bandwidth.

>The main concern there becomes processing power
how much ram would i need on my server?

Processing power, not memory. Blackhole routing effectively decides that certain incoming traffic is bad and routes it to /dev/null (discards it in a very inexpensive way).

J U S T
U
S
T

Attached: a.png (178x141, 11K)

How much power would you need to blackhole a bunch of shitposts

Attached: soyboys.jpg (853x480, 45K)

Does this still work well for sites like Jow Forums which experience a lot of bi-direction traffic?

Attached: soylent.jpg (853x480, 65K)

Fuck off

How many men do you need to storm a castle?
You're asking some really vague fucking questions here.

Yes, for the same reason that Facebook and Google can leverage the same techniques. Distributed systems because really, really effective when you scale them up with enough horsepower to drive them. The internet is really, really, REALLY ball-shockingly fast now and computers are even fucking faster than you ever thought they were, so replication of something as trivial as a string of text and at most a 4MB file is nothing.

You're talking about processing power. How much processing power would you need to make this blackhole routing effective against a DDoS attack?

>t. onionsboy

Yes, with the shitty performance on this site I am sure hiroshimoot uses sewer servers.

Fuck off

>loving onions this much

How many lanes do you need on a freeway to handle traffic?
There's no definitive answer to this without providing more information. How big is the DDoS? How many shitposts a second? What kind of DDoS is it? Ping of death? LOIC? Memcache attacks? If you want to survive the recent multi-terabyte-per-second attacks against github, then you better literally be cloudflare. If you want to survive some middling 1gbps attack, then any decent modern xeon should be able to handle it.

this mother fucker is on a rollhow do you know so much web dev shit

state your opinion:

I actually pursued an education instead of complaining about my uni giving me codemonkey classes. I'm in the middle of my Masters in Distributed Systems, and networks/the internet in general is just a gigantic distributed system. Also I got my associates in network infrastructure and minored my bachelors with an emphasis in cybersecurity.

Mods are fags.

>blog.Jow Forums.org
>https certificte for *.tumblr.com
This always kills me

kek. Never noticed that.

Attached: 2018-04-21-171350_1920x1080_scrot.png (561x394, 29K)

>I actually pursued an education instead of complaining about my uni giving me codemonkey classes. I'm in the middle of my Masters in Distributed Systems, and networks/the internet in general is just a gigantic distributed system. Also I got my associates in network infrastructure and minored my bachelors with an emphasis in cybersecurity.

motherfucker is on a rooollllllllllllllll
u work for amazon or some sheit?

faggot might as well
about:config

javascript.enabled false

onions

lol no. Don't think I'd want to either, maintaining AWS as an engineer seems like a fucking nightmare.

Except I want javascript selectively. Normally I have CSS, Images, and media on generic whitelist, then selectively enable scripts as I need them.
But ever since gookmoot pulled his ukranian datamining bullshit it's on full blacklist.

if you are looking for performant imageboard software, here's is one I've found to be good enough (written in Go): github.com/eggbertx/gochan/ though it's still under development (doesn't seem to be responsive, it lacks some functionality in the admin side, etc)

m00t at some point was using his own servers

Attached: 1494950298236.jpg (646x754, 171K)

as other anons have pointed out its not the hosting part that is the problem, but the serving part. Jow Forums has a whole lot of requests per second to serve, many of which is said pictures and videos.
the other problem, in my opinion way harder to "solve", is making sure you are not hosting illegal content like child pornography and such. and yes, sadly, there are people, even here, that have hold and will upload that shit

MYSQL CONNECTION ERROR

>yes, sadly, there are people, even here, that have hold and will upload that shit
lmao get the fuck out of here if you ain't postin sinks

So this server in the picture used to be our collective brain?

yes. This was taken when m00t decided to unplug Jow Forums to troll everyone

>sinks
wew thats an old one

>lmao get the fuck out of here if you ain't postin sinks
Haven't seen sinks in years

I've officially wasted 12 years of my life here.
There is no escape.

yes there is.
Just block Jow Forums in your hosts file

I vaguely remember moot saying he had a tower server at one point. This would have been around the time he moved to California. 2007, I think?

>Jow Forums is popular, it's not THAT popular.
/b/ and Jow Forums are culprits.

not the guy u replied to.

cool. i want to do my masters in distributed systems and security (finland) but my GPA is too low because i didn't care enough about the boring classes. i started caring in school right about when virtualization and cloud computing classes came along. so much coding and cisco that unix/linux courses and general server/distrubuted systems courses were nonexistent.

now i work at an AWS shop and i have a couple certs under my belt as well.

The admin I work under at our obscure board pays $200/year to keep the servers running.

Actually not, it is profitable.
t. Hiroyuki

Yeah, but at the cost of having to pay Cloudfare for this usage

Not same dick-sucking fag, but what are your thoughts on IPFS being able to replace CDN's generally?