/ipfs/ - interplanetary file system

but imagine
you could have;
- an episode link
- a link to a folder with the episode and the rest of it's season
- a link to a folder with the episode and the rest of the show
- a folder with the episode, the show, and god knows what else
and no matter how you get the episode, the same peers are seeding it in the same way from the same address

This. In fact, you have to really like botnets to use private trackers because you can't access them if they don't have full trackable information about you, basically.
Moreover, private tracker mentality means that rare, low-demand files disappear, which is not a problem with ipfs.

And for generic web it has even more advantages, such as being able to very efficiently compress and distribute the entire collection of text being shared because of this same mechanism.

Interesting, also a location hidden server so there is no DMCA submission form

okay time to start making this imageboard

Redpill me on pubsub, how does it compare to zeronet

ok, and what happens when someone posts cp?

You blacklist it

Can IPFS handle redundancy? Say I have 100 servers in a private IPFS cluster, I'd like every file to be on at least 2 servers automagically.

>still no tor or i2p support
>IPNS still shit
>IPFS.js still alpha

How many more years must it be, OP? IPFS is memeware at this point.

Is consensus really necessary? A post is just a UID and a text body containing other UIDs. Let the UID be the hash of the post, and accept that posts not saved/spread will be lost. Moderation could be approximated by just letting users 'drop' posts, preventing their spread. 'Moderators' would essentially be just standard gateways. Works for CP too - gateways use contentID or w/e to drop CP since IPFS isn't encrypted, what isn't caught is reported to them. Bonus is that it helps preserve IPFS's good image.

Sage for doubleposting.
Yes, but not automatically AFAIK. Each node can pin resources so they aren't GC'd. So you'd need a seperate algorithm to determine which nodes should pin.

What a shame.

>gateways
>gateways
To what? Last I checked (and I might be wrong, so do correct me if so) all nodes were essentially considered equal, with a couple of exceptions like the hardcoded default bootstrap nodes.
If my memory servers right you'd only involve a gateway if you were connecting from outside the IPFS cluster, e.g. over HTTP, but not for normal usage.

Daily reminder that IPFS shouldn't be used if you're privacy aware. I would love to run IPFS, but everyone can see all your activity.
Until they finaly add I2P or Tor support I won't be using it.

Attached: Surveillance_cameras.jpg (500x579, 251K)

IPFS itself can't, and it makes some sense because who would set up that centralized rule of "2 copies", and who is to say that the copied instances don't lie about havibg a copy or conspire to all go offline once the original does?

It makes more sense in a conventional DFS like Ceph, Moose. Or Syncthing. You might want to use somesuch.

You're right. My suggestion was to repurpose the existing gateway logic, either directly via an HTTP client, or indirectly by forking the relevent code. There's no point reinventing that wheel when a perfectly good implementation exists, especially one that can share blacklist data trivially.

I'm gonna give a hard no just because the whole point of IPFS is that you can literally add whatever the fuck you want. This goes against the whole decentralized wild west idea that IPFS started with

op here
its broken
dont use this shit

bump :^)

We might be misunderstanding eachother. user I initially replied to (I think) suggested elsewhere to just do nntp-chan but in IPFS. As I understand it, nntp-chan is federated such that each node decides which posts to forward to its clients. Thus it is decentralised as you have a network of servers with which clients interact and each server decides for itself what content to host. Additionally, my suggestion isn't to filter post submissions, but to filter post downloads. Anyone/node can submit since a post is just an IPFS object, but the gateway system allows for a federated moderation system to handle CP since IPFS isn't 'pushy'. Nothing stops you from just running your own 'gateway' with no filters. The only real issue with this scheme is that broadcasting the existence of your posts is a pain, but that's arguably an IPFS issue.

ipns over pubsub is actually really fucking good, but it's still considered an experimental feature and both the server and client need it implemented for it to work.
Consensus is necessary because you have issues like simultaneous posting and maliciously crafted messages. Imagine a site where messages are ordered by time (newest first). All someone has to do to kill the site is to spam messages from the year 9999.
Post ordering also matters. Without consistency (which would need to be via consensus if we want a distributed arch), doesn't mean anything because each user sees the post number 1234 as being a different one.
Letting users drop post would be the same as subscribing to no channel, which would of course be a valid method (it's necessary to defeat censorship, and should probably be the default).
However, manually dealing with thousands of posts a day, where you can expect shittons of spam, let alone cp, is not viable long-term. That's why it is needed to be able to subscribe to moderation 'streams' (or vice-versa: streams that only give parts of the posts).
ipfs cannot take down anything because there are no centralization (if there was, there would be no point in ipfs). Moderators need to be per-site for obvious reasons (i.e. we're back to the sorry state of the current web otherwise, but on steroid since it's even more centralized).
There are no 'standard gateways' by default in ipfs and doing a pure nntp-like system would not be optimal (often, all nntp nodes go down simultaneously).

IPFS supports bitswap, so every node's default cache contains random parts of the contents of the entire network. I don't know if it's still an experimental feature. Yes, this is an extremely important point. Anyone with more info should chime in.

The problem is the idea that 'anyone can submit'. How do you define this concept consistently and how do people discover your post? Pubsub can fix that but then you need to fix your channel communication layout. Not a problem, just something to think about.

why not both?

I don't think you'd need gateways per se, just have the "chan" essentially consist of metadata including data structures of posts and black lists. I don't know how pubsub works but presumably after making a post, you'd add a link to it to the central structure, and "publish" so everyone who's subscribed can get the new structure that has a link to your post. But if your post gets blacklisted, other clients won't download it.

Blacklists can easily be centralised, much like current host blacklists are. You subscribe to maintainers of blacklists, and the operators are basically the designated "mods" or "janitors". If they start going rogue or censoring, you're welcome to unsubscribe, and find an alternative, or even start your own. That does mean some people will be un-blacklisting CP, but it's their own problem if they wanna get v& since everyone else will hopefully have it blacklisted. Unless they do it over i2p/tor, but then that's a different story. Point is normal users won't have to worry about CP, and you can already use tor/i2p for that so this chan won't really facilitate anything in that regard.

Post lists, though, need to be decentralised, but have some sort of agreement, so as to enable actual discussions. Otherwise, on one extreme, it'll basically be the world's most convoluted traditional centralised chan, and on the other extreme it'll just be a collection of spontaneously forming ad-hoc p2p messaging sessions with no wider "forum" in the traditional sense of the word. And this isn't a problem I can solve on the fly at half past 10 pm, and is I suspect the main difficulty in getting a chan like that to work.

>let's make imageboard over ipfs
it already exists
0chan on zeronet

Wow epic

You need an ipfs link to address content, you can't just magic your way into it, nor can you reverse-search (i.e. search all files that refer to a chunk) because not only does the concept not really make sense in IPFS, but dedup is at the chunk level (so if someone were to quote an article, that article would share the exact chunk as the post's for that part of the text), among other issues.

Pub sub means the following: you register on a channel with a specific name and advertise that to peers, who relay that information. This is sub.
Someone posts a message to the channel with the same name. Everyone registered to the channel receives the message, as relayed by peers (peers who are subscribed read the message and then pass it on, those that aren't pass it on and ignore the content).
This process is currently extremely fast.
With pub sub, if you had a Jow Forums channel, a poster could send a post to that channel, and anyone subscribed to Jow Forums would receive the post and do whatever they want with it (e.g. update the html content displayed in the browser to add this post).
The centralized blacklist you describe can easily be handled through this same mechanism.

shill pls.
zeronet is not ipfs, and the js nightmare that it is is an insult. Not to mention the ridiculously spywary design where anyone can track anyone because you need a unique id to post anything and can't create a new one, or go back to an old one, easily (including artificial delays between key generations measured in hours).
Being able to change the content of your post at any time is also retarded, since it allows the likes of changing the timestamp, inserting arbitrary illegal content, etc. at any time.

>zerochan
nice botnet you have out there

the question is if pubsub scales well
now I don't think there are many pubsub messages so system works fast
what if there will be much more of them

Good question. No idea either. Normally it should scale even better than normal content delivery and I think the bottleneck becomes latency, but with enough nodes to actually have relevant volume I would think latency would become trivial.
Is there an easy way to perform such tests?

>ipns
I'll look into it again tomorrow; it'd be nice to be wrong.
>consensus on time
Is a timestamp necessary? If you want chronological order, have each post reference the latest post they can find. If post UIDs are their hashes, it is impossible to forge 'future' posts. I admit I can't think of a solution to faking posts from the past without muh blockchain or a similar authority. Practically, you can introduce two notions of time - time of discovery to client and ordering in thread - and prioritise the post showings from there. After all, the whole point of bumping is to highlight activity/unread stuff, and for a healthy network, new messages satisfy that. Then again, it's starting to sound more like a BBS than an imageboard.
>
If post UIDs are the post body's hash/posts are IPFS objects, this isn't an issue. It would even deduplicate some shitposts.
>subscribe
Opposite philosophies. I'm advocating for an opt-out model, where by default all posts that you can receive are displayed. Gateways are just delegating that responsibility of opting out. The downside to an opt-out strategy is of course you get every channel of an unbounded number of channels. Even if you used filters to out-out of channels that aren't in your whitelist, congestion/load may be an issue. How many objects can IPFS reasonably handle anyway? I don't remember getting a solid answer on this, and if every post is an object, it may end up like bitcoin after those gambling services spammed the chain.
>moderation streams
I think it's covered in my gateway suggestion - communities manage shared block lists on the 'exit-nodes'/gateways that they use to get posts. Moderation is then made just as hard as it regularly is.

Part 2
>take down
That's the beauty of it. Think of it like objects in a GC'd runtime. If no-one has a reference to it, it practically doesn't exist and is GC'd. Of course, someone can have a copy, but that's fine; you can't censor data on IPFS, but you can censor knowledge /of/ data.
>per-site moderators
I'm lost. Isn't this covered by sharing block lists. And arguably isn't even a problem given everyone has a different idea of what's legitimate? IPFS being a neutral platform in that regard is good.
>standard gateways
That's fine. I'm suggesting something closer to mastodon in that gateways are just special nodes you can use.
>not optimal
I'm an idiot. Please explain this for me. Why would all nodes go down? Even if they did, IPFS would have cached a lot of posts for the interim so you could still browse offline.

So what's the best way to do it?
- Computer proof-of-work blockchain, human pow is not possible because of AI
- No currency involved
- Javascript trustless node
- Both block and post require mining. The difficulty of the block is reduced for each new post included. Miners who actively seek to include posts will find new block faster than a hypothetical attacker spamming empty blocks. (The difficulty of each post is fixed such as block_difficulty + max_posts_count * post_difficulty = min_difficulty)
- Moderator is an IPNS link to a list of blacklisted posts. It can also store metadata and links to other IPNS to define a moderation graph.

I've made a simple chat using js pubsub: /ipfs/QmamLZdKm2XQ4seU2GYBh5Wvg2Xfu52nskXE3BMjDkg8oe/

>js-infested crap
Should I do it in go then? Being able to run in the browser sandbox is more convenient, webasm could be used if performance is needed, there are no official go-ipfs-core api and I don't think that many people are willing to run miners on their server all day everyday to gain nothing from it (assuming they are not compromised). There is the possibility to have client slaves mining on behalf of the server but then there is trust.

Attached: a.png (1042x521, 62K)

Don't run any logic client-side in a browser, period. If your code downright doesn't work without it, your shit is unusable. If it uses js for convenience but doesn't need it for functions (e.g. auto-update is js), it's OK but should not be the default.

It isn't a client but a full node with webrtc p2p.
github.com/libp2p/js-libp2p-webrtc-star

It says it shouldn't be used for production and as far as I've tried it crashes pretty often. It might be a bad idea after all.

Don't go the blockchain route, it's not needed for something like this. Forums/chans are better suited for eventual consistency.
I've thought about doing a peer-to-peer chan/forum in the past, so here's my design idea, do with it what you want, I don't have time to implement it anyways:
>each node in the network stores the complete ledger
>each user has their own "block" inside the ledger that they are in full control over, but append-only as other nodes wont delete your data once published, even if the publishing node deletes or modifies it
>in this block the user can add his own threads and posts
>each thread and post are cryptographically signed and include a proof-of-work to discourage spamming, this could be scaled based on the need similar to bitcoin, though it should be a simple equals operation to verify the POW
>posts reference threads inside the ledger
>other nodes can download the ledger from *any* user in the network and have the ledger (similar to blockchain)
>because the ledger is not a blockchain, some malicious users could "omit" data (but not forge), therefore it's best to download the ledgers of n nodes when starting up, the more the merrier.
>I haven't thought much about how you could reduce data transfers even further, but there must be a way to simply score nodes based on ledger completeness, or optimize the check on which data you actually need from the other nodes.
cont.

All of that shit applies to Jow Forums too, my dude

>peer discovery and new posts would be done in a pub-sub fashion
>assuming there is at least 1 non-malicious node in the network, the network can be considered functional, as in the worst case you could just download the ledger from all nodes in the network and have a complete ledger.
>the program should support anonymity out of the box, either through Tor or I2P or both. peer-to-peer should always be anonymous imo. as anyone can track your activity otherwise and is a privacy nightmare (e.g: bittorrent, IPFS). bake that shit in from the ver beginning.
This was for a more forum based design with users, profiles and and threads which will never get deleted. A chan is way simpler because the history is even less important than for forums. Any thread older than n days will simply be discarded by every node in the network including all posts.
>Should I do it in go then?
Please use C++, alternatively Rust if you want something reliable.

No, with Jow Forums only the Jow Forums server can log my IP, not the entire world. If they pass on that data to third parties can only be speculated.
With IPFS and Bittorrent you can get tracked by *everyone*.

iknowwhatyoudownload.com/en/contacts/

>only NSA should be allowed to spy me
Imagine the nightmare that is having a public list of anime hashes broadcasted by your IP. Anons shall not know!

I don't like revealing my IP to fucking everyone at all times and I don't like being forced to host CP.

...

That's already bad enough, now imagine the web was replaced by IPFS. Everyone would know every website you visited.
>employer looks you up
>sees that you visit 4channel
>your resumé goes right in the trash

Are you clinically retarded or merely lobotomized?

Opinions on 0xchan? My main gripe is that they had to go and make it blockchain based

I think signatures are unnecessary but this is a good idea. Though the problem I see is how to know post ordering and how to decide which posts to delete without it.
A blockchain can actually be counterproductive if other anons want posts to be forgotten, not to mention it doesn't scale as easily. Still it's more consistent and database is not randomly sampled between the first peers connected.

>the program should support anonymity out of the box
I'm planning to passively wait IPFS to support I2P but I agree.
>Please use C++, alternatively Rust
I want to keep it simple and use the same languages as the ipfs API (go or js) as starter.

Can I use this without having a minimal idea of what goes on behind the curtains like torrents or vpns?

Yes. There's even a tutorial to get you started. The only caveat is no gui. Otherwise it's "start the daemon" and then "add this file" -> receive ipfs link, or go to your browser and visit 127.0.0.1:someport/ipfs/address for whatever content you're trying to look at.

This could be related:
getaether.net/docs/faq/aether_cryptography

Can we have a poll? What's your preferred type of distributed imageboard?
strawpoll.me/17369094

Attached: 1535198543647.png (447x447, 91K)

I don't know what any of those means, at least in the context of an imageboard

Which of these options says "endless supply of downloadable porn" and "can be mirrored, can be filtered by lots of metadata"?

PoW is just a bad idea, period. Never stops spammer and absolutely costs normal users (because they'll be posting for a long time whereas spammers will be at it for very short periods).
PoS is better but has a bootstrap problem, making it unreliable at best and really bad for longevity.
No idea what floodgraph is or does, sorry.
"Other" is basically the only possible viable choice simply because of how much it covers.

Well...
Proof of Work: e.g. scanning hash until it is smaller than a target difficulty
Proof of Stake: e.g. require users to make a ethereum transaction to a smart contract and give them back after a fixed amount of time. I don't know about anonymity.
Flood graph: Each node manages it's own database. A post is simply forgotten when nobody is sharing it anymore or everyone has blacklisted it. It scales better since unlike a blockchain there is no need to download the full history and validate each block one by one. I don't see any straightforward way to make it work with proof of state.
Blockchain: There is a definite order and time for every posts. Everyone has an identical view of the imageboard. Mining is voting for "which blockchain is real".

None is resilient to shilling and spamming.

i have a small video website that is just static html + .mp4

Anyway, people have suggested using IPFS to help with server load etc

How does this actually work and what is the best way to implement it?

Attached: 1514252709118.png (1920x1080, 1.93M)

So, something resilient to spam and easily downloadable (in complete, as mirrorable thing) and easy to filter on companion metadata (generally smaller than the actual data, arbitrary contents yaml/json style) with your own filters would be what I'd want.

>Proof of work

Attached: 1548247232455.jpg (640x363, 28K)

BTW, "spam resilience" needn't be more than a set of "key owners" being able to edit / delete / ... the current version of data in the version history of the site according to what key / key signature they own. They'd define what the current version of the site is, as well as which revisions / contents of the site's history are recommended to the users for deletion or editing or whatever [clients should follow this recommendation by default].

Now, if you don't agree, you can go against the recommendation or not, take the site, and use your own fork of it (with now your or someone else's keys as the only authoritative ones) at a different "address". Everyone sees a different key is now responsible, done.

That's my ideal type of infrastructure.

>BTW, "spam resilience" needn't be more than a set of "key owners" being able to edit / delete / ... the current version of data in the version history of the site according to what key / key signature they own

I was telling a friend about this.

Make the whole internet git.

If the jannies are dicks you can just fork your mod keys and tell them to fuckoff by blacklisting your moderators instead of them blacklisting you.

Attached: 1548526165192.png (404x501, 378K)

reminder that lainchan is a couple dozen underage retards whose entire identity is centered around an edge website that they struggle to even maintain

Attached: kalyx2.jpg (351x466, 159K)

Ya, it would be great. While I'm thinking about forks for resilience against censorship and differences in moderation policy too, it is also very interesting for more minor differences.

For example, if we imagine such infrastructure had a site exactly like this, it'd be nice if a fork could decide to have 200MB attachments and endless length threads and allow prototype AV1 videos. And then some fag could say "holy shit, AV1 videos kill muh processor, let's do a fork without that".

And another fag could fork off a thread to a nearly static Jessica Nigiri image dump only he commits to. Stuff like that.

Sounds like the infrastructure enabling this would be the most interesting one, no matter which it is.

github.com/vasild/cpp-ipfs-api

>Want to make an IPFS torrent client with for make, C, and qt
>No one has updated the C API in years

I don't want to write the damn API myself I'm just starting out...

Attached: AHHHHHHHHHHHHHHHHH.jpg (550x550, 28K)

together we can make it

it's a scam
they showed nothing but want ethereum tokens

doesn't work

files.catbox.moe/t424ae.zip

thank you

btw we could use orbitdb as our database

reminder: if this would succeed, the FBI would flood it with CP so they can have an excuse to take it down and arrest users

Just don't pin the CP then

I didn't any job baka, id rather bring down big corp

Attached: 1547556290592.png (1000x494, 286K)

You need someone to actually host real storage. Most likely paid for.

IPFS lets you leverage the fact that the entire db can flow like liquid from server to server and have the users front some but it still needs to exist in full somewhere.

The answer is a tentative yes.

Huge free CP SHA256 blacklists maintained by rubn of the mill public site owners

About damn time.

Attached: 1548248514804.jpg (521x512, 23K)

what?

Huge free CP SHA256 blacklists maintained by run of the mill public site owners
What's hard to understand

That said hashes are terrible for identifying images, you need some sort of similarity database and I don't know how feasible that is

yeah you really think FBI CP spammers can't get around that? lmao

>you need some sort of similarity database
>and I don't know how feasible that is
NN AI trained on NN and CP.
Come on, it's Jow Forums(nel), we could all help it.
It shouldn't be so hard (pun intended)

you can train your ai for detecting nudity/pornography and for detecting children
if your ai detect nudity/pornography and child it's classified as cp

i read some paper on it recently

that would work as well.
It's not like we need breastfeeding pics on the imageboard anyway so who cares.

>You need someone to actually host real storage.
anyone who want can host it
some threads or images will be dead but it's part of the fun I think

You can have nodes deprecate content in sync or pseudo-sync as part of site rules, whether a node agrees or not is up to them and if they don't they end up acting as archives.
Everyone in the network eventually ends up able to host part of the content because of bitswap so there's not even a need for an ipfschan-specific network of people or anything.

cloudflare-ipfs.com/ipfs/QmRUPe2dBZdsoNpWMwkdBrCUfQm7JK1cPeg45HibUpX9Fn/

weak, still connects to 4cdn for stuff
cloudflare-ipfs.com/ipfs/QmTLEGiq8owGxDau3R52 7RkZda4ZeasRfPK3KCugxPV8bQ/boards.4channel.org/g/thread/69660869.html

how have you done it?

httrack --depth=2 --ext-depth=1 -c8 --disable-security-limits --max-rate=0 'boards.4channel.org/g/thread/69660869'
it's a bit overkill, for example, it grabs the front page of every board, because the page links to them all, but it's set like that so it can grab the full images, resulting in a rather complete copy

thank you

Stuff a 1-2U box full of drives, apply FreeBSD+ZFS or DragonFly+HAMMER2 or Linux+ZFS/BtrFS/bcachefs, and colocate it. Add an init service for the IPFS daemon and a cron job to pin content from the chan's front page. Done.