not there yet
In 2017, I wrote how to decentralize your website with IPFS. I wasn’t able to do it back then.
Two years later, a few things have changed. Instead of 52 million dollars, Protocol Labs has about 257 more, bringing the total over 300 million dollars. The kind of money where you start giving money away and hiring top-tier talent. Two years of development have produced a multitude of open source repositories - 245 in the IPFS organization, plus another 67 under multiformats, and 168 under libp2p, and 102 under IPFS Shipyard.
So what should we expect of IPFS? At five years old, is this a project that’s usable ‘here and now’, as the homepage promised in 2017? Are all the parts in place, just waiting for web and application developers to see the light? Have the stumbling blocks I noticed in 2017 been smoothed over?
No
IPFS is still not usable for websites.
I tried, again, to make IPFS work. Where I ran into bugs, I reported them. I asked around in the forums, and hopped on a very informative call with a team from Pinata. When the process required scripting and workarounds on my side, I implemented what was necessary. I wrote & proposed fixes for many of the documentation issues.
I want something like IPFS to exist. I’m skeptical of the ‘crypto’ ecosystem that IPFS sits next to, but I also take the crisis of the web seriously. If blockchains prove to be useful, I’ll welcome them as part of the solution.
In other words, this isn’t a knee-jerk rant. I tried to make IPFS work until I had to call it quits. These are my notes, and some conclusions I reached.
The goal
The goal is the same as it was in 2017: make a decentralized version of macwright.org. A decentralized website.
macwright.org is the easiest possible kind of website to decentralize. It’s lightweight, has few external dependencies, no trackers, and a simple, customizable build process. If you can make any website work, you should be able to make this one work.
My only restrictions on this mission are:
I won’t use a service. It’s great that folks are creating services around IPFS to make it easy. But I’m evaluating IPFS, not a service, and by relying on something prebuilt, I’d likely insulate myself from knowing about the kinks. I also don’t want to host on, say, DigitalOcean or a cloud server at first. Decentralization means it needs to work in my house.
Updates need to be reasonable. This is a blog. I write new posts, and link those posts from Twitter. IPFS aims for immutability of content, but this blog isn’t immutable, so I need a mechanism of updating a pointer to the latest version.
It needs to be addressable. IPFS uses lots of hashes. You’ll see a lot of references to things like Qmc5cLBCY6fsWkxqpMS1RTovhERoXZSSDrjkjogrVfrDfJ. That’s perfectly great for architecture, but that’s not human-friendly. There should be a way for humans to find things.
The process
What follows are my notes from trying to build a system, step by step. I aim for little goals in increments.
Step 1: Install IPFS locally
In 2017, to get from ipfs.io to the ‘Download’ button, it took 4 clicks through redundant pages. This number has been reduced to 3. But the inconsistencies remain: macOS is referred to as darwin in one place, OS X in another, Mac OS X in another. And there are some new bugs: the IPFS desktop app is included in the installation instructions, but the link to it is broken.
The naming inconsistency is going to be a theme. One of the biggest problems with the Protocol Labs ecosystem is its carelessness with language. Jargon and acronyms are reproducing without need or limit, and producing a world of words that mean too much and too little at the same time.
Step 2: Browse your website locally
Okay, so I’ve got IPFS running. Let’s check out the simplest case: serving my website locally and browsing it locally. In my case - using Jekyll to build, this looks like:
ipfs daemon &
bundle exec jekyll build
cd _site
ipfs add -r
This prints a long line of output, ending with
added QmesFFG92JmyZsPfSrAtbEMSWELsfkPfADhhcg8FjxKB3z _site/resources
added QmTrLEV3NQBhgYM2ZiW52eoSdvVqwU36p3qsQvH4oTfYkM _site/simple-statistics
added QmbkWSVC1VqbswL6NCN4VSZZRoKPtvcBEfkfFbeWVnddGn _site/swift
added QmNb7T9sa3zgvpoRs1k92MQ1pNZxjk3FxaHDc6U9tGAPfj _site/talks
added QmP662VnQbyxK6tbnqhueTzJnPgkFBx7G6hwwTwAtAYyz4 _site/tmp
added QmR2eSRGdDhnnUHWYFvwwbyMDkAJ2gPVMJQZh4TQwKKJW4 _site/topics
added QmVwS9a5LYs3rU3nCyMkmbdHj2ZguE5PpbQy14gjeSytQt _site
Rather terse, right? No ‘success’ method to tell me whether it’s done or the process just died. No indication of which one’s the root entry, or how I might be able to get to it. Just a lot of hashes.
Now, this is a CLI, but it’s also the recommended route for getting a website set up. Some creature comforts might be worthwhile here.
You access this website by copy & pasting the last hash on that line, installing the IPFS Chrome Extension, and prefixing it with http://127.0.0.1:8080/ipfs/. So here we are:
Step 3: Make links work again
Ready to publish it to the web? Not so fast. Clicking a link brings us back to my issue in 2017: the way that the IPFS gateway works will break your links.
Essentially, since you’re seeing your website rooted at /ipfs/HASH/, then any link you have that starts with / will go to the wrong place. And you can’t just add that prefix to all your links, because the hash comes from the content, and also because folks might be looking at your website from /ipns/hash or /ipns/domainname.com.
So, links don’t work. I posted an issue detailing this issue, and while I got an encouraging response that there’s a real solution planned, there’s no real solution. People use specific plugins just for IPFS, like this one for GatsbyJS, to get it to work.
I ended up writing make-relative, a script that rewrites my built site to use relative links. This is where the story about IPFS being useful here and now for web developers breaks down a little. I’ve done enough HTML-mangling and path-resolution in my decade in industry that writing this script was straightforward. But the knowledge required to do it is not all that common, and I think this is where a majority of web developers would call it quits, because IPFS’s ‘website hosting’ story would look broken.
And it is pretty much broken. The current way of using IPFS in a browser is a bad hack, just like it was in 2017. Websites will be broken by default by it, and it still has no semblance of the web security model.
Step 4: Give it a URL
So I’ve written a custom script and now have a browsable local website. I’ll want to have a local server that keeps this website online, but first I’ll dive into the URL problem.
Okay, so we rely on the web’s existing, working, good-enough addressing system every day. You’re on this page right now by typing in macwright.org, or clicking a link, and in the address bar is the domain name macwright.org. Domain names are things that you pay for: I pay gandi about $18 a year for the privilege of that name, and they split the money with Verisign, who runs .com, and ICANN, which runs the bureaucracy around disputes, renewals, and governance. Your browser then uses DNS, a decentralized naming system, to resolve that name to an IP address which is the IP address of the server that has this content.
Decentralized web projects have to decide whether they can replicate this system in a purely decentralized way (with no ICANN, Registrar, or company involved), or to rely on DNS to ‘bootstrap’ their systems.
The IPFS stack chose to implement all of the ways.
There’s DNSLink, a rather practical system that has you add another DNS record to ‘link’ it to your decentralized website.
There’s also IPNS, which is purely decentralized, but at the cost of being very, extremely, notably, slow. It also doesn’t produce pretty names – instead it gives you a long hash, just like the ones I saw with ipfs add -r.
Those are the fundamentals. Let’s add some detail.
So, an IPNS address looks like
/ipns/QmSrPmbaUKA3ZodhzPWZnpFgcPMFWF4QsxXbkWfEptTBJd
The documentation advises us that DNSLink is faster than IPNS and yields more pretty names. So here’s a DNSLink address:
Careful readers will notice that this address also begins with IPNS. But, as far as I can tell, it doesn’t necessarily use IPNS: it could be – and usually is – macwright.org uses DNSLink to indicate an IPFS address. Why would DNSLink use /ipns/ as a prefix if it’s often used instead of IPNS? I’m as confused by this as you are.
So, anyway, I set up DNSLink within IPNS, thinking that this must be what most folks do. Surely they don’t update their DNS records every time they update their website.
That was an incorrect assumption. IPFS-based websites do update their DNS records every time that they update their website, so that they can avoid using IPNS, because IPNS is just too slow.
This was a tough discovery, because it works against everything I know about DNS – a system that isn’t particularly designed to be fast or scriptable.
But, I had to forge on, so I wrote gandi-ipfs, a tool that would let me update my DNS records to use DNSLink. Reportedly other folks use CloudFlare and have their own scripts. I gleaned this information from a call with the very friendly and helpful Pinata team.
Step 5: Host it
To mark our point in the journey, here’s what my script is looking like now.
ipfs daemon &
echo "Jekyll building…"
bundle exec jekyll build
cd _site
make-relative https://macwright.com/
hash=`ipfs add -r -q . | tail -1`
echo "Root hash: ${hash}"
gandi-ipfs $hash
So I’m using my custom script make-relative to make all the links on the site relative, and the script gandi-ipfs to update DNS records.
But the server that I start with ipfs daemon stops every time I close my laptop’s lid. So the next step is setting up a server.
I decided to start with a Raspberry PI kit - the Pi-Hole in particular, which is intended to be set up as an ad blocker, but happens to contain all the parts I need for a simple server. I know full well that Raspberry PIs are small, low-power, low-memory computers and I might need something bigger for long-term performance, like an Intel NUC. But it should do the trick for testing.
Setting up the PI was pretty smooth, and it’s magic to just log into a tiny computer. It’s an ARM chip and runs a certain flavor of Linux.
Installing IPFS on the Raspberry PI was also smooth: just download the ARM distribution of go-ipfs and you’re set.
Unfortunately, once I started getting this set up as a ‘pinning device’, the fun stopped.
I tried running ipfs pin add with the hash generated earlier from ipfs add -r, but it just ‘hung’ - outputting nothing at all. After a while I realized that, like ipfs pin add, IPFS doesn’t communicate very well when it’s having a problem. So I figured out how to turn logging information all the way up, and then… I was never able to get past a ‘got error on dial’ failure, despite trying all potential configurations of the IPFS daemon, enabling logging, upgrading to the newest version, and so on. There are about 63 similar issues in the tracker, 21 of which are marked as bugs.
Epilogue
That’s as far as I got.
Like last time, I might have missed some critical step or made the process harder on myself. For ✨journalistic integrity✨, this post hasn’t been edited or reviewed by anyone. Hence the typos.
I tried to make the process documentation straightforward and fact-based, but couldn’t help but add some foreshadowing about the issues. And there really are issues.
Here are the problems
An overextended, under-documented, and unfinished constellation of projects. No single part of the ecosystem is truly finished and well-documented, and new parts are spawned every day. Projects that are almost universally avoided – like IPNS – are still included in the main documentation and recommended as if they’re usable.
The same goes for language: IPFS has a sprawling set of jargon that’s inconsistently used. Is it merkledag or merkle-dag? There are enough terms that there’s a glossary, but the glossary itself refers to both merkle-dag and merkledag. Is a node a peer or a piece of content? A low point was discovering two repos whose readmes referred to each other as the same thing.
And then there are the three big issues: usability, reliability, and performance. Do CLI commands have explanatory output? Are error messages informative? Can I count on tools working as advertised? Will the decentralized web have Bittorrent-like scalability, or will it be more like Bitcoin?
Here are the recommendations
First, make recommendations. I went down the IPNS rabbit hole because the documentation sent me, and only by a chance encountered did I learn that it’s almost universally avoided. There’s no shame in saying that a project’s not ready. Recommending unusable projects burns goodwill. Recommend paths that work.
Second, fix your words. Words are work, and that work is not happening. Finish a glossary, standardize usage and meaning, and cull unnecessary jargon. Treat new bits of jargon like technical debt, because that’s what they are.
Third, set realistic goals and make realistic statements. IPFS.io still has a web-centric message and promises that it’s useful here and now. It promises ‘fast performance’, and support for ‘huge datasets’. These are goals, not realities. An effort to put 300TB of data was met with mixed results and notes about adding and retrieving data being extremely slow.
Fourth, set a goal. This is a slightly different question than the last one. A core question is: is IPFS trying to be an internet? The website would say yes, and some of the documentation. But the 2019 goals punt the ‘decentralized web’ to 2020+, instead focusing on NPM on IPFS. Which then leads us to entropic, the most promising distributed package manager, which has a discussion about using IPFS that immediately brings up its performance problems.
Maybe I’m being too tough on IPFS. But this isn’t 2014. IPFS isn’t a new project, and it isn’t resource-limited. Protocol Labs has raised over 300 million dollars, and has been around for 5 years. That’s a lot of money to pay a lot of smart people.
So a few scenarios are possible. Maybe most IPFS users are using it for file storage and as an API backend, kind of like textile. I’m the odd one out expecting it to be useful for websites. Which would explain the haphazardness of DNSLink and IPNS, but not the performance issues. Or maybe I’m misjudging the arc of history – that Protocol Labs is a 20 year project, not a 10 year one. But really I suspect that some of the hype exists because folks are talking about IPFS but they don’t rely on it it. People excited about the potential of FileCoin and otherwise hyped on crypto technology want to imagine uses and combinations of technology without being tethered by the reality of what doesn’t work.
I hope that Protocol Labs sets a goal and achieves it. The IPFS future is exciting. But we aren’t there yet, and I’m not sure we will be.
Issues & PRs: docs/161, docs/162, multicodec/132, go-ipfs/6354, go-ipfs/6357, docs/173, forum thread
Read the original article
Comments
By janandonly 2019-06-125:3515 reply Who is going to defend the free internet agains Azure, AWS and Google Cloud? They are the very opposite of a free and open internet where everyone can "run a website" on her own machine.
It pains me to see a great idea like the Interplanetary File System still not working. I had similar experiences with IPFS and yes, we do need a project like this, only without the broken incentive structure attached to it. Why a "Filecoin"? We already have a tested and working native internet currency: bitcoin.
I've long believed that a browser itself should evolve to be both viewing visited websites but also to host said website for an x number of minutes/hour/days. A bit like webTorrent aims to work. This way a website that gets visitors gets decentralised hosters as well :-)
Filecoin isn't a currency. It's a token that proves "I replicated x amount of data, y reliably, for z amount of time", and (protocol-wise) can be exchanged for file storage services only. The market decides how much file storage is actually worth, by arbitrage.
The same principle goes for Namecoin, incidentally. The token embodies the value of the resource. I actually think this kind of thing is a much more stable base for a currency than Bitcoin's "it's valuable because we say so" system.
By mishmosh 2019-06-130:13 It’s worth noting that Filecoin and IPFS are independent, but mutually beneficial projects and networks. Using IPFS and Filecoin together is completely optional - as you can see by projects like Pinata, OpenBazaar, Qri, and DTube already using IPFS in the wild today.
By design, the Filecoin Network stores data on Filecoin nodes — not IPFS nodes. While we want to make these two networks even more interoperable (https://github.com/filecoin-project/specs/issues/143) — so that, for example, you could choose to supplement IPFS nodes with Filecoin nodes — we want to leave the choice up to users. There is a big opportunity to have a distributed marketplace for ensuring IPFS persistence, and multiple solutions optimizing for different use cases will likely coexist.
We don't need tokens.
We need P2P stuff that works.
Like https://github.com/webtorrent/webtorrent and https://github.com/amark/gun
They're both run in production, at scale (millions of users), and do NOT require any tokens.
>, and do NOT require any tokens.
You're looking at it from a pure technical perspective of pushing bytes around in a decentralized way.
What the folks pushing "tokens" are trying to solve is the game theory of financial incentives to store & serve those decentralized bytes. In contrast, things like Bittorrent/Beaker/webtorrent/etc depend on others' "altruism" to host and serve files.
Because altruism doesn't scale, that's why nobody wants to seed my 100 gigabytes of personal vacation photos. Sure, they'll be happy to seed & disseminate the latest cracked copy of Adobe Photoshop or a bluray rip of the latest Marvel Avengers movie. But my personal files are uninteresting to the current decentralized web.
(But I'm not claiming Filecoin actually solves the incentive puzzle. I'm merely pointing out that the "problem" Filecoin tries to solve is at a higher abstraction level (the economics) than webtorrent (the protocol).)
By blurbleblurble 2019-06-1217:331 reply > altruism doesn't scale
Incentives don't exist on a one dimensional scale from pure greed to pure altruism.
People have lots of contextually dependent reasons for storing or sharing information. Those reasons are dynamic and super diverse.
In your case, why would you need lots of people to seed your 100 GB of personal vacation photos? You can seed them just fine! But don't use bittorrent, use the right tool for the job...
As a matter of fact, for the past 4 years I've been using btsync/resilio/syncthing to backup my personal media across different devices, as well as 70GB of audio projects. It's been working great.
That seems like a relatively small scale. And I'll bet some of those photos would be interesting enough to family and friends that they'd seed them.
From my naive perspective, the information itself is much more raw of a "currency" than any abstract economy built on top of it. Its "value" is determined by its relevance to one or many people, and p2p sharing, I believe, has the ability to accommodate this across many scales.
Why did public trackers fail? It wasn't because people didn't want to seed the content, it's because seeding the content became legally dangerous (and the trackers got shut down). To me, that's not a failure of "altruism", it's a failure caused by greed and an inability of industry to adapt.
And also by the reality that the tools are prototype level. Bittorrent and bitcoin are two incredible examples of applied cryptography and network science. They've showed us what's possible, but they're only the tip of the iceberg.
For many applications, lack of commodification is a feature, not a bug. What's holding us back is that the tools aren't built yet, but that's a work in progress!
>, why would you need lots of people to seed your 100 GB of personal vacation photos?
I think you're missing some context for the motivation of Filecoin. One of the elevator pitches is that it can disrupt cloud storage like Amazon S3.[1][2]
People do put personal files (e.g. via a cloud backup service) on cloud storage like AWS S3. Instead of a paying a centralized Amazon, Filecoin claims they have a way for money to go to the decentralized owners of harddrives.
Therefore, if "backing up my personal vacation photos" to Amazon S3 is a use case, that means the decentralized peers hosting my uninteresting personal files is also supposed to be possible.
>But don't use bittorrent, use the right tool for the job...
Yes, exactly. That's what my reply to gp was explaining: his suggestion of webtorrent/bittorrent/gun is a protocol that solves a different problem than the one Filecoin tries to solve.
[1] https://www.google.com/search?q=filecoin+amazon+aws+s3
[2] deep link of Juan Benet of Filecoin mentioning the centralized cloud services: https://youtu.be/6h2WNxEV8q4?t=482
By davidgerard 2019-06-1310:31 > the motivation of Filecoin
was to sell tokens. Take an existing idea, build an incomplete implementation with a token attached, talk about features that don't exist as if they do - ICO success!
I agree, the problem is that Bitcoin and Filecoin (per author's IPFS scaling issues) do NOT scale though.
You must solve the technical scaling problem first, then sure, heck, add tokens if you dandy.
WebTorrent/GUN/etc. do scale. Add economics to that.
Preferably, add something that is time-scarce so people do not have to lose money (they don't pay FB or Google! If they have to pay Filecoin, they'll still choose free FB), something like BAT or Pirate Booty ( https://hackernoon.com/hollywood-crypto-behavioral-economics... ).
By dTal 2019-06-1217:49 You can't always just "layer economics" on top. Sometimes the problem is economical.
Consider what it would take to replace DNS with a distributed system. In a global namespace, names have value. You can't just operate on first-come first-served - there needs to be a system that ensures that names go to whoever wants it most. In other words, the challenge to be solved there isn't the technical one of having a DHT - that's a mostly solved problem - it's how the hell to make the names cost money, and who the money goes to. (And if you don't think the names should cost money, how do you propose allocation should work?)
It isn't valuable because "we" say so (whoever "we" is). One reason it is valuable is because millions of dollars of energy resource is put into it to ensure that it is decentralized
By ForHackernews 2019-06-1217:141 reply That's a ridiculous misunderstanding of value. Just because you waste valuable resources to produce something doesn't mean the thing you produce is now valuable. If I feed my dog millions of dollars worth of beluga caviar, it doesn't mean his dogshit is now worth millions.
To the degree that bitcoin is valuable today, it's because speculators think somebody else will pay them more for it in the future.
Did you read the last part of my statement "to ensure that it is decentralized"?
True decentralization at this level is why it is valuable
By ForHackernews 2019-06-137:13 The fact that you have to waste ever-escalating amounts of electricity to make bitcoins doesn't ensure that it's decentralized.
If anything, it means that economies of scale will favor larger organizations. In fact, that's what we see, with a few large Chinese mining syndicates controlling a majority of the hashrate: https://www.buybitcoinworldwide.com/mining/china/
Compare bitcoin to a real decentralized system like bittorrent, or hell, email.
By acdha 2019-06-1217:29 > We already have a tested and working native internet currency: bitcoin.
Bitcoin manages low 6 figure transactions per day, using as much energy as the entire country of New Zealand to do so. That’s far from internet scale (or even “single Raspberry Pi”-scale) and, because it was designed around gold bug economics, the protocol says it can never improve because the whole point was to scale the overhead as a function of network size.
Fundamentally, the reason why none of these have worked comes down to economics: if you’re storing things, you want provable access times and reliability but the costs of doing so in a decentralized system are much higher than in a centralized model because you need more copies to balance out the lack of trust. That also hurts on the other side of this: if you’re considering hosting, you need to charge more to cover your risk of getting involved in legal proceedings and there’s a fairly high threshold where that risk just isn’t worth it.
>Why a "Filecoin"? We already have a tested and working native internet currency: bitcoin.
Maybe the "proof" section of the FAQ will explain the difference: https://filecoin.io/faqs#what-s-the-difference-between-all-t...
If you've already read that FAQ and still disagree, are you saying you can embed the incentives to store terabytes of others' data and also the proof of that storage into existing Bitcoin blocks?
(I'm not saying IPFS "proof-of-storage" scheme will work. I'm just saying Bitcoin's proof-of-(cpu)-work doesn't seem to easily map to verification that the peers' harddrives actually has the data they claim to have stored.)
By simonebrunozzi 2019-06-1216:28 In fact, Bitcoin's blockchain is not the right data structure for something like this.
By tlrobinson 2019-06-126:203 reply > Why a "Filecoin"? We already have a tested and working native internet currency: bitcoin.
Because it’s a lot easier to raise $300 million on a bit of hype selling your own coin.
(Also Bitcoin micropayments aren’t quite there yet, but I’m not sure any other truly decentralized cryptocurrency really is either)
By sharperguy 2019-06-1212:19 The problem with any alternative token transfer systems is that they either introduce additional points of centralization (hence attack vectors) or end up being significantly inferior to bitcoin in terms of scalability, reliability, testing and user base.
By ukd1 2019-06-1217:38 Also, why filecoin -at all- when siacoin exists and works already. Filecoin is vaporware.
By dpacmittal 2019-06-127:111 reply I feel Nano is doing an outstanding job. It's what bitcoin was supposed to be.
The funny thing about every comment which positively mentions Nano is the writer invariably has zero idea what tradeoffs are made in the consensus protocol to make it faster. There are no free lunches in distributed systems which must contend with byzantine failures.
There is an alternative stack that works pretty much as you describe, its browser being Beaker: https://beakerbrowser.com/docs/tour/
There is an option in there to seed visited content as well as to host your own stuff.
By zachsherman 2019-06-126:062 reply Yeah I was so confused why the author didn't just use dat it's so easy
By rkda 2019-06-126:24 He also tried dat.
https://macwright.org/2017/07/20/decentralize-your-website.h...
Hasnt written about it again though.
By rzzzt 2019-06-126:12 I'd like to read a comparison between the two hosting experiences; I'm not sure that eg. the linking-to-other-content part would be easier in dat-world, unless they run a pinning/re-hosting/gateway service tied to their current domain (which is doable, either standalone, or via Hashbase).
By jota_rv 2019-06-1212:01 > I've long believed that a browser itself should evolve to be both viewing visited websites but also to host said website for an x number of minutes/hour/days. A bit like webTorrent aims to work.
As far as I can remember that's one of the features of https://beakerbrowser.com/
By jeremyboom8 2019-06-126:031 reply > Who is going to defend the free internet agains Azure, AWS and Google Cloud?
This is exactly what the http://metacurrency.org project aims to bring to life: http://holo.host/learn/ and http://ceptr.org
The distributed pattern they have created combines Git local source chains with cryptographic signing and BitTorrent’s DHT to create easily evolvable apps using http://holochain.org. Peer validation enforces rules defined in the logic of each app. Apps run locally instead of on a corporate server. If the current tech ecosystem is Platform Capitalism, then this pattern is Protocol Cooperativism.
I haven’t come across anything like this.
Why don't we go back to simple store and forward like mechanisms like net news, just with enhanced content formatting? Store and forward static websites.
By ssb1 2019-06-1210:27 Something like https://www.scuttlebutt.nz/?
By bergstromm466 2019-06-1212:201 reply Yeah I think we should, and storing and forwarding is exactly what Holochain does. I think there are different use cases for different tasks. I really like, and use, Scuttlebutt and DAT - which also store and forward data.
Scuttlebutt has been awesome for me to see that a social network can be completely hosted by it’s users.
I think Holochain and Ceptr are here to help us evolve how we work together and do our accounting.
What I love about Holochain's pattern is that it essentially provides a fully distributed, immutable and peer-to-peer crypto accounting framework for barter transactions/sharing work and resources. It does not focus on creating and managing artificially scarce coins (as is the case in the current financial system, and also in Blockchain). It is trying to show us that we are using Industrial Age tools (interest-bearing-debt national currencies), and that instead we could focus on creating peer-to-peer Information Age currencies [1].
Holochain's pattern enables communities to make their own judgements on what value something holds and how they want to account for it. It allows us to build that into a flow language - a currency. This means instead of using a fiat 'medium of exchange' backed by US imperialism and it's military might/control [2] - we create, redeem and destroy currencies based on asset-backed mutual-credit principles that mimic the way Mother Nature works. I think I read somewhere that today over 90% of money in the money system is speculative, and interest-bearing monopoly debt-money has no roots in the real world.
An example of a mutual credit currency could be a currency for seats on a train. When I buy a ticket, my ticket is tied to a certain seat on the train. If I went to Mars, my ticket would be worthless. The ticket is only valuable because the ticket is tied to the train system - to one of the seats available. This makes it asset-backed. Another example would be farmers being able to pre-sell their produce locally, without going through expensive middlemen. They can create currencies that represent their produce (the asset).
The MetaCurrency Project is bringing us a into a world where we have more agency when interacting with the world, using evolve-able peer to peer protocols that can be forked and re-forked, again and again, to better meet the needs of it's users.
They are giving us the tools to see the world with new eyes, and to allow multiple perspectives - instead of one rigid, centrally enforced truth.
Moving towards using mutual credit systems, together with reputation currencies (imagine your ratings on Uber, eBay etc. was hosted by your peer-to-peer community), allows you to build trust together in a way, un-intermediated or hindered by a corporate app and UI. These reputation currencies already exist, yet are contained and held hostage by our current paradigm. Other examples of reputation currencies I can think of are ‘FairTrade’ or ‘Organic’ labeling.
Holochain is BitTorrent + Git + Cryptographic Signatures + Peer Validation + Gossip [3] - making it agent centric instead of data centric. It is not anymore about the database in the middle, but about the relationships between peers and their balances/interactions, and the agreements we make together on what actions we are allowed to take in our different apps and networks. Together we can see and improve the app logic and I can bridge between my apps, as I now have complete control over my own data [4].
HoloREA is a project that is using Holochain to build a modular, cooperative and transparent economic supply chain framework. It uses the Resource Event Agent accounting model to enable users to steward and follow flows of economic actions. These relationships are today tightly controlled in corporations. HoloREA aims to create a cooperative web of interactive flows of people, actions and resources.
Essentially, the vision is to reclaim The Commons and give us better tools to steward the world together more peacefully and compassionately.
If you’re interested in this paradigm, a few keywords I've found useful: Protocol / Open Cooperativism (currently we have Platform / Surveillance Capitalism), mutual credit, myth of barter, buddhist economics, social darwinism.
I also like the work of Elinor Ostrom, Bernard Lietaer, David Graeber, Douglas Rushkoff (Life Inc.), Charles Eisenstein, E.F. Schumacher, Silvia Federici, Mariana Mazzucato and Anand Giridharadas.
For technical material: http://ceptr.org
Footnotes/links: [1] https://prezi.com/xmzld_-wayho/new-economy-new-wealth/
[2] “Nixon floated the dollar in order to pay for the cost of a war in which, during the period of 1970–1972 alone, he ordered more than four million tons of explosives and incendiaries dropped on cities and villages across Indochina—causing one senator to dub him “the greatest bomber of all time.”7 The debt crisis was a direct result of the need to pay for the bombs, or, to be more precise, the vast military infrastructure required to deliver them. This was what was causing such an enormous strain on the U.S. gold reserves. Many hold that by floating the dollar, Nixon converted the U.S. currency into pure “fiat money”—mere pieces of paper, intrinsically worthless, that were treated as money only because the United States government insisted that they should be. In that case, one could well argue that U.S. military power was now the only thing backing up the currency. In a certain sense this is true, but the notion of “fiat money” assumes that money really “was” gold in the first place. Really we are dealing with another variation of credit money.” — David Graeber: ‘Debt: The First 5000 Years’
[3] https://holo.host/faq/what-is-holochain/
[4] https://medium.com/holochain/holochain-reinventing-applicati...
By ambicapter 2019-06-1214:261 reply How is reducing the fungibility of money not a step backwards?
By jeremyboom8 2019-06-1215:161 reply Using this pattern increases fungibility. Could I ask you to share what makes you think it decreases fungibility?
By ambicapter 2019-06-1216:031 reply > When I buy a ticket, my ticket is tied to a certain seat on the train. If I went to Mars, my ticket would be worthless.
What if I don't want a train ticket? What if I want to sell my widget-coin for some brussel-sprouts-coin. Now I can't go to any farmer because he's only got carrot-coin or broccoli-coin, I have to find the brussel sprouts farmer-and not only that, but the brussel sprouts farmer who wants to buy widgets.
And if you say "nonsense, you can exchange any coin for any other coin", how is that different from what we have now?
By jeremyboom8 2019-06-1620:19 It’s different because the US dollar is the worlds’ reserve currency, and it is violently backed by US imperialism.
The MetaCurrency Projects’ Holochain is the next generation P2P no-middle-man accounting system which open sources and decentralizes the rules around money creation and usage. It allows us to evolve the rules together and build mutual credit, interoperable, open and unencloseable carrier’ currencies. Currencies are backed by real world commodities, leading to a rich network of flows between a community.
Your comparison shows me that you might not yet have had the privilege of critically exploring the underlying patterns and stories that have made Industrial Age money the tool it is today. For me ‘The Future of Money’ by Bernard Lietaer, and David Graeber’s ‘Debt: The First 5000 Years’ are very paradigm shifting books and helped me to challenge the underlying stories of economics that I believed. These stories stopped me from seeing a more beautiful future.
By _prometheus 2019-06-131:32 Decentralizing how we link, address, and move content is the basis of IPFS and Filecoin both. Both are solving different parts of moving our digital infrastructure to infrastructure that we control, that we govern, and that we are not held hostage by.
While we still have a long road to go w/ IPFS usability, it’s worth pointing to what works today -- there are millions of end users benefiting from IPFS, 100Ks of libp2p nodes, we see PBs/mo of traffic in the infrastructure that we run, and millions of daily requests to our gateway. Look to fully decentralized applications and systems like OpenBazaar, Textile, Dtube which use IPFS and paving the way. New technology -- especially new technology that seeks to build new platforms from the ground up -- is hard and extensive to build, and to polish. We’re working on it, and though nowhere close to where we want to be yet, there’s a lot of utility already provided.
Re Filecoin incentive structure -- this comes from recognizing that computers are not all the same -- there is huge utility to be had in dedicated infrastructure running 24/7, well maintained, high performance spread out around the world. P2P systems of the past have failed to match the reliability, uptime, and performance levels of centralized systems. Cryptocurrencies enable the creation of Open Services (like Bitcoin) that run public infrastructure w/ high uptime and reliability guarantees. With Filecoin, we aim to bring that to file storage and distribution. More here: https://www.youtube.com/watch?v=6h2WNxEV8q4
Sadly a lot of the mechanisms behind bitcoin make for it to be difficult to use in a vast transaction setting (millions/sec, similar to credit cards), which would be required for using its blockchain for proof of work purposes.
That’s why alternatives exist in order to solve these “problems” that the BTC community either choose not to fix or simply won’t. It’ll be interesting to see what crypto people would intend to use for something like IPFS.
By davidgerard 2019-06-127:511 reply IPFS is from the crypto people. This is what blockchain advocates put forward as the solution to mass storage for their blockchain dreams.
IPFS is basically BitTorrent with magnet links - hashes to address content, if you want to be sure your obscure content will stay around then you have to seed it, etc. There's some fancier stuff on top, but if you know that's how it works, all else follows.
You can tell how "blockchain" IPFS is from the way that advocates seem chronically unable to tell "could" from "does" - stuff that doesn't exist and/or doesn't work is routinely talked about as if it's here and working in the present. This leads to disappointed posts like the one linked.
IPFS is useful to blockchain projects because of the permanence of content hashes. That doesn’t mean it’s not also useful to many others. QRI is using it as the publishing medium and backing store for all kinds of data sets, for example. No blockchain, just shared storage across interested parties.
You make a really good point about communicating readiness - the author of the post is expecting more from pre-alpha software than it is ready to provide.
IPFS has had public releases since 2015 with rhetoric implying general utility, not to mention $300M in funding. If it’s still “pre-alpha software” that’s not a mere communications problem.
Er, I meant pre-beta not pre-alpha. It's still in development. Sounds like the core API is pretty much settled and not expected to change (much).
Building a platform is not like building an app. And a platform that changes significant parts of internet infrastructure is different from just building any platform. I worked on Firefox for 13 years... the internet is a harsh and fickle thing.
That said, people are using IPFS with millions of users today.
I very much agree about the rhetoric. The homepage is polished, and speaks to a bunch of features that are possible to varying extents in varying environments, but not easy to demonstrate. Bandwidth savings is a good example of this.
By acdha 2019-06-1323:59 > That said, people are using IPFS with millions of users today.
This seems like the thing they should lead with: what are those users doing and why would be a great way to explain the value.
By quickthrower2 2019-06-125:433 reply The middle ground might be to use a local hosting company that has it's own servers and racks. Pay a bit more, but boycott the big clouds for personal websites. You could probably host a few dozen static blogs on a physical machine for say $100/m. If 100 people do that it's affordable.
> You could probably host a few dozen static blogs on a physical machine for say $100/m.
100 bucks!? I host a dozen unique domains with static content (and a few other services) on a 1/1 VPS for 3 euros/month. At Hetzner, not at a hyperscaler.
By quickthrower2 2019-06-1210:031 reply Yeah I was thinking a whole server not vm.
By dspillett 2019-06-1214:53 Even a real-metal physical box can be had for far less than $100. Even excluding the ranges that are out of stock Kimsufi has a number of options that are more than capable of running "dozens of static blogs" with ease for <$20/month (the €8/month Atom based machine would do the job, as would the currently-out-of-stock €4/month ones).
There are other providers with comparable offers, I mention Kimsufi because for some time I've had one of their low spec units running a couple of simple tasks.
You are not going to be getting brand-new unused high-spec kit with that sort of deal, of course, but you pays your money, you takes your choice!
By x3ro 2019-06-126:08 Uberspace [1] does something like this in Germany. You get to pay what you want, too.
Edit: I'm not affiliated with them other than being a customer.
By dspillett 2019-06-1215:09 > If 100 people do that it's affordable.
If one of those people doesn't mind running as sys-admin and help-desk for the rest...
Though your $100/m is rather higher than it needs to be for that level of service so you won't need the same number of people for that cost/person, someone is still taking on a part-time job doing admin for others.
By StreamBright 2019-06-126:53 The real threat to free internet is not Azure or AWS but things like AMP. You can still host your website even on a server in your basement.
IPFS is a great idea but it is very hard to implement something like this. It was tried before (hello Wuala) but no success. I am not sure if this idea can be implemented at all.
By rohan1024 2019-06-1212:35 Opera 11 could act as a server back in 2008 but nobody use it then so :/
By stdcli 2019-06-1222:09 There happens to be a great solution for this known as gaia hubs: https://docs.blockstack.org/ which you can learn about in these docs in conjunction with the immutable identities associated with them.
I have also in more context spoken recently with someone and convinced them why they should use gaia over IPFS: https://forum.blockstack.org/t/cannot-find-ipfs-driver/6147
and they brought up some other usability issues not mentioned in this critique, particularly about file pinning to keep files alive, and in general the waste behind the idea of data duplication by not separating out the auth of the user owned data with the duplication of the data itself in the DHT network, for which Blockstack has "Atlas".
> Why a "Filecoin"? We already have a tested and working native internet currency: bitcoin.
Filecoin's promise is to be tightly coupled with file storage, to set up contracts that are automatically enforced by the network. Bitcoin does not do this.
I don't know much about it, but maybe Ethereum could be used to do that, though
By ukd1 2019-06-1217:39 Or you could just use siacoin, which already does this for storage
By emmanueloga_ 2019-06-126:128 reply Just today I was looking into IPFS vs DAT, does anybody have any insights about the similarities/differences other than the ones listed here [1]?
From far away, DAT looks smaller and better documented (perhaps less ambitious, too?) Apparently the best IPFS overview is the 2015 paper [2] which looks pretty daunting and does not seem to cover any practical considerations.
1: https://docs.datproject.org/docs/faq#how-is-dat-different-th...
2: https://github.com/ipfs/papers/blob/master/ipfs-cap2pfs/ipfs...
I consider dat:// to be the better protocol, in part because of what you mentioned. Other advantages are the lack of duplicating data on disk (IPFS makes a copy of all data it shares) as well as having a versioned history of all changes. That way app owners can'tp ublish malicious versions while preventing people from using the non-malicious ones.
Essentially, dat:// behaves like BitTorrent but the torrent data can change.
The only downside for both protocols I can think of is that the integration story outside the browser and CLI tools is very poor (there is no FFI/C lib Ic an bind my Rust app to)
> (there is no FFI/C lib Ic an bind my Rust app to)
Language choices in both have set off my "this is doomed to obscurity" alarm bells for precisely that reason. You don't write the reference implementation for a new Internet protocol—especially the core library for it, and especially a very complex one—in a language that can't easily be included in most other languages. So, probably C.
Dat in particular seems great but ain't no way I'm relying on a large JS project for anything I don't absolutely have to, on my own time, especially if it deals with my files.
By eudoxus 2019-06-134:54 With regards to IPFS (don't have much experience with Dat), I have a hard time understanding how choosing Go to implement their protocol, as well as Javascript makes it doomed for obscurity. A lot of distributed/decentralized applications and platforms are being written in Go. This is the first time I've heard this argument.
Not to mention you can call into Go funcs over shared libs from C.
Rust is a great language definately, but not using it is a completely sane design choice. C has been a source of a number of security issues due to memory management, that type safe languages solve.
Not to mention, there are various parts of the IPFS/LibP2P stack that are being written in Rust by other teams.
There is a BitTorrent extension for updatable torrents:
By zaarn 2019-06-1212:34 Yeah but there isn't any client that supports it. Or more importantly, easy-to-use libraries that allow me to use it in my projects.
By blurbleblurble 2019-06-1210:121 reply Dat-rs exists but seems to be on pause.
Datrs dev here. I implemented all of the Hypercore feed protocol last year. The next step was to add the networking layer, which was blocked on async IO in Rust. So that's what I've been working on this year (the Runtime project).
Datrs development should be unblocked again soon, starting by moving the fs layer over to async IO. And then tackling the network layer.
By blurbleblurble 2019-06-1211:39 Awesome to hear that! Thank you so much for your work!
DAT is run by influential hobbyists of independent means, with occasional funding from non-profit organisations.
IPFS is essentially run by a very well funded (300musd) private company.
For this reason alone I think DAT is the more likely to succeed. It seems hard to reconcile the longevity of a truly distributed protocol with the need of a private company to retain control.
By dunkelheit 2019-06-1211:061 reply > IPFS is essentially run by a very well funded (300musd) private company.
That is news to me - I've thought they were a scrappy startup. The issues mentioned in the post like glitches in the docs are excusable for a project that relies on volunteers who prefer writing code to polishing docs, but if you have $300M in funding then wtf? Just, you know, hire good project management and docs people.
My understanding was that most of the team focus has shifted over to the filecoin project in the last year or two, and they aren't dedicating as many resources to ipfs. That said, I agree that this is pretty deplorable.
By stebalien 2019-06-1222:06 Several of the key members have shifted focus but there's still a core IPFS team. But we're still spread quite thin.
We're still paying down years of documentation debt but there has been quite a bit of progress:
* Expanded https://docs.ipfs.io/ with a concepts section. * Tutorials https://proto.school/#/ * A ton of work on libp2p specs (https://github.com/libp2p/specs/commits/master) along with a full time documentation writer.
My bet is both likely to fail. NIH syndrome is strong with both cases.
For efficient file transfer protocol between peers, Bittorrent protocol have multiple independent implementations that is working right now. They should build on top of that. Instead, both DAT and IPFS try to implement their own protocol with dubious additional features. For IPFS, it even relies on traditional DNS. What are they thinking?
> NIH syndrome is strong with both cases.
Seriously. IPFS decided the standard URL format wasn't good enough so invented something worse, which was pretty funny/sad. I never saw reasons for it that made any sense.
See:
https://github.com/ipfs/ipfs/issues/227
[EDIT] here's the original, deeply "LOLWUT?" justification for it, quoted in this comment on another issue. Other justifications were given but this was the motivation. Oh man. Wow.
https://github.com/ipfs/go-ipfs/issues/1678#issuecomment-139...
[EDIT] farther down, same author as the quoted text in the above comment: "wish unix:// wasn't taken by unix sockets." Oh FFS, guys. Hahaha.
By willglynn 2019-06-1217:09 IPFS also needlessly rolled their own TLS replacement:
https://github.com/ipfs/specs/issues/29
> I would add that if TLS can be used without all the X.509 nonsense, and with our own choice of pubkeys, including using different signing public keys (not the DHE keys) in the same conn, then we can consider breaking our "not TLS pls" stance.
TLS fit their requirements all along, they just… decided to reinvent it instead of reading about how to use it.
By tetraca 2019-06-1216:28 Their scheme as described in #227 would make sense, if they were designing IPFS as a service for Plan 9.
By blurbleblurble 2019-06-1217:12 This strikes me as a little disingenuous. Neither IPFS nor DAT are inherently dependent on DNS. If you're referring to HTTP the gateways that lots of people use with both IPFS and DAT in order to host static sites, arguably that's because support for UDP and other useful p2p tools in the browser is experimental! But people still find both IPFS and DAT useful for hosting static files. (libdweb is really exciting by the way: https://github.com/mozilla/libdweb)
From what I can tell, most "serious deployments" of DAT and IPFS are made by people directly using the loose underlying collections of libraries that implement each of them. These people often end up putting together application specific transport and discovery layers that work for their specific application.
By polyfractal 2019-06-1212:031 reply Except that, from the article, the only way to really use IPFS for pretty names is to use DNSlink and not IPNS. Because IPNS is unusably slow.
So IPFS might not theoretically rely on DNS, but it seems that it does practically rely on DNS if you actually want to use it.
By dboreham 2019-06-1215:34 If you want to use it with DNS names, yes.
I have a hunch that IPNS is just broken in its implementation (manifesting as "unusably slow") but I haven't yet had the spare time to investigate this theory..
By dboreham 2019-06-1215:42 afaik several of the Dat folks worked on Bittorrent implementations beforehand so it's possible they were thinking...something?
By dboreham 2019-06-1215:39 There are a bunch of differences but the important ones (imho) are:
Dat doesn't use immutable addressing (addresses stay the same when content changes) while IPFS does.
Dat at the lowest layers is stream-oriented, allowing stream-oriented services and applications that are near-real-time. IPFS is static blob/object oriented.
IPFS has a better developed "discovery" network at present (if you use Dat today you are typically in your own island whereas with IPFS you're part of "the" IPFS network). This is being worked on however.
By momack2 2019-06-132:18 Some decent answers in this stack overflow thread: https://stackoverflow.com/questions/44859200/what-are-the-di... and a good response from IPFS creator jbenet here: https://github.com/ipfs/faq/issues/119#issuecomment-21827839...
I think it's possible to view Dat and IPFS as two different layers of a stack that can interoperate and each solve useful problems at their layer. For example, Dat has more UX focus and high-level abstractions making generic app development smooth and easy (an area IPFS is weaker - though https://medium.com/textileio has been working to make this much better), while IPFS has the benefit of global name-spacing and content-addressing primitives that enable deduplication across identical datasets and validate the content is what you asked for (used by tools like Qri to do dedup within a data commons: https://qri.io/faq/). I've seen demos of projects using both together, each for their unique strengths - but there's still a ways to go to make interop easy.
If you were looking for nice IPFS overviews, I'd recommend: - https://hackernoon.com/understanding-ipfs-in-depth-1-5-a-beg... - https://medium.com/textileio/whats-really-happening-when-you... - https://docs.ipfs.io/introduction/overview/ (see the concept guides for easier to parse explainers on CIDs, Pinning, etc)
By langitbiru 2019-06-128:151 reply Don't forget Swarm. https://swarm.ethereum.org/
So Swarm vs DAT vs IPFS.
If it has a blockchain in it, it immediately crosses the "too heavy" line for me.
By thunderbird120 2019-06-1211:181 reply Swarm would probably cross that line even without any of the blockchain stuff. It's more akin to Freenet than Bittorrent and would probably function better as a decentralized backend for most people. Expecting an average person to run a Swarm node is probably unrealistic in anything resembling its current form but allowing resilient decentralized sites which are accessible from any one of a number of gateways should be doable when/if they actually get around to implementing insured data storage that is the key component to the entire idea.
By StavrosK 2019-06-1212:46 You're right about that, but I do wonder if this will be perfect for making distributed darknet marketplaces and torrent trackers a la what.cd.
By jmozah 2019-06-1314:46 No. It doesn't depend on blockchain except for the ENS resolution.
By folex 2019-06-129:11 There's also Arweave, which is targeted on hosting web sites in a decentralized way.
- https://www.arweave.org/ - https://github.com/ArweaveTeam/arweave
By AgentME 2019-06-1210:11 I've got a comment comparing IPFS to Dat up in the Dat thread that's up now: https://news.ycombinator.com/item?id=20162881
tl;dr: I feel that IPFS is the smaller more tightly scoped project that fits better into the existing web ecosystem. Dat has its own browser and versioning and other stuff bundled in, and IPFS works with normal browsers (https://news.ycombinator.com/item?id=20162972) in a way aligned the web's graceful degradation principle.
By unicornporn 2019-06-126:16 I would recommend this recent take on Dat:
I work for a cryptocurrency company as a software engineer and I definitely agree that a lot of projects backed by crypto funds tend to over promise and under deliver.
It's clear that those who control the funds aren't always the best at judging tech talent. Big tech corporations have had years to settle down and build a reputation to attract top talent. Crypto companies tend to attract greed over talent and it shows in practially all projects.
Its improving for some projects but not others.
I've seen this argument before and tend to disagree. There is certainly greed, but I think the larger problem is that crypto doesn't have users.
I don't think it's particularly novel in 2019 to say that a successful app is built off of the feedback of its users. Any app, regardless of intentions or engineering prowess, is going to struggle if it doesn't have a sizeable user base providing feedback (and devs who listen to that feedback).
I think you're touching on an important point. There are already other vessels for the niche that cryptocurrency fills—at least as far as websites and web services go—that are way easier to both implement and use. With that said, I'm still rooting for cryptocurrency as a general competitor to state-backed currency, that in turn gives people all over the world more freedom to conduct their business without state or corporate intervention. That is most of all about freedom, and the only bogeyman they've yet to conjure against it is "it's used for criminal stuff". Like the dollar isn't also used for crime...
By plywoodtrees 2019-06-1510:451 reply The distinction between "gives people more freedom to conduct their business without state intervention" and "enables crime" seems very elusive.
I can see an argument that enabling crime under oppressive regimes is moral. Or even enabling particular crimes that you feel shouldn't be crimes.
I have trouble seeing the legitimate use case for cryptocurrency in first world countries. If the main use case is crime, money laundering and speculation, it will either be squashed or remain niche.
By drawnwren 2019-06-1512:23 I tend to agree with your assessment, but you are dancing around an important point.
Most people in the world don't live in a first world democracy. Today, many of these countries are stable and providing currency for their citizens. But ask a Venezuelan if they can see a reason why we might want an alternative to a state backed currency and I feel like the answer is self-evident.
Vitalik saw this use case and talked about how disheartened he was by the ICO boom because it was use cases like this that he believed in, so it isn't at all an afterthought.
By davidgerard 2019-06-1310:39 > Like the dollar isn't also used for crime...
Crypto people say this like it's an argument in their favour - whereas it actually means: even for its one and only genuine consumer use case, crypto is utterly trounced by conventional currency.
By tracker1 2019-06-1222:17 I work in an election services company... you wouldn't believe how sales driven the space is ridden with blockchain based solutions. In the end, most would either not add enough value or require too much client buy in to work with the variety of districts/cities/counties/states we deal with.
Are there any ICO-backed projects that didn't overpromise and underdeliver (or not deliver at all)?
By davidgerard 2019-06-1310:43 tl;dr no. http://archive.fo/7H3MM
By stebalien 2019-06-1222:13 IPFS was started in 2014, long before Protocol Labs started working on Filecoin.
