Decentralization

All timestamps are based on your local time of:

Posted by: stak
Tags: reciprocity
Posted on: 2011-06-17 14:07:24

Back when I was going through interviews for co-op positions (circa 2005), one of the questions I got asked was "what technology in the last decade do you think has had the greatest impact?" (or something along those lines). I thought about it for a few seconds, and then said Napster, because it really brought the concept of P2P to the public's mindset. (I didn't get the job.)

Fast-forward a few years, and the trend towards decentralization and peer-to-peer transactions has expanded into all sorts of different areas. Open-source software has always been one facet of this. Other, more tangible facets are things like auctions, peer-to-peer loans and even room rentals.

Decentralization has some obvious benefits. It allows for reduced dependence on single points of failure. It distributes much of the burden back to the individuals involved in the transaction, who actually have an incentive to perform well. It allows for greater customization. But it also has disadvantages. You lose the ability to mass-produce and take advantage of economies of scale. It's harder to ensure a consistent experience, and therefore harder to guarantee a minimum feature/quality/support baseline. Like most things, it's a trade-off.

I think most things start out in a centralized system because it's more convenient. It requires less of a commitment from the user to try. For example, if you've never bought a gizmo before, it's usually more convenient to go to the nearest hypermarket and pick one up than it is to scour the web and try to decide between the umpteen billion different varieties out there. However, as you buy more gizmos, you become aware of your specific needs, and are willing to pay more for specific features (and also not willing to pay for features you don't need). Then, scouring the web becomes a better option. Taken further, you might even decide to build your own gizmo because none of the umpteen billion varieties are perfectly suited to your needs and requirements, or because finding the perfect one is just too hard.

This is why I think it's hard to introduce a decentralized system for a new product or idea without first going through a phase where there is a centralized version. It requires too much of a commitment from the individual at a time when they do not yet fully understand the benefits it could provide. However, once they have had a chance to try the generic centralized version, they realize the benefits, and are more willing to invest the effort required for the decentralized and customized version.

Exercise to the reader: apply this idea to social networks (e.g. Facebook) and cloud storage and sync (e.g. iCloud), and see where it takes you. Please post your thoughts as comments.

Posted by varun at 2011-06-17 16:01:33

Three thoughts came to mind:

1. In addition to robustness, decentralization allows you to route around things like censorship. I forget who once said that the internet treats censorship like a malfunction and routes around it, but that certainly true. In fact, if I recall the early history of what is now the internet, I think the design was picked in part to reduce the possibility of single nuclear strike wiping out all communications in the US.

2. Curious why you picked Napster. I would have said the single biggest technology between 1995-2005 was https, which I think was standardized around mid-2000ish, though I think Netscape added https support in 1994/1995... It made the modern web possible.

3. We're working on a system for reporting electrical outages in [country] right now, primarily through mobile phones - send a text to a number, your address and it gets logged for checks. We also "seed" the system with automated reporting at the substation level - literally an Arduino and a Nokia keeping tabs on whether the power is flowing, and it automatically sends back the same kind of info. How would you classify this? As centralized/decentralized? I would never have classified this as ever having gone through a centralized phase (per your penultimate para), but ... thoughts?

[ Reply to this ]

Posted by varun at 2011-06-17 16:02:50

Gah, sorry. Re: point 2 - what I meant is, curious why you picked Napster as an example of P2P. Was it actually P2P? As I recall, it was all uploaded centrally (ala iCloud) and then downloaded by many.

[ Reply to this ]

Posted by stak at 2011-06-17 21:20:04

1. Good point. That's kind of what I meant by avoiding single points of failure, but it's worth noting.

2. I'm pretty sure Shawn Fanning didn't have a server farm running out of his dorm room; as far as I recall the only thing centralized was the directory but the actual music transfer was P2P. And sure, HTTPS is nice and all, but I wouldn't classify it as revolutionary. And while we're on the topic, the centralized nature of certificate signing in HTTPS is it's single greatest weakness.

3. Interesting- I would still classify that (and in fact the entire power grid) as a very centralized system. In some countries you can buy solar panels and plug them into the grid; you can sell power you generate back and make money. That's starting to shift towards a decentralized power grid - imagine if everybody did that; the risk of widespread power outages would be dramatically reduced.

[ Reply to this ]

Posted by Sebastian Wain at 2011-08-17 20:58:52

I have a short answer on an important issue against decentralization (i.e.: Search, Social Networks). It's because it will be very difficult to analyze the data if it's distributed on different places with different owners?

[ Reply to this ]

Posted by stak at 2011-08-17 21:02:31

Yes, you're absolutely right. This shifts the balance of power more towards the individual owners. It's a tradeoff that you can view as either a positive or a negative depending on your goals.

[ Reply to this ]