This is part one of what I hope will be a series of posts describing the drivers behind our vision at .pw. At .pw our goal is to provide a unified, ubiquitous, intuitive and extensible communication platform to users and businesses worldwide. Decentralization (as described in this post) is an important element in our vision and product strategy.
Decentralization is not a new concept. The IT landscape in the early 80s moved from a centralized, time-share based, thin-client computing model to a decentralized, poweful, rich, thick-client desktop. In hindsight this shift from centralization to decentralization seems obvious now. Computers started out as large, expensive machines that occupied enormous rooms affordable only by a privileged few, and utilized as a precious resource in a shared manner through teletype access terminals. Moore’s law eventually reversed the direction of the pendulum, resulting in the shift of balance to an extent where today’s desktop workstations rival the most powerful machines in those days. This was the first wave of decentralization.
The second decentralization wave is now upon us. The Internet connection usage model has always followed a client-server paradigm. This centralization has been a result of several facts we take for granted that are very rapidly turning into false assumptions.
Drivers of the 2nd wave of decentralization
1. The transition to ipv6
Perhaps the most critical juncture in this decentralization wave will be the transition to ipv6. Infact if anything, this wave of decentralization has been considerably delayed due to lack of ipv4 addresses, resulting in end user devices having to exist behind NATs which are not service friendly. Devices behind NAT cannot expose a service endpoint even if they have the processing capacity and bandwidth, without complex port forwarding. Due to the nature of TCP/IP, a server must be available on a unique public IP address.
In the early days of the Internet, it was the lack of reliability of endpoint connectivity that resulted in a clear demarcation between clients and servers. Today however that distinction is blurring, with desktops having access to similar bandwidth and processing capabilities as servers. The NAT-imposed limitations however continue to maintain that demarcation. Servers are no longer distinguishable from clients by their power or connectivity, but simply by the fact that servers have unique IP Addresses and end user desktops don’t. End user clients invariably live with a virtual IP address, unable to expose their desktops to the Internet as a server, despite having equivalent connectivity and processing power.
With ipv6 however we have billions of ip addresses per human on this planet, so unless interstellar travel makes a monumental leap, life is discovered on thousands of other planets, and the Internet becomes the backbone of inter-galactic communication, we will likely never run out of ip address space. This means every single device produced can have a unique endpoint address - cellphones, home appliances, automobiles, heck even your t-shirts and undergarments may be connected to the mesh with a unique IP address, and will likely run a web service 
2. Connectivity ubiquity
Connectivity is becoming ubiquitous and bandwidth cheaper. With the advent of wimax, edge, EV-DO, CDMA we are already connected almost everywhere through every device. Gone are the days when 99.99% network availability was a datacenter-only luxury. Today many of us already assume always-online connectivity at our home, on our office desktops and even our mobile devices.
3. Bandwidth costs are tanking
The only thing falling faster than the post-subprime stock market is the cost of bandwidth. Broadband has now become affordable to the masses. The success of youtube and other media heavy websites is a testament to this important change.
Impact of the 2nd wave of decentralization
1. Blurring of the client-server paradigm
As a result of the above changes, the distinction between a client device and a server device will begin to blur, and all devices will be expected to act as both clients and servers in a computing cloud/mesh.
2. Protocol re-definition
Many of the popular protocols we use are designed keeping in mind limitations that we discussed above. For instance many popular protocols require servers for guaranteed delivery - eg SMTP/POP/IMAP assume a clear distinction between an always connected and available server and a disconnected/offline client. Email delivery therefore requires a chain of intermediate servers to send a message from one user to another. However in a world where all devices are always connected and have unique IP addresses, an intermediate chain of servers is not required for message delivery. Additionally HTTP, our most popular protocol is designed to be unidirectional, wherein a client sends a request to a server and the server responds with a response. These protocols seem appropriate in an environment where there is a distinction between a client device and a server device. However with the blurring of this distinction, newer protocols will likely be more conversational and P2P. XMPP is one such protocol (though Jabber currently does follow a client-server deployment model)
3. Push vs Pull and the era of instantaneous communication
The killer app of the Internet - email - is, as of today, non-instantaneous. An email sent from one user to another traverses through a series of mail servers to a final mail storage server from where it is downloaded by users using POP/IMAP/Webmail, all of which are Pull/Polling based protocols. If your client is configured to fetch email once every 5 minutes, each email received by you is likely to be 5 minutes delayed.
Some of the Web 2.0 touted protocols like RSS, Atom etc are no different. They are all pull-based polling-type protocols. Pull is necessitated with an offline client. The assumption is that since the client is not always online, it will pull the required content when it actually does come online. This does not connote subscription in its true form. Though RSS/Atom are touted as content subscription protocols, they are not really subscription protocols. Compare this to the offline world. If you were subscribed to a magazine, would you expect it to be delivered to your doorstep every month (Push), or would you expect to visit the local magazine store every 5 minutes to check if it is out yet and buy it once it is out (Pull eg RSS). Subscription connotes a push based behavior. RSS can only simulate it, but it is not true subscription.
Pull based protocols are slower and inefficient (both bandwidth-wise and processor-utilization-wise) than Push based protocols.
Decentralization however will result in the proliferation of new push-based protocols. Content and Communication will be pushed to endpoints rather than polled. Only in the rare scenario that a device is not connected will content have to be temporarily stored on an intermediate server until it can be pushed to the client once the client connects.
What does this have to do with .pw?
At .pw we are creating a communication platform that will enable various devices to leverage a decentralized, always-online, reliable network - a computing cloud of peers so to speak.
Sign up at http://start.pw and join the ride!!
June 5th, 2008 at 12:01 am
The Internet has always been peer to peer. You are confusing the web and the Internet. HTTP has always been client-server.
IP itself is peer-to-peer. SMTP is peer to peer. IRC is peer to peer. Usenet is peer to peer. The Internet existed before 1994 and assumed that all hosts would always be able to communicate with each other (the principle of end-to-end connectivity).
http://www.sanog.org/resources/sanog7/apnic-randy-nats.pdf
http://www.faqs.org/rfcs/rfc3424.html
Push based protocols assume the remote end is always connected. This is not always true. Handling push involves complexity both in technical terms (reliability), as well as in social terms (consent).
Your push is my spam. Email is asynchronous. However, it is both push and pull. SMTP is push. End user clients using POP3/IMAP have to pull. Power users using shell accounts can eliminate those and just read mail directly.
The web _IS_ centralised. The Internet is not.
June 5th, 2008 at 5:20 am
Very interesting video bouncing many problems with today’s networking infrastructure and listing some crazy sounding ideas
http://video.google.com/videoplay?docid=-6972678839686672840&q=networking&ei=PrtHSLPgBYnQwgPmuPy1BQ&hl=eN
June 12th, 2008 at 7:04 am
The era of ‘pull’ is definitely going to fade out or be conquered by push in the near future. There are a variety of advantages to push rather than pull.
However, push does contradict the idea of a decentralized architecture. To explain this, push requires a server to bear the load of delivering messages on the fly. Before delivering the message it can perform actions like filtering the message for spam. Thus, it definitely takes the load of the client and burdens the server in perspective.
Also, Push need not be restricted to email itself. New technologies speak of “push” to be used for applications as well which drastically takes the load of a client. For instance, Apple has come up with a concept of “push” for its iphone as well as mac applications. You need not have a background process running whatsoever. To explain this with an eg: you need not have msn running in the background on your iphone or mac, waiting to receive a message. You can kill the process completely and a server would automatically push a call to your phone/mac asking it to launch msn or yahoo or jabber whenever some one has messaged you.
Thus ipv6 aids centralization to a certain extent where processing power can be borne by a server and the results displayed on a terminal ; “push” being an example of this.
August 28th, 2008 at 2:05 am
Hmm.. interesting post guess i came across this late..
A few thoughts lying around curiously in my mind..
Hmm NAT essentialy is a pain however it does fix the solution temp.. ipv6 would give us tonnes an tonnes of ip addresses essentially having all pcs online..
But take a sec, think about this not all of the worlds population is tech savvy.. think about how many pcs could be victims of trojans, or used as zombies for DOS if directly exposed? more than this that concerns me is the cost..
i know change is inevitable and ipv6 would definitely make the world a better place but at what cost? at the cost of changin the entire infra? its estimated Implementing IPv6 will incur an infrastructure cost of around $200 billion, and that’s just for the U.S. Figure another $200+ billion for the rest of the world…
expensive… i rem my dad tellin me.. If it aint broken dont fix it..
Now when i talk of change of infra.. think of changing all existing routers and also probably reduction in performance.. why you ask? well todays packet size is roughly 64bytes, where as ipv6 would be arnd 86bytes! next think of technos based on Asychronous Transfer Mode.. atm uses 53byte packet with 8byte packet.. switching to 32byte adressing would be reduce the payload to 16bytes from approx 40bytes.. not quite good.. woah gues then to solve this we would use a NAT like structure for ATM haha back to square one..
So probably at the end i may get a new internet which is slower than the olden one…
August 29th, 2008 at 12:43 am
@Leo: Interesting comments. Quick observations -
While I cant verify the cost of upgrade (the $200 billion you speak of) - this number is continuously reducing (IT equipment costs continue to reduce year after year). Also most new purchases are IPv6 compatible and older equipment will get phased out slowly but surely
From what I hear the IPv4 address space will likely run out in the next couple of years … so it is ‘broke’
Given moore’s law, I would not worry too much about additional processing overhead. Infact NAT induces an additional processing overhead too.