Email is dead! Long live email!
Email has long been the killer app of the internet. It has taken us to a world were everybody has an address and anybody can send an email to anybody else. Email works incredibly well.
At the same time, email is totally broken. Address books are painful to maintain, and they don’t tell us about somebody’s ability to actually reach another person. Spam is a major problem, i have no easy way of saying who i want to be able to send me messages, no way of saying, i don’t want messages from you any more. Spam, and getting around spam filters with legitimate email is a huge problem. The vast majority of email is spam.
There is a reason the myspace/facebook generation hardly use email. They’ve got a system which solves the spam issue, built in is a buddylist which lets you define who can send you messages. It’s also a realtime system rather than a store and forward system expecting users to be mostly offline. The problem is these message systems are walled gardens.
Jabber, and it’s XMPP protocol, were built for IM, but they made it super flexible. It can easily be used for email to solve the delivery permission / address book issues. Now i can easily authorize people to send me messages.
The addresses are even compatible, so you can have a bridge, attempt to deliver via xmpp and if the domain doesn’t handle xmpp roll back to smtp.
Of course i’m far from the first person to have thought of this, it’s come up in 2004, 2006 and 2007. So the question is, why hasn’t it happened. Well first off, these things don’t just happen on their own, somebody has to do the work, write software, organize it, make the change you want to see.
How could we get from here to there? Kill email so that email can live free?
Well the way i see it there are several things which need to happen. It’s a chicken and egg issue, nobody is sending email via xmpp because nobody can receive email via xmpp, the clients don’t exist. Nobody’s building xmpp email clients because nobody’s sending email via xmpp to receive.
There are some things working towards adoption of xmpp for email. Critically, the email address can stay the same. Many of the alpha geeks already use the same address for their email and their jabber IM accounts. The email address is deeply embedded in the culture and any attempt to vary from it would doom the move to email over xmpp.
The second thing which can help lead to adoption of xmpp email is the integration of email / messaging and IM. You see it in yahoo mail, facebook, and myspace. The big email / messaging providers are already routing IM messages alongside the email.
The third third thing which will help is we don’t actually have to get very many providers to adopt xmpp email to get critical mass. Despite email’s incredibly federated nature, there are a few providers who have hundreds of millions of accounts. Get one, or several of those providers to switch and provide xmpp email support in addition to smtp email, and you’d be able to use that to shift everybody over. The promise of getting out of spam filters will be enough to get the big senders to jump over, and once you’ve got both sides, then it can take years for the rest of the net to move over, it has to be a gradual process. Once you’ve got a big provider who accepts email via xmpp, then you can work on building out library support, create the email extensions to ejabberd, openfire, etc… and bridges to sendmail, exim, postfix, etc..
The obvious people to do this are the google gtalk & gmail teams. They’ve already pushed the idea that my gmail address IS both email and IM. They’ve got the servers running in parallel. It’s simply a matter of building out a test setup, defining how the standard will work, and getting gmail to support it.
This is not to say that there aren’t others who could do it. My ex-employer, Yahoo! could do it, but fixing the future of email is probably not a high priority. Microsoft could do it, but they have a hard time adopting open standards and wouldn’t be trusted by the open source developer community who maintain the current technology stack which makes email work. Other than google, the only other player who i could adopting this and pushing it forward is actually AOL. While mostly ignored, AOL has been pretty decent at adopting and pushing new technology and could see this as a way of getting back in to the lead setting trends for the future.
While we wait for somebody big to adopt it, building proofs of concept, making a system which will work, would probably push forward the case for replacing smtp with xmpp as our global email delivery system.
14 Comments, Comment or Ping
Rabble, I love this idea, but how does a buddy list transate into an email white list?
At scale, won’t I be “spammed” constantly by permission reuqests?
June 20th, 2008
This is a cool idea. And XMPP seems to be getting a lot of attention in another sphere now too – as a messaging protocol, say, in cloud computing applications.
http://brainspl.at/articles/2008/06/02/introducing-vertebra
What would also be cool is if someone like facebook did this, and supported xmpp for im and email, and NOT smtp. Then everybody gets a soandso@facebook.com address, using just the shiny new stuff. That plus adoption from gmail/gtalk would get pretty much everybody’s attention
June 20th, 2008
Email can already do what you are asking for, the problem is the interface that email providers give you to there spam filtering technology plus mail server white/blacklists.
June 20th, 2008
Looking at this another way, by automatically adding your gmail contacts to your gtalk buddylist, does google make it’s own version of IM more spammy?
June 20th, 2008
Wait, EX-employer? where are you now?
June 20th, 2008
I think leveraging social networks for this is a bad idea because they are insular and are not considered to be viable or particularly productive by all. If GMail/GTalk could manage to push the envelope here, that would be great, but older users and non-tech users tend to shy away from IM technology. Not to say that IM is inherently bad, on the contrary, there are studies suggesting that IM makes people more productive. I would argue that with the point that e-mail is almost identical, but that’s another story.
If it’s just a matter of the underlying protocol, then the change could be made easily and most easily by Google and AOL as you suggested. Why? Because it would be transparent to users as they both control the server and client software that users are accessing. As for the spam issue, there are some good points that arise. How do we allow new people onto the whitelist? Can people request to be added in the manner of social networks, or do we bring our own friends? Do I as a user then need to add a contact every time I want to receive e-mail from a new sender or source? That would be rather painful and not acceptable. Administering a firewall is bad enough without having to micromanage e-mail. Technology is supposed to make lives easier, not more complex. I’ve rambled on enough. Interesting idea, you have.
June 20th, 2008
Let’s see if I’ve got this right… The idea is to switch the transport of messages from SMTP => XMPP. Fine at the core that’s like saying I wan’t JSON formatted data not XML. It’s just wire format.
The problem becomes that while you can transport a message, who do you receive messages from?
If you only accept email from people in your address book (which hotmail supports) then you’re back to needing a way for anonymous communications. The “biggest” user of this is that order confirmation from XYZ corp that’s going to show up in your mailbox, do you first need to add them as a buddy before a message is accepted? That new website that isn’t OpenID compatible sends you a confirmation message, where did it go?
At the end of the day you still need that anonymous communications channel, but adding more and more structure to the display of messages makes a huge difference.
June 20th, 2008
great idea… thinking if spam would cause any problem
June 20th, 2008
great idea… thinking if spam would cause any problem
June 20th, 2008
I think there are actually a couple of good alternatives to the current mail problems we all have, however I see the bigger problem being the backbone providers, core hardware providers and the carriers. Spam is bad for us, but good for someone selling bandwidth. I see that being the problem with the lack of wider adoption of IPV6 at a carrier level as well, because less private address space means less spoofing and in turn spamming but this would cost the bandwidth providers precious revenue in spam traffic. I think the barrier in adoption of something like XMPP is the loss of revenue rather from the problem it will help to solve rather than the know-how or adoption process…
June 20th, 2008
Maybe there could be some sort of crypto challenge/response aspect whereby people can silently add themselves to your whitelist by correctly answering a question.
From a spammers point of view, they could:
A) spam the email address with whatever, my mail daemon sends a crypto challenge that they don’t know how to answer/no response/wrong answer, I’m none the wiser as an end user and they stay off the whitelist.(admin would see the attempt obv)
B) spam the email address, by computing the response to a challenge for a single email address.
If the challenge was some sort of fun crypto problem like in DES that uses the problem of factoring very large numbers, the spamming network would drop like a sack of potatoes before sending out any volume of unsolicited spam.
If a spammer wants to be on my whitelist without my intervention they’d have to earn it
Now that I think about it, those “secret questions” when you forget your password would make good challenges for other users to respond to to get on a whilelist.
June 20th, 2008
This seems to sound like gmail.
June 20th, 2008
I think there really is a business market out there for such a client. I would personally pay for such a client. It’s cheaper than a spam filtering solution, and would work better.
The only issue is that if you don’t use it exclusively, that is you use it in addition, than people (or solutions) still have to filter out the spam from the SMTP part of the AND equation. The perception, remember most people don’t really understand SMTP in the first place, never mind know there is a protocol to email, will perceive it as one and the same. I still get spam, so what’s the point…
Therefore you’d also most need two separate email addresses. One for XMPP and one for SMTP. It could be as simple as replacing the ”@” with another character. Whatever it is, the client has to show there is a difference between SMTP and XMPP otherwise it won’t be perceived as working.
June 20th, 2008
XMPP does not have any of the email tooling or meta-data. Each program does its own logging / index /search, each one does it diferently, and theres much less context to work with than there is with email (where conversations have subjects and are naturally threaded). The notion that unbroken streams of conversation can replace packaged messages is, to me, laughable. I think XMPP’s dominate in an inevitability, but only after many decades of adding in the kinds of context that allow email to work across long time spans in occasionally-connected environments. You talk about plenty of drivers for convergence, but it really doesnt sound like you’ve given any thought to what role email serves, or have any idea how different IM is in function and form.
June 20th, 2008
Reply to “The Future of Email: From SMTP to XMPP”