Does genuine tech innovation happen better in a recession?


I know this may sound a little counter intuitive, but bear with me. There are some important kinds of innovation in technology which require a recession and not full employment. In the late 90’s there was the huge dot.com boom, everybody who could code at all was working from dawn to dusk. Some people made huge amounts of money, some also lost it, others just worked endless hours.

Then there was a collapse, huge numbers of geeks were unemployed, hundreds of thousands in the san francisco bay area alone. Out of that collapse there was time for people experiment, play with stuff, do things which were not required for meeting the next deadline. That recession and massive underemployment of geeks created blogging as we know it today.

It also laid the groundwork for the small agile startups which optimize the whole web 2.0 thing. Ludicorp which could go from building a multiplayer game to photosharing site. Design shops who have time to roll their own web framework in an obscure programming language from japan, and the like.

When you are busy and having paying clients banging on the door, you don’t have time to focus on exploring new areas, discovering the depth and meaning of your medium. When i was working at odeo we were all heads down, coding, fixing, building, getting the damned thing done. There was real competition, apple and yahoo were both wanting to crush us with their huge legacy advantages of userbases and platforms. It was only once we realized we’d lost, that Odeo wasn’t going to be the podcasting platform we’d envisioned, that Ev decided to do a series of hack weeks, play with new ideas, see if there wasn’t something new which could be created. That playing around seemed like floundering to me at the time. I was frustrated and burned out. But that process is what helped Jack pickup some ideas he’d had floating around in his head for years, but no time to work on. Those ideas became twitter, called twttr at the time.

The dot bomb wasn’t the only time when many techies were underemployed. I was reading a review of Matz’s keynote at Ruby Fools in denmark a few months ago and something jumped out at me.

“The recession in Japan meant that Matz – while not unemployed – didn’t have many assignments at work, so he secretly started to hack away at Ruby at the office.”

Not only was blogging the result of industry wide underemployment of geeks, and twitter the result of a similar thing within a single company, but Ruby was created in a very similar environment during the recession in Japan in the 90’s.

To the vast majority of programmers, it’s a job, they’re the %80 who are vocational programmers. They program during their work day, and go home to do other things, on the weekend they aren’t working on their own projects. Then there are the people who are obsessed, the geek’s geek as it where. We believe in learning a new programming language every year.

Real innovation in software seems to happen when we have underemployment of the true geeks. Either inside of companies where the geeks are allowed to explore, or outside of companies through economic forces. Long hours and tight focus is often needed to get a startup going, or ship software, but it’s not what creates something really new.

It’s ironic, the very forces of modern capitalism which praise the internet economy as being the ultimate in friction free globalized markets hold back real innovation. It’s the down cycles, either within companies or in the broader labor market, which give the innovators space to risk walking down dead ends. It’s a cliché to say that you need to be able to fail if you are going to take real risks. When geeks are free to hack on their own then no deadlines, just interesting spaces to explore. It’s then that we see real innovation.

I spent a year working at Yahoo Brickhouse, we were supposed to be yahoo’s ‘inner startup’. We were going to find a way to create new stuff, employed to wander in the woods and come back with amazing ideas. We were told that it was ok to fail 9 times out of 10. We built several things which never made it out the door, two which made it to beta launch, Bravo Nation, and Fire Eagle. I’m not sure what the future of Bravo Nation is, but i know Fire Eagle is going to launch and it’ll be a major part of making location based systems interoperate. Fire Eagle was a project which was started by Tom Coates, Paul Hammond, and Simon Willison, they were working from London and largely ignored by everybody at Yahoo. Before they could finish they were taken off of the project to finish higher priority things. Then Yahoo Research Berkeley said, oh there are some neat ideas here, and they played with it for a while, they were also ignored by the larger yahoo who focused on shipping things rather than playing with interesting concepts. Eventually Tom talked Brickhouse in to joining in on this crazy location broker idea and it took another 6 months of work to get a beta out the door. The whole process went round and round all the while yahoo was doing it’s own internal soul searching. At the end of the day Brickhouse had to make either BravoNation or Fire Eagle succeed or it was getting restructured in to who knows what. The day of the big layoffs back in February, Brickhouse lost it’s director, Salim Ismail, and his boss yahoo vice president Bradley Horowitz. Caterina, who had the brickhouse idea in the first place, had already left on maternity leave. Chad Dickerson was left in charge and did amazing work to help fire eagle get out the door and support Yahoo Live. Clearly we weren’t actually allowed to go off and fail 9 times out of 10 in the process of coming up with a crazy new and important idea.

What i find truly ironic about the process is we did end up coming up with a pretty damned good idea. That of using oauth and webservices to broker information between other applications and services on the web while leaving the user in control and protecting privacy. Were we the only ones who came up with it? No, innovation is about a community of thinkers, there were many people involved who work at many places including twitter, pownce, magnolia, six apart, wesabe, flickr, many others, and of course yahoo and google. You see it now with interest in federated social network, the gnip project, and other stuff which is still in the works.

Shipping a product and being innovative are often not compatible goals. The innovation happens in the downtime, then once an idea has gelled, the innovation stops and the work of creating production code begins. Sometimes it’s the same folks, most of the time it’s different folks. Often there is a long gap between the ideas being created and somebody building something useful on top of them.

We need recessions and underemployment of the innovators for them to have time to do their work. In a more ideal world, we would support this just like we should support having artists in society. Until we change it, we’re stuck with dumb brutish capitalism which has lucked in to creating these spaces for innovation.

Update: I fixed some of the spelling and grammar problems, thanks. :)



Group Forming, Indymedia Crediblity, and the Forming of Asambleas in Argentina


I find the process of how groups form and articulate themselves to be very important as we settle in to a networked society. I think if we can create systems which take advantage of this end to end network, we can use this to create new social organizations which are more egalitarian. The ability for people to find and associate themselves with other like minded individuals easily across geographic, national, and even potentially linguistic barriers is amazing. Now that the dot com boom has died down we can the get to fun part of using the net to do amazing things instead of just using it to sell more ads.

With indymedia we’ve faced a real need to tackle the problems of forming groups. We’ve got a news philosophy which embraces everybody’s ability to become their own journalist. The problem is that some people write what seem to be conspiratorial rants or espouse political perspectives which are off the wall.
In the American news media you know a story is credible if the New York Times runs a story on it. Once that happens the story gets picked up by other papers, radio stations, websites, and the TV news. Because we represent both a politically dissident perspective and advocate a model for the news with is fundamentally participatory we need to develop a different model for what is credible. For example, Slashdot has one model. They take hundreds or thousands of potential article submissions a day and have a group of editors who pick the most interesting or important 25 of them for each day. With indymedia sites, we have something vaguely similar. Each local site has an editorial group which makes decisions on which stories to feature and promote. Those go in the center column. All of the other stories stay up on the site and are listed as being news, where as Slashdot deletes them. There are still serious problems with the way we do things. When you look at an article you have no way of knowing the history of the author, what other people think of this or the author’s work, or generally assessing the credibility of what’s stated in the article. With the New York Times you don’t need to know the author. The history of the institution embodies all the credibility. With indymedia for us to develop credibility we need to realize that we can’t have the single monolithic institutional credibility that a traditional news organization has. We advocated many people becoming involved in the process of journalism. We often don’t have any connection with, nor have we even met or confirmed an identity of the people who submit news articles to our site. The solution I’d like to come up with is find a way to allow groups of users to form and let them to the work of determining specific credibility. With EBay a buyer or seller has credibility ranked based on their history of delivering the goods. With news credibility isn’t such a one way street. What may be credible to me is both irrelevant and totally non-credible to you. The thing that is true is that these opinions about what is credible and what isn’t tend to be held by groups. For example one group of people might think that any article about Palestinians dieing under the Israeli occupation is ipso facto anti-semitism. An opposite group would say that all stories of Palestinian suffering and massacres are legitimate and credible. A third group might find credibility and relevance in some stories and not others.

The point of this is that news and what is a credible telling of truth is relative. For indymedia and other projects such as p2p journalism to work we’ll need to find easy ways for groups to form opinion and articulate it about the work. Those opinions will then need to be made easily accessible to users outside of any of the groups.

For more info about crediblity you can read my posting on Indymedia, Credibility, & Covering Palestine from a couple of months ago.

I’ll finish up with a little story about how groups can form. It’s not directly related to online group forming but I find it interesting to mull over. To understand it you’ve got to get a little background first. In Argentina a year ago the government shut down the banks to prevent a run on deposits after years of recession. In a country where the cost of living was roughly the same as the US or Europe there was overnight a restriction to $1000 a month on withdrawals. The government then after a time converted much of people’s savings deposits in to long term government bonds. This whole process tended to upset pretty much everybody who had a bank account. People weren’t able to pay rent, car payments, or even buy enough food to feed their families. Many companies weren’t able to pay their employees. Economic activity shrank by %70 in one month alone.

Now the point of this isn’t to dwell on the economic problems of Argentina but to understand how groups formed as a result. As people got upset about this situation they would go out on the street in front of their houses and bang pots and pans. It was an “I’m mad as hell and I’m not going to take it any more” type statement. Because everybody else was in the same situation other people would here this and come out of their houses banging their pots as well. Pretty soon you’d get a whole neighborhood standing on the street corners banging pots and pans. One day some of the people got even more desperate and started marching downtown to the presidential palace. When the TV and Radio reported on this, other people got the same idea and joined them. It didn’t take more than a day and the president partially responsible for the mess was forced to resign. His palace and the congress were besieged by the pot banging protesters.

Once they got rid of the president the banking problems didn’t go away. The banks were still locking down their deposits and economy was in shambles. But now there was a critical difference. Everybody had watched their neighbors come out of their houses to join them. They knew they were in the same situation. Now when they came out to bank pots in protests they started talking. People who hardly knew their neighbors before started trying to do something about the problems because the government had failed them. Over the last year these assemblies, asambleas in Spanish, have organized protests but they’ve also helped workers reopen closed factories, setup community soup kitchens to feed themselves and their neighbors, work with hospitals to make sure there they are stocked with medicine and supplies, organize open non-money based barter markets, and generally tried to support their community and keep society functioning.

While the asambleas weren’t organized online, they are groups which formed from a specific constituency, for specific reasons. Then the groups have evolved to continue to serve their members. The most significant thing about the asambleas is they were not planned or organized by some outside group but they really arose organically out of the pot banging protests without leaders dictating their growth. In this way it seems similar to how many online groups for around specific interests or activities.

For more info on asambleas in argentina read an interview i did about them with dru. Dru’s also on the group-forming list although he hasn’t contributed much yet beyond his introduction.