Sunday, May 28, 2006

What's Missing in Web 2.0?

Web applications replacing the desktop? Here are two problems we need to solve before that can happen.

I'm sitting in the New York Café in Budapest, Hungary. It recently got redone after being in a sorry state since 1956, when it had a run-in with a Russian tank. It happens to be just two blocks from the college where my dad used to teach, so I remember walking by it when I was a little kid. Back then, it looked unceremonious. Now, with all the shiny gold plating everywhere, this place looks opulent and luxurious, if not a bit over-the-top. A smart girl once remarked that I have the tendency for big-picture philosophizing when surrounded by luxury, so here we go.

In 1995, Netscape offered the vision that one day, everything will run inside the browser. The operating system would be just another layer of computer architecture. The golden copy of your files would reside on central servers. Your word processor, spreadsheet application, and e-mail client would all be displayed inside a window with a meteor shower animation in the top-right corner.

Well, that didn't happen. Microsoft bulldozed Netscape and for a while, blue e's and rotating globes are still everywhere. But the vision is still alive.

Moving applications and data to the network makes a lot of sense. For storage, Google and Amazon can offer higher reliability at lower cost than home-brewed solutions. In fact, consumers often completely forget to backup their data. For applications, Firefox is available for all popular operating systems, and may easily become a popular application platform with some massaging. Imagine being able to log on with any computer in the world and having your data and favorite applications instantly available, just like you have it at home.

We're not there yet. The Web 2.0 boom introduced many applications that were formerly desktop-only territory: Google Calendar, GMail Chats, Kiko, Writely, Pixoh, and DabbleDB come to mind. But imagine spending an entire workday using only your browser, without any desktop applications. For anyone who uses more than just e-mail, that day would be a very unproductive one.

I believe that before web applications can truly make the desktop obsolete, we need to solve two important problems. Both might seem a bit surprising at first. Even though Javascript is a mess, neither of these points concerns building better AJAX UIs or frameworks to write them. (It's pretty obvious that client-side scripting could greatly profit from cleaner, more cross-compatible features, but there are people more qualified than me to discuss that.)

One of my pet problems is engineering, not science; the second is science, not engineering:
  1. Offline Access
  2. Encrypted, searchable storage

Problem 1: Offline Access

We won't be able to cover the entire planet with always-on Internet access. We need a simple, standardized way to run web-based applications offline.

Earlier this year in Mountain View, I saw huge boxes on lampposts, designed to cover the area with WiFi. This is a great idea for populated areas, but don't extrapolate too far: Even if these boxes were a lot smaller and had much higher range, even if UMTS became free and universally available, even if Connexion receivers were reduced to the size of cell phones – even then, we wouldn't be able to cover the entire planet with Internet access. There will always be a long, dark train tunnel that will spoil the master plan. [1] On top of that, I doubt all this infrastructure could be provided for free, and my tolerance for paying $6/hour for spotty T-Mobile Hotspot reception is low.

An online application becomes completely useless the moment you lose connection. That's why we need to find a way to make online applications work offline.

This isn't the first time I complain about this. In a previous post titled "How do we solve the offline problem?", I described the technical challenges and options.

Today, it seems to me like the most natural way to do this is via a mechanism similar to extensions in Firefox. [2] First, take a browser you can run on any platform, then add a mechanism to easily create applications that perform three things:
  1. Caching: keep a copy of your online data locally.
  2. Presentation: Display the UI in the browser, either by faking it or by actually running an application server locally.
  3. Synchronizing online and offline data.
In essence, you would want to re-create the presentation logic of web applications offline, in the browser. I have to admit that this is a very ambitious goal: You can't just take your server code and distribute it to clients: They may not be able to run it in their environment and you may not be too enthusiastic about giving out your server-side source code, however obfuscated it may be.

Creating offline-enabled web applications will take a lot of work. But without an offline option, web apps will never overtake Microsoft Office.

Problem 2: Encrypted, Searchable Storage

We need to devise a scheme where we can store encrypted data remotely, with the ability to quickly and efficiently search it.

Search is a killer feature. The power you gain from being able to search your own data as quickly as you can search the web is immense. The more data you have, the more useful search becomes.

Recently, there has been a proliferation of new online storage providers, and there are rumors that even Google wants to get in the game pretty soon. For a list of current players, check out this comparison chart. Some of them, like Omnidrive, offer encrypted files, but some aren't even truly secure from a cryptographic perspective: XOR-ring data with the user password doesn't really help.

The key point is this: Many users won't completely trust their storage provider, and won't store the golden copy of their files online, unless they're really, positively sure it's encrypted, and no one else can read it. As an extra benefit of encryption, the storage provider won't even be able to hand data over to the DOJ for their 'statistical evaluations about children accessing pornography.'

That's why encryption should be default. It should take place on the client side, and storage providers should never even see user data in plaintext. [3]

Storing all files in an encrypted manner has a huge drawback. The storage provider won't be able to index and search them anymore. Unless, of course, you found a way to encrypt data but still be able to search it, without losing security. And that's exactly what we need.

Obviously, I'm not the first person to think of this problem, and there's plenty of research on this topic. For example, there's this paper by Song et al. titled "Practical Techniques for Searches on Encrypted Data". You can safely skip to Section 5.4, where they discuss building indexes. Their solution is relatively simple, but requires two round-trips to the server and the storage provider is still able to learn some information about the documents from the user's access patterns. But that seems tolerable.

Two side notes: Any client software for accessing encrypted software would need to be open source, at least in the core parts. With a closed-source client, how would your users know you're not really sending along your encryption key? Also, while it looks like I'm talking exclusively about online storage, this also applies to all data stored in a web application. Wouldn't it be great if Google Calendar didn't know the plaintext of your appointments but sent you an encrypted record which is then decrypted and rendered in your browser?

My opinion is that encryption should be standard in any kind of online storage solution. Without search, however, online storage is useless.

Conclusions

My speculation is that the current crop of web developers will at first resist solving problem 1, because they're too much in love with their server. Also, someone needs to come up with a good example solution that everyone else can copy - much like GMail and Google Maps first came up with neat uses of AJAX. This may be very hard, as it may require hacking deep inside the browser.

As for online storage, I believe it is an important problem. But will users appreciate this functionality? Not before the media makes a huge story out of teenagers hacking into some celebrity's online picture collection, or Chinese students getting arrested at a dissident meeting they had entered in Yahoo Calendar. With some public awareness for the issue, I think people will flock to the provider offering encryption, and they'll be happy to see a search box.

--

Acknowledgements
Thanks to Markus Egli and Bálint Miklós for reviewing drafts of this.

--

Footnotes
[1] I guess only someone who lives in Switzerland would come up with a train tunnel as the primary example.
[2] Dear readers, if you have an idea about how this can be done with current Firefox extensions or other, existing technologies, let me know.
[3] A successful online storage solution needs far more than encryption, the most important aspect being extremely good desktop integration. Also, with any encrypted storage solution, we'd need to train the user to keep offline backups of his encryption key: Without the key, all his data is lost.

Monday, May 15, 2006

Google Trends Measures User Interest

A tip of the hat to my friend Douwe Osinga and the team for bringing to the world Google Trends, which has been the subject of much attention in the last days.

Searches signal user interest. The reason why users enter something into the Google search box is because they are interested in the topic. That's why Google makes money: If you're interested in buying something, you'll enter it into their box and they'll give you plenty of ads to click on. Ka-ching.

Google Trends will be successful for the same reason that Alexa Traffic Rankings is successful. Marketers use Alexa to get imperfect estimates of traffic data - who really uses the Alexa Toolbar, anyway? -, while Google Trends can be used to get almost-perfect estimates of interest in topics. The results are imperfect because not everyone uses the Internet, not everyone uses Google, and Trends only takes a sample of the total search data.

Some examples: A comparison of the three recent tech homeruns: Netflix, Skype, and Firefox; the popularity of sex in Pakistan; and Microsoft vs. Google vs. Yahoo.

Wednesday, May 10, 2006

IndieKarma

I'm trying out IndieKarma, a new scheme for micropayments on the web. Sign up for IndieKarma and put some money on your account. Every time you then visit a website with IndieKarma enabled, you automatically "donate" $0.01.

Many webmasters are currently using AdSense or similar schemes for making money off of web traffic. My view is that charging people outright would be a better, simpler, and more honest solution, as long as the price is fair. One cent seems like a good deal, especially if the scheme is voluntary.

Clearly, IndieKarma suffers from the traditional chicken-or-egg problem. Also, if the idea really turns out to be great, such a site would be simple to implement, so there could be a rush of competitors doing the same thing and cluttering the space.

I originally read about the site on kottke.org. He has an interesting request: He says they should introduce the option of charging different amounts per site. I disagree. The iTunes model – one price for all songs, err, sites – seems a lot better. However, they do need to add an "undo" button for bad sites and offer standard ad-format donation images instead of a huge, intrusive banner. Until then, click here. I'll report on the proceeds in a later post.

Monday, May 08, 2006

A Weekend in Göteborg

Babies and Toyota Priusses – those are the lasting impressions of my weekend trip to Göteborg (English: Gothenburg). I was visiting my friend Peter Mathoy, who is a business student at ETH Zurich but is on a semester abroad in at Chalmers University.

Soon after I got off the plane, I noticed like there were pregnant women and strollers everywhere! This isn't a coincidence, though: While the average woman in the Switzerland bears 1.43 children, the number in Sweden is 1.66. The slowing birthrate is a huge problem in the European countries: The young need to finance the state-run social and pension systems, and in order to achieve economic growth, you need workers. To keep the population size constant, you need to figure out a way to increase the birth rate to 2.1 (the so-called "replacement rate") or spur immigration – something that EU countries have been reluctant to do. After a cutback in the 1990s, the Swedes seem to be at it again – they provide significant support to working women with babies: free child care, time off after birth for mom and dad without the danger of getting fired, and so on.

A similar state-run incentive system seems to be the major cause for the huge number of Toyota Priusses in Göteborg's streets. Apparently, It's not unusual to see dozens of them parked in the same street. The reason: Hybrid car owners get a massive reduction in car taxes and free parking. Incentives and special tax reductions seem to be a popular tool of government in Sweden. A student I talked to favored tax cuts specifically for students – but is it really smart for the government to cater to particular interests, thereby creating a nation-wide loophole bonanza? I wonder if the high birth and hybrid rates could be replicated with more market-driven model.

The students I talked to seemed pretty relaxed. Most conversations at ETH Zurich at contain some bickering about the intensity of the workload. The relaxed attitude, however, is understandable: Students can re-take exams as many times as they want. Also, they receive 7000 Swedish crowns (US-$ 960) – half of it as loans – each month by the government. Still, I found it quite amazing was that everyone seems to be studying something very practical, even the girls. While around 80% Swiss girls seem to immerse themselves in sociology, linguistics, ethnic studies, and psychology, the Swedish girls are all studying engineering and economics. I'm not sure if my sample was large enough, though, to really make such general statements.

Even the coursework seems to be more practical. Anyone who's participated in the ETH-Chalmers exchange program tells me that the courses seem to be focused on technical topics and particular technologies. Very little theory is studied. I'm not sure this is a good thing to do: Twenty years from now, the only thing I'll still be able to use from my MSCS is the theory.

One thing the Swiss and the Swedish seem to have in common, though, is their patriotism. Blue and yellow everywhere. Of course, I had to have my very own Sverige shirt, purchased from the fine folks at H & M.

Pictures are here.