Speed changes everything?

Until recently, I would have answered this question with a 'Yes, of course". But now... not so much.

The thing is, speed does change a lot of things. Let me give you some examples:

Faster search and thereby access to my mails, totally changed the way I deal with them. In the past, they got filed away neat and tidy in folders. Now, I don't care. If I ever need access again, I just search for them and boom, there they are. Same goes for files and bookmarks and my notes. I trust the servers they lie on, to find what I'm looking for.

Higher transfer-speeds transformed the way I work with data. Keeping some "installation-files" in case I ever need them again? Don't do that anymore. Copying files between computers, especially with my notebook? Not doing this anymore. Anything I need is accessible via Dropbox or my own central server. Even really big files aren't a problem. A single file that can't be streamed is almost never bigger than a couple of dozens of megabyte and if, in rare cases, I need something that is maybe a gigabyte or two, it means waiting minutes, not hours.

Now, back to why I'm writing this post. As you know, I'm developing "Krikkit for iOS". A friend of mine is doing something similar, but with a different approach: OPD. The main difference is: While I'm trying to do most things locally on the device, OPD is a pure web app, that has to do all the heavy lifting on a server. And the second big difference: The OPD-guys host all the data by themselves (although it's technically AWS, but for the sake of this argument...), while I'm trying to rely on free, publicly accessible services like OPS, Depatisnet, Google (patents) and a couple of other services.

So, I had to ask myself:

Which approach is better?

The first question that comes to my mind is: "What can they do, that I cannot?"

There are two main things:

  1. Being faster
    They can (in theory) be at lot faster than my app. Not interface-wise, but with an initial answer to the users question.

  2. Do more with the data
    Having all the (raw) data at your disposal, gives you a lot of freedom of what you can do.

Now, let's have a closer look at these two advantages (that come at a price, but more on that later).

Being faster

The premise of Krikkit for iOS is finding relevant patent-information. If you do a full-text-search, having (near) instant results is crucial, as searching this way is a very recursive process. You enter something, get a result, quickly scan over the result, change the search, do it again.

The thing is: People don't search that way when it comes to patent-information. And that isn't just a hunch, there is hard evidence to prove it.

Krikkit for iOS has two distinct search modes: One freestyle search with a single input box, and a more "detailed" search where you can select search-fields, enter date ranges, use proximity operators and use AND/OR to connect search terms.

Krikkit currently is in a closed beta with about 300 active testers. 100 are employees of our customers, 100 were selected via XING as being "interested in IP" and 100 were found via postings in some iOS developer forums. In case anybody wonders about the >100 testers: The answer is, Enterprise signed IPAs.

This beta, and the monitoring that comes with it, gave me some pretty interesting insight. All the data is from the last 3 weeks.

Total number of searches: 14.001
Number of successful (non null) searches: 12.818
Searches via detailed-search: 9.938
Average result set: 94
Average time to first result: 3.7s

Average time spent on browsing search-results: 2.2 min
Average number of detail-views per search result: 10
Average number of marked patents per search-results: 5

Let's have a closer look at these numbers.

More than 75% of all searches were conducted with the "old-school" search form, although it is not the standard option. Why is that? My first idea was: The results from the simple search are so bad, that people use the alternative and stick with it, because it is slightly better. That wouldn't prove anything. So I had to dig a little deeper and started browsing the sessions logs, which essentially show me every little thing, someone does in the app. And that's where things began to get interesting...

As it turns out, people started using the simple search and got a result. But then, they switched to the detailed search and refined their search. They filtered by assignee, country of origin and dates. And after that, most of them, never touched the simple search again. They merely used it as a starting point to get a better understanding of how and for what to search.

The average time spent on looking through the results, is more than 2 minutes. That's a long time, but is in good correlation to the 10 opened detail-views (which show you the patent in detail). The quality of the results also seems to be quite good, as the ratio of 2/1 for documents marked as relevant, shows.

So, what?

People give drafting their searches a lot of thought, so the number of documents to look through, goes down. And if you spend 20 or 30 seconds of your time, putting together a good query, you don't mind if the answer takes 3 or 4 seconds, especially if the result is good. Heck, even if it is 10 seconds in some rare cases, that still wouldn't be a deal-breaker. I guess, you can put it this way:

The acceptable timespan of waiting for an answer, directly correlates to the time used for formulating the question and the subsequent quality of the answer.

In conclusion, there is absolutely no need for an instant answer. Sure, it'd be nice to have. But the price I would have to pay, is too high. Maintaining my own set of data is just too expensive. Whether it's the cost for AWS, the cost for the data itself or just the manpower to keep all this running.

Now, does speed change everything? Yes and no. It does change a lot of things, but sometimes it's just not necessary, and essentially, too expensive. I guess what I learned (again) is, to look behind the initial "that's easy, I know the answer".

As this post already got 2 cups of coffee longer, than I initially planed, I'll have to save commenting on "Do more with the data" for a later date.