-
Website
http://paulbuchheit.blogspot.com/ -
Original page
http://paulbuchheit.blogspot.com/2009/01/communicating-with-code.html -
Subscribe
All Comments -
Community
-
-
Top Commenters
-
Popular Threads
All Comments
http://almaer.com/blog/the-genius-behind-the-go...
Cheers,
Dion
Thanks for what you do. I'm a bit addicted to all your weird projects.
Prototypes make a connection between buyer pose and user pose. A few hours of coding can be work a few weeks of talking and presenting. And it can be a make it or brake it for an idea.
A difficulty I find with prototyping is the "ugly and hackish" code: one can start again, but it's difficult to read the old code (for clever algorithms and ways around tricky problems) and I end up wishing I'd spent more time designing modularity; less time hacking. But that defeats the purpose of a prototype...
Therefore, I'm curious about how the modularity of GMail evolved as you rewrote - for example, did the module boundaries change much?
http://blog.mikevosseller.com/2009/01/communica...
You're totally going about this the right way. Great post.
We went around on this on and off for a couple of weeks, but there was no code to disagree about, so nobody won the argument. Now we're doing the JS-like thing...
Your point is well-taken.
However Robert Scoble's comment encapsulates the problem perfectly; coding is harder than producing slides, and there is a bigger barrier to adoption of coding than there is learning to use power point or it's ilk.
What's key here is including developers early on in product life cycles so that mock ups and prototypes can be created; and giving those coders the opportunity to do so and recognising the value that it brings.
Google ads are usually pretty decent, but Gmail ads ... well, the only time I've ever clicked one is because it's just so hilarious, I have to find out if there's something relevant hiding behind it (there wasn't).
Though I guess from Google's point of view there's absolutely no difference between "ads so good people click them" and "ads so hilariously terrible people click them". :-)
But what about all those ideas and products that, by there very nature, are not immediately "hackable"? What if you are trying to grow the company, by creating a new product? Say, for example, that you have an idea about making music on the web searchable, by rhythmic tapping. You want to enable users to tap out a rhythm, and then your search engine will find songs with similar rhythmic structures. Would you say that code, rather than two powerpoint slides, still the best way to convince others of the usefulness of such a search engine?
Well, in order to write the code that makes this work, I first have to write some complicated digital signal processing code, to find periodicity in a music signal (beat tracking). I then have to write a metric levels analyzer, to determine which of those beats are downbeats and upbeats. Then I have to further do some statistical machine learning work, to tease out rhythmically salient onsets from syncopations. And all that is just for one song. I then have to write a web crawler, to find all the other songs on the internet. On top of that, I have to write a digital signal "parser" for the user's input. Since common user's tapped query isn't going to be as rhythmically accurate (evenly spaced, etc.) as a professional musician's, I am going to have to write an inference module that determines the most likely quantization of the user's tapping input.
So if I were at Google, and trying to convince someone using code, rather than using two powerpoint slides, it would probably take me 1 year.. 1.5 years.. maybe more.. of 20% time (using only one day a week) to put together all this code into your suggested "minimally impressive demo" form.
There is just too much overall work on the basic subsystems for anyone to just "hack something together" the way you can with web text and ads, or with friendfeed streams.
So it seems like what you are saying is that this sort of communication and convincing others of unpopular ideas only works for incremental improvements to a company's existing code base. You had the text ads code base. You had the web page crawler code base. You just needed a few unix pipes to put 'em together.
But when you're talking about an idea that fits squarely within a company's overall mission statement (organize the world's information.. which of course includes the world's music information), but is a radical departure from any existing code bases, what is one to do? Does 20% time really still hold the power to produce the necessary big change from within the company? It appears to me that it does not.. because of the extraordinary amount of time it would take to put together such a demo, using only 20% of your time. If one had 100% time available, that would shorten the development to 2-3 months, which isn't so bad. But when you're spending over a year on a single 20% time project, just to get the first minimally impressive version up and running, it seems to me that you're going to need a very strong manager as your champion, to make sure that your 20% time really is respected, that you aren't politically sidelined while you work for all that time with nothing to show...and that you are left alone for such a long time, to make it work. From what I've heard from friends at Google, there are not a lot of managers like that. Maybe in 2001 there were. But today the pressures are different.
So it appears that there is a whole class of interesting ideas and problems that can never be explored in a 20% environment, simply because they can't be relatively quickly "hacked" in the way you are suggesting.
Thoughts?
I agree with you 100% that people often look at the success of something like Gmail or 37 Signals and want to generalize the case that all software development can be done by prototyping and RAD. Obviously to prove that your idea would be functional, you have to go a long way towards the final implementation. I think your example, however, misses the point that Paul was making about user's liking a feature/idea based on the UI. if your goal is to get people to test out the UI and see if your idea is a good one, you could probably just slap something together that shows an acknowledgement of each beat and then after a few seconds of inactivity spits out a pre-populated list of songs. There is definitly a difference between a proof of concept and the more general 'would anyone even want to use this'.
But the real power of the system lies not in the interface. Remember, Paul didn't just mention Gmail. He also mentioned AdSense. And with AdSense, the power of the minimally impressive demo/prototype is not the UI.. it's the relevance of the ad to the text. Let me quote Paul:
"However, we needed a way for Gmail to make money, and Sanjeev Singh kept talking about using relevant ads, even though it was obviously a "bad idea". I remained skeptical, but thought that it might be a fun experiment, so I connected to that ads database...[snip]...I then released the feature on our unsuspecting userbase of about 100 Googlers, and then went home and went to sleep. The response when I returned the next day was not what I would classify as "positive". Someone may have used the word "blasphemous". I liked the ads though -- they were amusing and often relevant. An email from someone looking for their lost sunglasses got an ad for new sunglasses. The lunch menu had an ad for balsamic vinegar. More importantly, I wasn't the only one who found the ads surprisingly relevant. Suddenly, content targeted ads switched from being a lowest-priority project (unstaffed, will not do) to being a top priority project..."
So in this AdSense-to-Content demo, what was necessary to convince people was not the UI. It was the quality of the underlying information that was being mashed up -- how relevant could you really make it? Initially, even though the idea had been floating around about placing ads next to full-text content, nobody really believed that it would work all that well, and were even hostile to it, until Paul tried it. Once he tried it, and people started to see ads for sunglasses and vinegar alongside related content, then they were convinced, and the project became top priority.
Likewise, for my rhythmic tapping song search example: Yes I could easily mock up an interface in a day. But what is compelling about such a search engine is not the interface, but the results that one can get from that interface. Just like other Googlers had their "oh wow" moment when they saw the sunglasses ad, a song tapping demo would need to be live enough to demonstrate that "oh wow" factor, enough to surprise and amaze the person seeing the demo. They need to be able to tap something themselves, and then have the system come back with a song that they weren't expecting, but that relevantly fit with their rhythmic intention. All of the subsystems that I mentioned earlier would absolutely have to be working, first, in order to make the "interface" demo compelling.
So we both agree that RAD-based Gmail or 37Signals development cannot be generalized to all types of software development and search engine building. But Paul's point, in the post, was not just about rapid prototyping. It was about the idea that having 20% time at Google gives you the ability to work on ideas that everyone else things are worthless, and then prove those ideas to everyone else, through code.
And so my main question to Paul remains. Paul gave an example of how and why 20% time works, if the feature/idea that you're working with is RAD-able, is easily hacked up. But what if you're feature/idea takes 1.5 years of 20% time, because it involves deeper thinking, harder unsolved problems (e.g. the quantization of user tapping intentionality), and fewer existing code bases? Can one still manage to develop *those* sorts of features/ideas, the non-RAD features/ideas, in a 20% time environment? Because the only examples that Paul gives in his post are of RAD-compatible features.
And again, if one cannot, then is there a whole class of interesting ideas/features that will never go explored at Google, because they do not fit the 20%-time paradigm.
You've describe a system vastly more complicated than one needed to get initial feedback. You also describe writing everything from scratch, whereas he repurposed clearly inappropriate stuff and then went back and did things right for the ideas that had legs.
You could do a basic proof-of-concept of your idea in a week. You don't need a crawler, any machine learning magic, or DSP voodoo. You need 20 MP3s of well-known songs, one button, and enough code to record timestamps of button presses. You record yourself tapping out the rhythms a bunch of times, and write a little best-match code. Then show your sample users the list of 20 songs and tell them they can start playing one by tapping out the rhythm. You can even cheat a little by picking 20 songs that your best-match code can easily tell apart.
Then you put that in front of users and see how they like it. If they shrug and move on, then you can write off the couple of days it took you, or you can try another approach. But if they're excited, now you know you have something. Something worth investing in.
Edison didn't get the light bulb right without trying thousands of different filament designs. A lot of innovation isn't driven by having one great idea; it comes from having a lot ideas, and a way to sort the great ones from all the rest. The cheaper your per-idea evaluation cost, the more likely you are to succeed, and GMail is a great example of that.
Oh, absolutely, William! I completely agree. :-) But the way one approaches things is often as important, if not moreso, than the solution one ends up with. Let me explain:
To begin with, let me just say that this rhythm example is just that, an example. I'm not going to go out and build it, because many other people, outside of Google, already have.. five or more years ago. I chose it as an example because I know these people, and I have seen them demo their systems, and seen the reactions that they've gotten.
The reactions tend to fall into two camps. In one camp are the people who just "get it", and don't even need a powerpoint slide that tries to convince them about the idea. In the other camp are the people that say, "Yes, I see the system, see that it works in a rudimentary way. But I need you to clearly show me that I'm going to be able to discover something interesting with it. Yes, yes, I see that you can tap, and that songs appear. So what? Your collection is so small, that it really doesn't tell me anything."
For the former camp, I don't even have to build a prototype. I can do what Chris Duesing, above, suggests, and just fake it. Heck, I know powerpoint well enough that I could probably create a text-less powerpoint slide, and fake up the whole thing by importing mp3s, creating a few squares and circles, and using their whole animation/trigger action scripting to make it look like I have a working prototype. That's an ugly notion.. faking an interface using powerpoint. But I could do it. But even if I do it, that will only satisfy the former camp.
The latter camp, on the other hand, is going to see the two week-old working demo, with 20 songs, and still shrug their shoulders and say "Oh, that's neat.. but.. so what? It's only 20 songs. How do you know how well it will do with 100,000 songs? How do you know that what you're discovering is really interesting, or whether you're simply cherry picking the best query for this 20-word set?"
It's not that they've shrugged and moved on, because they don't like the prototype. They like it well enough. They just aren't convinced that it will work with 100,000 songs the same way it works with 20. They aren't not convinced, either. They just don't have enough information to be able to make an informed decision.
By analogy: Imagine if someone came up with a new idea for a web search ranking algorithm. They mock up a little prototype, code their algorithm, and then set it running on an index of 20 web pages. Yes, I said twenty. "Look, Larry and Sergei!" they might shout, "the correct document appears at the top of the list every single time! Isn't it amazing!"
That person would be laughed halfway to Kansas. Right?
So what is needed to build a convincing demo is those 100,000 songs. And to get to that scale, in a way that makes reasonable sense, you really do need machine learning magic and DSP voodoo. Maybe I could skip the crawler step by getting fifty of my friends to let me copy their mp3 collections (shhh! don't tell the RIAA), just for the purpose of this demo. But you can't even get the system up on its first legs, at that scale, without having to do some of the other magic.
Yes, Edison didn't get the lightbulb right, immediately. And I'm not saying that the builder of this prototype would have to get every single piece right, immediately, too. They could build a system that does beat tracking correctly on hard rock pieces but has a harder time on violin solos of music from the romantic period (which is actually true of most beat trackers). They could build a metric level detector that is off by half a phase on lots of songs, i.e. it might wrongly shift level 1 to level 3, and level 2 to level 4, and still work well enough to demonstrate the proof of concept. I never said you have to get everything in every little subsystem 100% correct, the first time around. But every little subsystem has to at least exist, to have a system that works on more than 20 songs with a cherry-picked query.
And so that's the way I am approaching things, because of my experience in seeing other people (honestly, not myself, but other people) present this type of rhythm-finding musical search engine. The reactions either fall into "yes of course that's a good idea.. I don't even have to see your toy system.. let's go for it" to "yes, I see the working system, but the working system is not scaled enough for me to be able to decide, one way or the other, if that idea is good or bad yet. I really need to see how it works on real, full-scale data."
That's very different than the Gmail idea. Once people saw Gmail, saw the interface, they knew that Paul didn't have to know right away how to build this system at scale. Because everyone intuitively knew that it could be done. Email processing is parallelizable, etc. So there was no question on that part of the issue. With rhythm search, on the other hand, the true value (or perhaps even lack thereof) only starts to become apparent once you've built to scale, and with a reasonable degree of rhythmic tokenizing (all the machine learning magic and DSP voodoo).
It is just a fact of life that there are good ideas out there, for which the per-idea evaluation cost cannot, by the very nature of the idea, be cheap. I can't just want into a VC funding office with my new search algorithm and 20 web pages. Sometimes you need the time and the resources to build to scale, and to do the pre-processing right.
So again, my question to Paul is: Do those ideas fit into 20% time. The managerial concept of 20% time is very good at producing results from "spaghetti-against-the-wall" ideas with cheap evaluation cost. But is it robust enough to handle those ideas with more expensive evaluation cost, too?
Because those expensive ideas do exist, and cannot always be reduced. Some idea can. (Re-purposing a text-based adult-content classifier to do a different type of text-based classification is not as far of a stretch or as clearly inappropriate as you might think.) Many ideas cannot.
http://the.echonest.com/analyze/
and more specifically:
http://developer.echonest.com/docs/method/get_b...
Leveraging another company's product may or may not make sense for a deployed product (internal development/maintenance costs vs. licensing costs). However, in a prototype scenario it allows you to bypass building complicated functionality up front and confirm or deny the justification for the product's existence. Assuming the licensing costs aren't prohibitive, leveraging someone else's product should see you through public beta/initial growth phases as well (and hopefully through explosive growth and mass adoption).
You also don't need a large catalog of tracks to start as you suggest- merely a representative sample of music from those who will be seeing the prototype. As for quantization, you don't need to be fancy up front. You can look for beat hits that fall within a reasonable distance from the user's entry (maybe 1/8s on either side, maybe add bell curve weighting if you want to get fancy).
All that said, limiting the scope of 20% projects has the benefit of keeping projects within the general realm of the company's core direction. If there aren't resources to get started on your idea, maybe it makes more sense as a personal project outside of work. Maybe it would even make sense to quit and try to start your own company or find a company where your idea makes more sense. No company can do everything.
What you've described feels very much to me like a lone employee in a canoe paddling east while the ocean liner with the rest of the company heads north. Maybe the canoe is headed north-east, but the goals feel miles apart.
Take Paul's story as an example. Gmail was a logical extension of Google Groups. It was something that had an existing code base that could be built on top of. FriendFeed was more of a tangential product that made more sense as a standalone company (in addition to offering new opportunities for its founders).
To bring things back around, functional prototypes and (from my understanding as an outsider) 20% time are both very much about hacking things together and seeing what sticks. They're both fail early approaches designed to weed out bad ideas (or ones that don't make sense within the context of the business they're created in) and bubble up the good ones.
Ok, in this particular example, the APIs from Echo Nest would be a decent way to get started with the beat tracking. I do actually know the Echo Nest guys (Brian, Tristan), and this is exactly why they released their APIs. But look.. you're bogging down with specifics, where I'm trying to ask a more general question. Maybe that's my fault, because in my illustrative example I did get too specific. But my general question is still.. what do you do when these APIs don't exist:? I think I mentioned above that the notion of rhythmic searching first gained mass popularity in the scientific literature about five years ago. So let's pretend we're having this discussion five years ago. The Echo Nest APIs didn't exist back then, and someone wanting to do rhythmic search in their 20% time would still be at square one.
My basic issue/question still holds, because I just like Echo Nest didn't exist 5 years ago, when people were doing rhythmic searching, so also does some other API not exist today, for some new idea I have today.
So the basic question still remains: Is the Google style of innovation, which is to allow employees to pursue their own wacko, unpopular ideas in their 20% time, capable of producing fundamentally new products? Or is it only capable of producing incremental improvements on existing lines of business?
You write: "Take Paul's story as an example. Gmail was a logical extension of Google Groups. It was something that had an existing code base that could be built on top of. FriendFeed was more of a tangential product that made more sense as a standalone company (in addition to offering new opportunities for its founders)."
So it sounds like your answer to this question is No. If there are no existing APIs already within the company, that offer quick repurposing capabilities, then 20% time is incapable of producing anything outside that realm. Google cannot innovate in that space, under a 20%-time managerial regime.
That type of answer is what I'm after. I'd still like to hear Paul's opinion. But yes, that is what I am trying to understand.
Because, in my mind, the whole purpose of 20% time is exactly as Paul says, above:
"This is also where Google's "20% time" comes in -- if you want innovation, it's critical that people are able to work on ideas that are unapproved and generally thought to be stupid. The real value of "20%" is not the time, but rather the "license" it gives to work on things that "aren't important"."
So if the whole point of 20% time was to enable employees to work on exactly those sorts of ideas that were generally unapproved or thought to be stupid. To go with your canoe analogy, I thought the whole point of the 20% time was to give the employee license to paddle northeast, when the rest of the company (ocean liner) is paddling north. Right? Isn't that the whole intention? But if you're telling me that Google 20% time is incapable of producing something like the Echo Nest, from within its walls, then that really seems like an extremely limited model for innovation.
It's not like rhythmic music searching is that far away from Google's mission statement. We're not talking about someone spending their 20% time analyzing the structure of stem cells or something like that. We're talking about a search engine. Musical, rhythmic searching, yes. But still a search engine. That's right up Google's alley.
But I'm still waiting for Paul's opinion. Paul?
I guess I come from the school of thought that iterative innovation produces better results than big bang innovation.
Again, having never worked at Google, maybe I don't fully grasp the concept. To return to the boat analogy, it seems to me like projects headed as far as NE/NW would be valuable regardless of their length. Head further to either side and you're doing more to pull away from the goal than towards it. That may be fine in short bursts, but it seems like it would be hard to sustain.
At least that's what I've seen for every "20 percent time" project I've heard about. Even MacFUSE (which seems pretty tangential on the surface) fits with Mac team's mission "to contribute to the Mac community and make the Mac OS X experience better for users and developers." from the Official Google Mac Blog. It also builds on a pre-existing foundation.
Maybe there are a bunch of 20% projects I've never heard of that are bigger leaps from Google's current offerings. But isn't that exactly the problem? I haven't heard of them. If you want visibility for your project, it's going to need to fit within the bounds of the organization you're in. 5 years ago, beat detection was a much more academic pursuit that probably would have made more sense in a university or at a company like Fraunhofer.
Perhaps there is a better way to phrase the answer I am seeking: Is there, inherently built in to the notion of 20% time, an assumption that iterative innovation produces better results than new pathway, leap innovation? Is 20% time really, as you seem to imply, a mechanism for insuring that employees stay focused on iterative iteration, and don't spend any time at all on leap innovation?
Paul?
To slightly switch tracks for a moment: Where I think I will disagree with you is when you say "The reason I brought up The Echo Nest was to illustrate that for almost any computing problem, there's going to be something else to build on."
I think the Echo Nest example illustrates exactly the opposite point, which is that if you're really trying to do something interesting, something that you're competition isn't doing yet, those APIs are not going to exist. The Echo Nest API was released.. when.. like six months ago? And searching music by rhythmic similarity was popular when.. like five years ago?
To me, this illustrates the fact that for really interesting, innovative problems, the lower level APIs trail the high level projects by a significant amount of time. And it also means that if you are a company that is going to wait for someone else to implement the low level beat tracking API before you start building your very first demo of the higher-level project, then you're far behind the curve. Because the reason why someone else is creating that API is because there is already a need for the higher level functionality, the rhythmic similarity engine.
So I suppose it's fine to wait until someone else exposes the lower-level API. But if you competition has already built out the product by that time, because you had to wait until the API was available, then it seems like you're (as a company) not doing a very good job of innovating.
I'd still like to get Paul's thoughts on this, though :-)
Because if there is a 4.5 year lag between when when you want to do something like rhythmic searching, and when someone else makes low-level APIs available, then it seems to me you lose the ability to communicate with code. You're probably better off with 1.5 powerpoint slides, illustrating the idea. Otherwise, you're going to be spending a year or two writing all the code that you need to, in order to do your communication.
So I'd still like to get your thoughts on this.
Are those just products that will simply never get built?
One suggestion: friendfeed's current format excels as a small footprint for 'reader' like fast consumption. The added personality of seeing a user's face or icon can create a better association with the personal connection to the feed item, and even more recognition than the username or service icon. It would be great to offer this as a viewing preference, but not a requirement since it adds visual noise to the mix.
Also, how does someone as a UX designer get comfortable with hand-coding/prototyping at this level? I'm starting with jquery and dreaming of being a fast-prototyper. thanks!
i completely agree with you that a working prototype is far more convincing than arguments. i believe ideas without "code" is a nothing more than fantasy
It may be a stretch to compare our two industries, but I truly think you've written a axiom that is relevant in any and all. Many thanks for the great read.
FriendFeed is (and will become even more) THE integration tool for 'power users' on the web - integrating information streams with no programming efforts (or with a little effort; built your own twitter like UI) as shown.
This convinces me even more to stay on the choosen track: I am using FriendFeed more and more for scenarios where I have used different tools in the past. Examples:
- tracking groups of high volume twitter users (using rooms) - not using tweetdesk and similar tools; I prefer browser only tools; it makes switching from one PC to another much easier
- embedding FriendFeed rooms in other sites and contexts.