Valleywag just wrote about something that’s been bothering me for a few weeks now.

http://www.google.com/url?sa=t&ct=res&cd=1&url=
http%3A%2F%2Fwww.smellypoop.com%2Fpoop.html&ei=7xAM
R62WC5mIhAPn3I0w&usg=AFQjCNFsw83KR3JmGnnBi_n89GtI
dgWgFw&sig2=Th-qrd3nEEx_368yjwK7cA

Google has stopped linking directly to sites in their main search results and is instead passing each url through a really long, gnarly redirect. This is something Yahoo! has been doing for a long time and it’s pretty lame. You can’t cut and paste links out of the search results any more, and it takes longer to get where you want to go because you have to go through a middleman. It’s a subtle thing, but it’s really annoying. I’ve always believed that the lack of such redirects was a big reason why Google always maintained a lead over Yahoo! in the search wars.

What’s the point of these redirects? They’re there so that Google/Yahoo can accurately track clicks on their search results pages. I previously assumed that Google, because they weren’t beholden to archaic link tracking methodologies like Yahoo!, was using javascript click handlers and ajax to do the tracking behind the scenes. I guess I overestimated them.

This is a bad, bad sign for Google. It means they are, perhaps for the first time, willing to sacrifice the quality of the user experience in order to feed more data to the reports read by upper management. Google wins here (short term), but the user loses. It’s very un-dude.

A change like this could only be made to their #1 money-maker if their fantastic culture and spirit is eroding and giving way to typical big company BS. It means that some VP managed to put his short term goals ahead of the company’s long term success. Perhaps they’ve let too many Paul Buchheit’s get away and hired ladder climbers in their place.

If I owned any Google stock, I’d sell it soon (although, to be fair, I’ve said that a few times before and I’ve been wrong every time).

Note: If this speculation is off the mark, then this is why they should care about their Reader.

The NY Times recently had an interesting piece with a behind the scenes look into the world of Google search and their ranking gurus Udi Manber and Amit Singhal [#1]. One part of this article that caught my eye was where they discuss the “freshness” problem.

Freshness, which describes how many recently created or changed pages are included in a search result, is at the center of a constant debate in search: Is it better to provide new information or to display pages that have stood the test of time and are more likely to be of higher quality? Until now, Google has preferred pages old enough to attract others to link to them.

To solve this problem, the article says that Mr. Singhal thought of a solution that he calls QDF, or “query deserves freshness”. They aim to show fresh results for queries that are topical to recent events.

THE QDF solution revolves around determining whether a topic is “hot.” If news sites or blog posts are actively writing about a topic, the model figures that it is one for which users are more likely to want current information. The model also examines Google’s own stream of billions of search queries, which Mr. Singhal believes is an even better monitor of global enthusiasm about a particular subject.

Cool idea, right? But there’s a question that begs answering. Once you’ve figured out that a user is looking for fresh results, how do you figure out which ones to show to them?

Google’s web search index has a lag time of up to a few days. This may be due to the sheer size of the index and the technical problems that come with re-indexing the entire web very frequently. But, I think it’s because updating it any more often than that provides little further benefit. If they don’t wait a few days, then people have no time to create links to the new web pages out there and provide the page rank algorithm with its secret sauce. In the page rank world, fresh pages have no value. People need time to link (or not link) to them and tell the algorithm whether or not they’re worthwhile.

This means that a new mode of thought is required to index and rank very fresh information. Information for which the usual trust metadata is not yet available.

To fill this void, so called “blog search” has emerged. Blog search is actually just an arena in which “query deserves freshness” is always true. It’s the same problem. Technorati is the leader in this area, but others are catching up. Unforunately, Technorati’s results are often littered with spammy and irrelevant links. They try to mitigate this by assigning an authority score to the source of each result, but it’s not granular enough to get at the real issue (Google doing more frequent indexing won’t really help web search for the same reason). All the other blog search providers have similar ranking issues. So, what are they to do?

The way to solve the ranking problem for fresh information is to analyze the attention streams of people that are consuming very fresh information.

And where do people consume very fresh information? That’s right, they do it in a feed reader.

Google Reader is a way for the big G to get extremely reliable data about which new web pages are worthwhile and which ones are not. If lots of people email a story in a feed to their friends, that’s a clue that it’s interesting. If lots of people star something, yet another. If they tag it, even better. This all happens very quickly, within minutes of an item’s publication. You see, if they get a large enough group of people to consume their fresh information inside Google Reader, then they are acquiring massive amounts of structured, valuable, implicit metadata that can help them to solve the freshness ranking problem.

This attention data is infinitely more valuable to Google than the potential advertising dollars they could obtain from showing ads to Reader users. That’s why Reader is ad free and will remain that way. They just want to know what you’re looking at.

This attention data is why Technorati acquired the Personal Bee and why Ask bought Bloglines. They want at this type of information too. This is also why Yahoo! is really blowing it by not building a proper reader of their own. At this point, they should just buy one (nudge, nudge, wink, wink). [#2]

If there’s one thing I learned while working at Yahoo! it’s that search is the cash cow, the big prize. Everything else matters only in regards to how it can improve search and drive increased search market share. This is probably even more of a truism at Google. Google Reader may have started out as a 20% project, but all that fresh attention metadata means that it will eventually become an integral part of their search platform.

Notes:

[#1] These two get all the attention while all of their hard working, un-named minions get no recognition. I really hated when that happened with Yahoo! Answers in the media. Some big shot VP would get quoted as the “man behind Yahoo! Answers” and half the time it was someone I’d never even seen or heard of before. Oh, and I’m still annoyed with Mr. Manber for stealing my corp. username at Yahoo! and forcing me to use something clunky other than just udi@. When your name is as rare around here as mine is, you just don’t see these things coming. Damn you Manber! ;)

[#2] Yup, in case you didn’t already know, I’ve just built a new breed of feed reader. It’s pretty awesome. You should really check it out. Now. Yes, you. Right now. Seriously. Do it. http://feedeachother.com

In February, I made the tough decision to leave the incredible Yahoo! Answers team and dive full time into a venture of my own. That venture is a new site called Feed Each Other (feedeachother.com). Today, the invite only beta period is over and the site has been opened up to the public. (That means you should check it out and spread the word!)

What is Feed Each Other? Well, it’s your new feed reader :)

If you don’t know what a feed reader is, you can read up on the idea here. In short, it’s a tool that lets you read the latest headlines and content from many different websites quickly in one place instead of visiting each site individually.

Those of you who are already familiar with the concept are wondering what Feed Each Other adds to the equation. The answer is that in addition to being a world class reader, Feed Each Other lets you harness the power of your network of friends and colleagues to help you filter and explore the web in an fun, enlightening, efficient way.

Wouldn’t it be nice to know which feeds your friends subscribe to? Shouldn’t you be able to find new feeds by topic? Wouldn’t it be cool if you could browse feeds related to your subscriptions? Shouldn’t you be able to share things that you find in your reader without clogging up your friend’s email? How great would it be if your reader could automatically point you towards other interesting, like-minded people?

Feed Each Other makes all of those things a reality (and more!). You know what your friends like, and they know what you like. Feed Each Other lets you help each other, and the community at large, find new things and stay informed. I see it as a logical progression in the way that we go about exploring the web and consuming information online. It simply makes sense, and this is why I felt compelled to leave my job and go and build it.

It’s been a crazy 7 months, but I’ve already learned a lot and hopefully the journey is just beginning. I got some great help from my partner in this new venture, the brilliant Dan McKinley, and I’d also like to thank all my family and friends that have been incredibly supportive and helpful. Thanks for all the feedback, encouragement and enthusiasm!

Lastly, I just want to mention that the end of the private beta means that the major functionality is in place and is stable. That definitely does not mean that the site is finished. Expect to see many new features, fixes and adjustments in the weeks and months to come. Hopefully all of that will happen with the help of some great feedback from the people like you, the reader of this post.

So, what are you waiting for? Go and play with it. Tell your friends!

Related Links:

• A more detailed introduction to Feed Each Other and what it offers.
• A special shout out goes to our new Manager of Creative Engagement, Sierra Chappell, for all of her help and her wonderful spirit.
• More acknowledgments and thanks.
• From now on, Feed Each Other specific announcements will be at the Feed Each Other Blog.

Some advice and observations from the past couple of months.

• It’s all fun and games until the lawyers get involved.
• Don’t pick movies with subtitles. They’re really hard to watch while you work.
• No matter how insignificant a feature seems at the time, someone will quickly complain about it if it’s not done right.
• When someone asks, “when are you going to launch!??”, the most precise answer you should ever give is “soon”.
• Google scares me more and more every day.
• When someone uses what you’ve built and really loves it, well… it’s pretty damn cool.

I was reading The Sports Guy’s latest mailbag today and he had this to say about why the big TV networks keep putting up bad NFL programming:

You know how everyone always wonders why networks make so many bad decisions and ignore common sense so often? Well, the problem is every network has too many executives, and when you have a lot of executives, you have a lot of meetings, and if you have a lot of meetings, those same executives feel obligated to come up with ideas for those meetings just because they don’t want the head boss to say, “Gee, that was weird, Bob didn’t come up with a single idea in today’s meeting.”

That leads to people feeling obligated to throw out bad ideas because a bad idea is better than not having ideas at all. And there are times when everyone in the room talks themselves into a bad idea — mainly because they couldn’t think of any other ideas — and once the bad idea springs into motion, everyone starts working on it and eventually talks themselves into the idea.

He later says:

Second, when people have a ton of money to spend, and it’s not their money, they’re always going to end up spending it … even if they’re spending that money like a drunken sailor at a strip joint.

As I read this, the United States Congress popped into my head. If you think about it, this is the biggest group of executives in the land and their main duty is to sit around and think up ideas, for laws.

Well, what if we already have enough laws? Have we properly considered this possibility? These people go to work every day and sit down and think really, really hard about the next piece of legislation to propose. If they don’t add any laws to the book, then everyone calls them lazy. Is it any wonder that the tax code is 16,000 pages long? Is anyone surprised that you can’t move an inch in this country anymore without consulting a lawyer?

We’ve been making the rules more and more complicated every day for 200 years and it’s only going to get worse. As the law becomes more complex, we’re all worse off. Doing business becomes increasingly difficult. Freedoms are eroded.

Maybe the fact this current congress is only naming post offices is a good thing? Maybe it’s time we put a cap on the size of the book of law? For every new law that gets added, an old one must be repealed?

Maybe congress should stop legislating altogether and shift all of its efforts towards executive oversight. At least for a little while. Let things settle.

I just did a search for “Which country has the most complex laws” and I arrived at this essay. You’ll never guess which one he says is the answer:
The world’s least-free country