The Diffen Blog

2 million unique visitors in September

Mon, 01 Oct 2012 19:40:00 -0400

It took Diffen several years to get to a point where we crossed 1 million unique visitors a month. The next million took only 1 month.

After flirting with the 1MM mark a couple of times, it finally happened in August 2012. And in September traffic to Diffen crossed the 2 million unique visitor mark.

Totally awesome!

But we’re not going to get carried away with this success. Growth cannot continue at this pace unless we make significant investments in content, product and marketing. In fact, it is only October 1 today and yet we think traffic in October, November and December will experience negative growth.

And traffic is only a small part of the big picture. So many things matter more than traffic. For example:

What are our users saying about Diffen? With temperatures running high (metaphorically) this election season, some users have complained about perceived bias in some comparisons of politicians. In most cases their feedback was valid and we made corrections.
What percentage of users are using the wiki features to contribute positively to information on the site?
How many users return and how often?
How many users love us enough to share Diffen with their friends?

As we’ve said before, one algo update by a major search engine has the potential to decimate traffic. And yet while we recognize the risks, we also know when to party ;)

link rel=twitter - A proposal for mobile browser makers

Tue, 25 Sep 2012 15:03:58 -0400

It is generally easier to share what you’re reading on a mobile device than desktops. Mobile browsers allow you to email, share on Twitter and, with the latest Chrome and iOS 6, share on Facebook.

One advantage of Twitter’s sharing widget aka Tweet button is the via field. Mobile websites cannot afford to add bulk by loading the Twitter widget. So perhaps mobile browser vendors should start supporting a way to add the via hint. It would work like this (without the mangled smart quotes that Tumblr introduced below):

<link rel=”twitter” href=”https://twitter.com/Diffen”>

With that in your <head> the browser now knows the publisher’s Twitter handle and will automatically append (via @Diffen) to the tweet text.

Analysis of the “bump” caused by the Republican and...

Wed, 12 Sep 2012 12:49:00 -0400

Analysis of the “bump” caused by the Republican and Democratic National Conventions.

Here’s a look at the bump caused by the two major conventions. Both events caused an increase in the number of people comparing the Democratic and Republican parties but the DNC caused a bigger spike.

And what of the candidates? More people where comparing Romney and Obama on each day of the DNC than after the crowning moment of the RNC i.e. Romney’s speech. So it seems that Michelle Obama, Bill Clinton and Barack Obama all had more of an impact than Romney.

In other interesting developments, the Democrats have edged the Republicans on the ratings scale. After being stuck at 3.8 for a loong time, the Democratic party now has a rating of 3.9 and the Republicans are still at 3.8.

Obama’s rating is 3.9 compared to Romney’s 3.4 but that should be taken with a grain of salt because Obama’s rating has a historical advantage from the previous election.

Experiences hacking GA event tracking

Thu, 06 Sep 2012 19:39:00 -0400

Last month we started using event tracking in Google Analytics (henceforth forever called GA). Should have started years ago but as they say - better late than never. Thought it might be a good idea to share a few things that we learned.

1. Have clear goals

We have a pretty high bounce rate so we wanted to understand what people are clicking on to see how we could make the product more engaging. The second goal was to track which content categories get how much traffic. Believe it or not, we weren’t tracking pageviews by content category. With so many content categories, each with their own super- or sub-categories, it’s important to know where you’re lagging and where you’re doing well.

2. Learn from others

It’s weird but it was impossible to research GA hacking without running into Cardinal Path. We thought Wikia has many content categories so their GA implementation would probably be a good model. So we set out to find where the magic happened and ran into Cardinal.

3. How event tracking actually works

Geek out on this StackOverflow thread to explore how GA event tracking actually works. If you’re tracking a click on a page, the browser probably doesn’t give GA enough time to send its GIF request with the payload to the server. It’s possible the browser opens your link before GA’s Javascript gets a chance to do its thing. (UNLESS ga.js introduces some sort of document.unload handler which finishes all GA tracking before allowing the original document to unload and the new page to load. That would be cool for tracking purposes but totally UNcool for our users. Given a choice between making the site experience faster for users and tracking their clicks, we’d choose our users every single time. Even if you wake us up at 4am to ask. Fortunately, it seems that GA does not force an unload handler on the original document.)

4. Use onMouseDown

So how to handle scenarios where the browser is too fast for ga.js? That’s where we ran into Cardinal Path again with this awesome blog post about onClick vs onMouseDown when using event tracking. Most examples for GA event tracking will show you code like

<a href=”/wherever” onClick=”_gaq.push([‘_trackEvent’,category,action,label]);”>click me!</a>

But you can increase the probability that clicks will be tracked if you fire your GA event onMouseDown rather than onClick.

5. Page-level custom variables are useless. Use events instead

At first we tried using page-level custom variables for tracking what content categories are most popular. So for example when a visitor gets to the MacBook Air vs. Pro comparison, we want to track a hit for all the categories in the breadcrumbs. The top-level category is Technology and the second level is Consumer Electronics. We assigned the first custom variable for the top-level category, the second custom var for the 2nd level category and so on. But we wanted to do all this tracking after page load i.e. in $(window).load(function(){…});. But for it to work correctly you need to set your custom vars before GA’s trackPageview call. That’s a major difference between using custom vars and event tracking.

trackEvent fires a request to the big G’s servers immediately but setCustomVar waits until some other function fires a request and piggybacks on that to send its data back to the mother ship. In most cases no other events would be fired so we wouldn’t be able to accurately track custom variables.

Enter event tracking for content categories. We now track each category of the bread crumb as an event where:

The event category is hard-coded to ‘PopularCategories’
The event action is the content category in the breadcrumb. (one event for each item in the breadcrumb.)
The event label is the page that the visitor is on.

The event label is interesting because apparently it’s not needed. GA’s web interface gives you the ability to use the Page Title as the primary dimension when viewing data for a particular Event Action. We didn’t know that so we tracked the page ourselves in the label. What’s interesting is that the numbers we see via the Labels breakdown are different from what GA shows when it breaks them down by page title. We’re still not sure why that is. It’s not a huge discrepancy but it’s not teenie tiny either.

So there you have it. That’s how we implemented GA event tracking.

UPDATE: CardinalPath has released an open-source project GAS (Google Analytics on Steroids) that looks very promising.

1 million unique visitors a month

Tue, 04 Sep 2012 16:56:00 -0400

On Aug. 27, we hit a milestone in the history of Diffen. 1 million unique visitors in a calendar month. We had come close before and flirted with 1M on a couple of occasions. But missed the mark by a few thousand. But August was different. We crossed the milestone with a few days of August to spare.

Yes, we know that we’re too dependent on Google right now and one change of their algo can crush traffic. But we’re celebrating in September and have a very cool sponsorship lined up. (Details next week)

W00t!! We love round numbers.We know that one Google algorithm...

Thu, 16 Aug 2012 20:36:46 -0400

W00t!! We love round numbers.

We know that one Google algorithm update can easily set us back but today is a day to celebrate!

Joe Biden vs Paul Ryan ratings Biden’s rating includes...

Sat, 11 Aug 2012 18:54:00 -0400

Joe Biden vs Paul Ryan ratings

Biden’s rating includes votes from the 2008 election comparison so it’s not going to be an apples-to-apples comparison with Paul Ryan immediately. This snapshot is a good indication of their starting scores so when we examine their ratings a few months from now we’ll know how each candidate did.

Behavioral psychologists will tell you (and we would too) to take these ratings with a pinch of salt. Biden performed well because in the last election he was being compared to Palin. His ratings would most likely have been lower if people were rating him on a page that compared him to Obama instead. So making note of this starting line is important because come November, we’ll be able to do a more accurate head-to-head comparison of Ryan with Biden, without letting Biden take advantage of his former comparison with Palin.

Mysterious rise in 404 errors in GWT

Sat, 11 Aug 2012 18:12:00 -0400

Google Webmaster Tools is reporting that in the last month the number of 404 errors has skyrocketed from about 9,000 to 77,000.

We can’t figure out why. In fact, when we check our server logs and look for 404 status codes returned to Googlebot in the last month, there’s only about 10,000 (which might even have some duplicate records..we haven’t checked).

So what in the name of Tim Berners Lee is going on?

Let’s look at the types of URLs Google Webmaster Tools (GWT) is reporting:

URLs from this requests page. These are requests people have made and this page hand hundreds of requests listed. So that makes sense. These are non-existent pages (the comparisons haven’t been created yet!) so they return 404s. And Google found the 404s so all that make sense.
Hundreds of non-existent URLs that were never on the site. Like /Intel_EP80579_with_Intel_QuickAssist_Technology,_1200_MHz (the correct URL is /difference/Special:Information/Intel_EP80579_with_Intel_QuickAssist_Technology,_1200_MHz). Google doesn’t say where it found them. Generally GWT tells you whether the URL was found in a sitemap or is linked from somewhere. But for the URLs it’s reporting to us, no such info is available.

Here’s the deal, though. All the processors and all the user requests don’t add up to more than 2,500 URLs. So where the hell is Google getting 77,000 of them?

What we did

Added <meta name=”robots” content=”nofollow” /> to the comparison requests page so all the non-existent user requests aren’t crawled. This is something we should have done a long time ago. Also stopped listing the hundreds of non-existent URLs on this page. Now we only show the most recent user requests. The nagging feeling we get, however, is whether the nofollow tag will prevent Google from de-indexing all the old 404 URLs that were listed on this page. Will Googlebot notice that these URLs have now been removed from the page or will it see the nofollow tag and start ignoring the page from a crawling and link finding perspective? We don’t know.
We now return 410 for most of the non-existent URLs that Google is reporting. We found a pattern for the 404s that Google is reporting and the ones we uncovered in our server logs. And with some nginx config settings we can now return 410 because Google apparently removes 410s faster than 404s.

Google Webaster Tools: The Good, the Bad and the Ugly

The Good: They tell you there’s something wrong with your site! +1 for transparency and proactive information sharing. They even give you a list of URLs they found, as well as where to fix the problem (sitemap or list of pages that link to the bad URL). All of this information is very helpful and kudos to them for providing it.

The Bad: They only tell you about 1,000 URLs and don’t give you the full list. The CSV download is missing crucial info viz. where the URL is being linked from. Where did Google find the URL?

The Ugly: The rise reported by GWT does not reconcile with the number of 404s returned to Googlebot as indicated by our server logs. And there’s no way to know when Google will remove these 404s from its index.

Why could this be happening?

We honestly have no idea. There are only guesses. Right now we’re leaning towards a theory that blames spam. Spammers sometimes create user accounts on Diffen (which is a wiki and runs on the awesome MediaWiki platform - the same software that powers Wikipedia). Mediawiki has user pages where registered users can write a little about themselves. We require reCAPTCHA when users sign up but spammers manage to defeat it from time to time. They have been posting spam by creating such user pages. It’s possible they are linking to thousands of such user pages but only succeeding in actually creating a few of them, (which we find and delete). It could even be negative SEO by a third party but that’s not our favorite theory right now.

Mandel vs. Brown - who's leading?

Thu, 26 Jul 2012 00:38:00 -0400

Mandel vs. Brown - who's leading?:

Brown is ahead in the polls but Mandel is all over YouTube. This is the most expensive race in Ohio’s history - over $24 million have been raised.

Market share for different versions of Internet Explorer. IE 6...

Sat, 21 Jul 2012 15:58:40 -0400

Market share for different versions of Internet Explorer. IE 6 is dying!!

Browser share among Diffen users in the last one month (Jun 20 -...

Sat, 21 Jul 2012 15:51:00 -0400

Browser share among Diffen users in the last one month (Jun 20 - Jul 20, 2012)

WHAT MAKES YOU FEEL BETTER WHEN YOU ARE IN A BAD MOOD?

Wed, 18 Jul 2012 16:01:00 -0400

Reading comments on Diffen. People (usually!) get what they’re looking for and leave happy comments.

Hello, World!

Wed, 18 Jul 2012 14:26:26 -0400

This project has been a long time coming. In the course of building Diffen, there have been many occasions when we felt we learned something or found something interesting. On such occasions, we said to ourselves “This would make an interesting blog post. If we had a blog, we’d definitely write about this.” Well, we didn’t have one because we were too lazy. And perhaps a little shy.

But no more! This blog is now live.