jondyer.net

Posted by Dan Rodney, Software Engineer

Google Analytics features 3 types of custom variables: page-level, session-level, and visitor-level. The official Google Code documentation on custom variables is pretty explicit about the fact that it’s best not to mix types:

“Generally it is not recommended to mix the same custom variable slot with different types as it can lead to strange metric calculations.”

What isn’t exactly clear is what happens if you do decide to mix types. Google Code provides two cases, but surely there are additional cases. To this end, I decided to test 9 total cases:

Case 1: Page to Page
Case 2: Page to Session
Case 3: Page to Visitor
Case 4: Session to Page
Case 5: Session to Session
Case 6: Session to Visitor
Case 7: Visitor to Page
Case 8: Visitor to Session
Case 9: Visitor to Visitor

Granted, three of these cases are not mixed type, but I decided to include those cases as well since the behavior for those cases should be well defined and good benchmarks for the other cases.

It would also be good to know exactly how Google Analytics attributes conversions in these types of situations. To that end, I set up a test with 5 different pages:

Page 1: Standard Google Analytics Tracking Code
Page 2: Set first type of custom variables
Page 3: Standard Google Analytics Tracking Code
Page 4: Set second type of custom variables
Page 5: Standard Google Analytics Tracking Code

In Google Analytics, I set up 5 different goals, one for each of the above pages. The idea is to see whether or not each custom variable gets credit for a conversion on a previous page, the current page, or any successive page.

I also included a second visit, with two additional non-goal pageviews just to confirm behavior for visitor-level custom variables.

Normally you’re restricted to using 5 custom variables, but it is possible to get more than this. This allowed me to run a single test with 9 custom variables at once, which looked something like the following on Page 2:

<script type="text/javascript">

var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-12345-1']);
_gaq.push(['_setMaxCustomVariables',9]);
_gaq.push(['_setCustomVar',1,'page_page_1','page_page_1',3]);
_gaq.push(['_setCustomVar',2,'page_session_1','page_session_1',3]);
_gaq.push(['_setCustomVar',3,'page_visitor_1','page_visitor_1',3]);
_gaq.push(['_setCustomVar',4,'session_page_1','session_page_1',2]);
_gaq.push(['_setCustomVar',5,'session_session_1','session_session_1',2]);
_gaq.push(['_setCustomVar',6,'session_visitor_1','session_visitor_1',2]);
_gaq.push(['_setCustomVar',7,'visitor_page_1','visitor_page_1',1]);
_gaq.push(['_setCustomVar',8,'visitor_session_1','visitor_session_1',1]);
_gaq.push(['_setCustomVar',9,'visitor_visitor_1','visitor_visitor_1',1]);
_gaq.push(['_trackPageview']);

(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();

</script>

On page 4, I changed all the names and values from _1 to _2 and made appropriate changes to the scope of each variable (3 for page-level, 2 for session-level, 1 for visitor-level).

There are a couple interesting things to note if you ever decide to run a test like this. First, if you run your test too soon after creating a new Google Analytics profile, it’s possible that the data from that test will never show up in your reports. So it may be a good idea to wait an hour or so after creating the profile before running the test to make sure that you’ll see any results.

Second, certain types of information take longer than others to show up in your profiles. While you can segment by custom variable keys and values pretty much as soon as the data shows up in your reports (in well under an hour in some cases), the dedicated Custom Variables (in the old version of Google Analytics) report actually takes as many as 48 hours before any data starts showing up in your reports. So this is something to be aware of if it looks like things are missing.

The result? Take a look:

Just to be clear, when the custom variable appears with an _1, it means that the first custom variable set was reported. When the custom variable appears with an _2, then the second custom variable set was reported. For example, the entry for session_page_1 means that the session-level custom variable was reported, while the entry for session_visitor_2 means that the visitor-level custom variable was reported.

Based on the data, we can conclude the following:

1. If a visit only includes page-level custom variables, then each custom variable set will receive attribution for the page it was set on, including goal attribution if the goal was set for that page.

2. If a visit contains either a session-level or visitor-level custom variable, then the last of these set will be the only one that’s reported. Furthermore, that same custom variable will be the one that receives credit for any and all goal conversions during the visit, regardless of whether those goal conversion appeared before, at the same, or after the custom variable was set.

3. A corollary to #2 is that for a visit that contains a page-level custom variable and either a session-level or visitor-level custom variable, only the session-level or visitor-level custom variable is reported.

4. Based on the custom variables that received more than one visit, a visitor-level custom variable only persists if it was the last custom variable set. For example, while the entry for visitor_page_1 indicates that the visitor-level custom variable received credit for the visit, because the page-level custom variable was set after visitor-level custom variable, the visitor-level custom variable was not reported on the subsequent visit, as indicated by the fact that there was only 1 visit for this custom variable. If you watch the __utmv cookie, you’ll see that the data for the visitor-level custom variable is actually removed when a page-level or session-level custom variable is set for the same slot.

The results of this test also prompted a follow-up test for an edge-case:

Case 10: Visitor-level on the first visit, page-level on the very first page of the second visit.

It turns out that the visitor-level custom variable is reported on the first visit, as expected, but the page-level custom variable is reported on the second visit.

Based on these results, it looks like two cases is sufficient to account for all mixed type situations, but it might help if the first case provided in official Google Code documentation more clearly show-cased these rules.

For the first case, instead of the following scenario:

The following would more clearly illustrate the precedence property:

The fact that the visitor-level custom variable set on page 2 would be the one reported more clearly indicate the way the precedence works. There would also be an opportunity to mention the fact that because the page-level custom variable occurred after the visitor-level custom variable, that the visitor-level custom variable would not be reported on any subsequent visits.

Feel free to leave any questions or comments about custom variables, especially if you can think of additional scenarios that are not covered by the official Google Code documentation or this post.

Web Analysts are blessed with an immense amount of data, and an amazing amount of valuable, even sexy, metrics to understand business performance. Yet our heroic efforts to report the aforementioned sexy metrics leads to little business action. Why?

Sure your organization could be to blame (org structure, bad boss, ineffectual team). Perhaps your client did not provide you with the all important Web Analytics Measurement Model. Maybe something else (surprisingly excuses are not all that hard to find when you are looking for them ).

Even with all that I think the problem is you.

You: The person responsible for "providing data" / "analyzing metrics" / "reporting."

You are the problem because you, mesmerized by the enchantress that is web data, are reporting "crappy" metrics.

Since crappy sounds bad, let's just say you are reporting super lame metrics. (Sounds better right?)

Oh and I don't mean reporting super super lame metrics like % of Exits from a Page. I think we've realized that is absolutely useless for unstructured experiences (most web behavior).

I mean the metrics contained in 99% of web analytics reports: Visits, Page Views, or Time on Site. I mean the metrics contained in 99% of digital advertising reports: Impressions, Clicks, or Emails Sent. I mean Social Media campaign metrics like Number of Followers (or Likes), Video Views, or (just kill me now!) # of Press Reports.

Surprised to find the bedrock of your (and my) existence classified as super lame?

They are. And it is time we accept that. How else will we be recognized as the magnificent creators of business value, rather than people who whine about data (or worse, much much worse, JavaScript tags and sprops and custom variables)?

These metrics are super lame because. . . .

1. They are supremely tactical.

There is nothing hugely business bottom-line impacting we can learn from them. Ok we got 17 million fans on Facebook. That was success? People saw 3.2789 pages last month and it was 3.3592 this month. Ok, now what? What does the number of press reports or views of our video actually tell us?

2. They simply report, usually, "top of the funnel" activity.

Getting people to just show up to our digital existence or just sending them email is not the point of what we do. Our goal as a non-profit is to ensure we get more people to our protest, more donations, more leads for places we can add value to. Our Goal as a B2B business is to connect a high value prospect to an Authorized Dealer / Rep, to try and revolutionize our marketing by going after the search long tail with AdWords, and more. It is not hard to imagine what a B2C site is trying to do.

So why is our reporting dominated by reporting of visits and time on site and impressions and all that. It's work, but what about what happens from that work?

3. They require too much inference.

When you present large number of Visits or Page Views or Followers what you are essentially inferring is that more is better. You are inferring something that is not there, success. Or you are hypothesizing, when you report that data, that these large numbers mean that customers are happy and business is successful. I believe it is dangerous to make that inference. Why not seek direct success indicators? Even for our branding campaigns?

4. They are "one night stand" focused.

This is perhaps the thing that makes me hopping mad and scream like a banshee. In the early days of the web it was cute to just get someone to come to the site and most stuff happened within one session / visit. I am so mad that by 2011 as web experiences have become richer and more frequently and more complex that our life is not dominated by pan-session metrics. That our campaigns are not focused on Visitors (not just Visits!), longer term understanding of people and customers (not just entrances and bounces!), customer lifetime value (and not just single visit conversions!), and all that stuff that is mandatory for the long term success of any company.

Frustrating when you think about it, right?

The bottom-line is that while they are all "standard" metrics they measure tactical top of the funnel activity requiring too much inference and cause us to simply chase one night stands. Let us not make them the center of our web analytics existence.

Let us banish them to the land of "oh yes that was cute when our intern measured that when we first got a web analytics tool, but don't be silly our dashboards don't contain that anymore."

Let us move to the land of the "OMG that was such high speed from data to business action that I think I might have experienced a datagasm!" I.E. super awesome land.

Super awesome metrics. . . .

~ Force strategic analysis of metrics that contain data material to the business bottom-line!

~ They are infused with direct customer voice so we don't need to infer and look at data with our own biases (just let them tell us!).

~ Make us look at behavior across sessions to encourage a business focus on long term value (customer and business).

~ Due to their inherent nature in most cases they make it very obvious if the performance you are looking at is good or bad. It is hard to get to this, but it is so magical!

Salivating? Can't wait to hop into bed with 'em?

Now here is the amazing thing. You know which metrics I am referring to. I doubt I am going to shock you below. The problem is reaching datagasms requires some hard work (surprise!), they require some fine caressing of the data, they require going beyond the standard report in Google Analytics.

Here are some metrics I consider super awesome. . . .

Visitor Loyalty & Visitor Recency.

Notice my immediate focus on Visitors. Cookie issues be dammed (use first party ok?), this is the best we can do for now, use it! I love Loyalty and Recency because they measure what actually matter (repeated frequent visits by an individual – do not be tempted by the not very useful % Repeat Visits).

For content sites (NY Times) it is focusing on a longer term relationship (and that might even mean not focusing on Page views per visits and reducing the number of ads on a site!).

For non-profits (The Smile Train) it means focusing on creating a connection with that causes future donations.

For ecommerce sites (like Amazon) it means focusing on an experience to cause the next 20 purchases, or (like TurboTax) focusing on such a delight this March that I'll be back next March.

I love these metrics. When you focus on their performance it will force you to materially change your website experience, customer relationships and business value.

Why focus on conversion rates from your AdWords or DoubleClick campaigns? What was the quality of that traffic beyond the one night stand? Visitor Loyalty, bam!

[Learn ins and outs and how-to: Visitor Loyalty, Recency, Length & Depth]

Days & Visits to Outcome.

In some sense an enabler of the above. If your business leaders continue to want one night stand data then report these metrics (standard in Google Analytics and WebTrends and every tool).

Two different visits shows the length and depth of an experience leading up to an outcome (typically a ecommerce conversion, but could just as well be a lead, a donation, or any outcome you desire).

It helps force changes in customer experience (why just have a Search Travel Deals now and not a Save Travel Deals You Found So That We Can Email You When The Price Goes Up Or Down button?), it forces you to worry a out multi-channel campaign attribution (or not!), and more such delightful outcomes.

Strictly speaking is a pan-session metric (as are the above two), but I still think of it as a single outcome focused. So do this, migrate to above.

[Learn ins and outs and how-to: Analytics Tip #6: Measure Days & Visits to Purchase]

Conversation Rate.

Social Media is all the rage. Tweeting, Facebooking, Blogging, YouTubing. For most brand that, thus far, has simply translated as: "Oh we shout at people via TV, Magazines, Radio etc, what is the best way to shout via these new channels."

Sad. Heartbreaking. Your inner happy child just died.

Fear not. (She has nine lives!)

The true essence of social media, if you want to do it right, is the ability to reach your audience directly (no intermediaries) and have a relationship with them. If you want it. The essence of that relationship, one key facet, is the ability to spark and participate in conversations.

So why measure Followers and Likes and Posts? Why not measure Conversation Rate? For blogs: # of reader comments per post. For Twitter: # of replies sent per day, # of replies received per day. For Facebook: % Feedback.

Find out if you are conversing, who you are conversing with, is it adding any value to your brand, should you be doing it more? The roads to all those interesting questions, and incentivizing the right behavior by your company, starts with this super awesome metric.

[Learn ins and outs and how-to: Social Media Analytics: Quantitative & Qualitative Metrics]

Task Completion Rate.

The sub-title of Web Analytics 2.0 is: The Art of Online Accountability & Science of Customer Centricity.

That second part? That's why I love Task Completion Rate as a metric.

I see Analysts and Marketers and HiPPO's (of all sizes) torture the clickstream data, make leaps of faith, and start their recommendations with: "I think." Sub-optimal on so many fronts.

Don't think. Just ask.

Task Completion Rate is the % of people who come to your website who answer yes to this question: "Where you able to complete the task you came to this website for?"

Combine that with the Primary Purpose question ("Why are you here?") and you have a gold mine of fantastic data. Why people come, how much you let them down. No guessing. No making stuff up. No inferring things from Time on Page or % Exits!

Use the customer voice, hit people on the head with it, reward those who work hard to improve the customer experience (based on, I can't stress this enough, customer identified pain and Omniture or Yahoo! Analytics guessed), and reassign those who don't. I call that #winning.

[Learn ins and outs and how-to: The Three Greatest Survey Questions Ever]

Economic Value.

What can I tell you about Economic Value that you don't already know. Oh, after Task Completion Rate, it is the single greatest gift a Digital Anybody can give to their boss / company / mom / angels.

It is the total $$$s (or Pesos or Rupees or Krones) in Economic Value added to your business bottom-line by visitors to your website completing all the possible Macro and Micro Conversions.

Our inability to quantify the value of our digital existence is the single biggest reason for the sad undesirable level of appreciation. We just focus on the 2% conversion rate (orders or leads) value added. The thing we call Revenue. We forget all the other Micro Conversions. Absolutely criminal.

If your web analytics tool does not allow you to (in five minutes without touching the JavaScript tag) set up goals and goal values then ditch it. You are wasting everyone's time, your mom is not proud of you.

Once you identify the Economic Value of your website you'll be able to clearly articulate: 1. All the jobs your website is doing. 2. True and complete value of those jobs. 3. Identify holistic value of your digital marketing campaigns. 4. Identify where you are falling short and where you are glorious. 5. Make your mom proud.

How is any metric that can help you do that not super awesome?

[Learn ins and outs and how-to: Excellent Analytics Tip #19: Identify Economic Value]

Five simple, effective super awesome metrics.

It is not Visits and Page Views and Impressions and Fans and Clicks are useless. They are not. They are just not worthy of all the attention you give them. They are not. . . super awesome.

In the small chance that a picture is worth a thousand words, why take chances , here's a slide from my recent keynote. . . 

So you can see there are many more super lame and super awesome metrics than mentioned in this article.

My hope is that the specific examples outlined help you find specific examples you can start your own revolution with.

My wish that you'll look not at the specific examples above but more why I choose them for my Super Awesome list. The thinking behind the decision, the insights and value I wished to desire. If you get that you'll be prepared regardless of what your company is up to on the web, regardless of what web analytics tool you are using.

. . . . and one more thing

I'll leave you with a practical example of how evolution from super lame to super awesome words (with a just regular awesome stage in between) looks like.

This is a real example of trying to measure how well advertising done by a company is performing. They start at the normal place everyone else does, Clicks and CPC's. The company impact is just keeping costs low. I call this the Toddler stage (who, as a father of a small child let me point out are super duper awesome and I don't mean to imply anything else), but it is, super lame. How are you winning? How deep are your insights?

Then we mature to the regular awesome stage. The focus is on Conversions (yea!!!) and Revenue. Company impact is maximizing Revenue Per Acquisition. The Rockin' Teen stage. Good. But not great.

[Click on the image above for a higher resolution version. The slide above is from my 60 min webinar on YouTube: Agile, Outcomes Driven, Digital Marketing]

The super awesome stage is the last one. Ninja's live there. Measuring complete Economic Value to the business as the company impact. Kissably super awesome.

Consider this. Just look at the colored rows. You could have made a decision about which campaign was working better at each stage. But at only one would you have made the right decision and, because of how important that number is, earned a promotion.

That's what I'm talking about!!

Being super awesome.

You can create the above evolution for your company for any metric you desire, for any outcome that is expected by your business. And now you know how.

Good luck!

As always it is your turn now.

Surely you have your personal favorite super lame metrics. . . what are they? Why do you think they are "super lame" (or to be polite, less than useful)? And which metrics, if any, do you consider super awesome? Where have you fully focused your attention in order to identify valuable insights that have actually been actioned? What do you love?

Please share via comments.

Thanks.

Your Web Metrics: Super Lame or Super Awesome? is a post from: Occam's Razor by Avinash Kaushik