Measuring the Wrong Thing: Data-Driven Design Pitfalls

Yesterday I got the chance to give a brand-new talk at the Midwest UX conference, here in Indianapolis this year. It covers some of my experiences working with data in the design and project process: trying to get it, struggling to interpret it, and using it to make decisions.

Slides from my talk are now up on Slideshare, but I’ve posted my complete transcript of speaking notes below. Thanks to everyone who was able to be there, especially the folks with smart questions and comments on Twitter and in-person.

Measuring the Wrong Thing: Data-Driven Design Pitfalls from Jen Matson

Transcript:

[Slide 1 – Title]

[Slide 2]
Hi, my name is Jen Matson and I work as a Senior User Experience Designer at Amazon. And for most of the past two and a half years I’ve been there I’ve focused on creating solutions for third-party sellers – everyone from big businesses to mom-and-pop shops to individuals looking to unload some used books or CDs.

I’ve been designing and building web sites since the dawn of the web, for both product companies and agencies. So I’ve seen our understanding of web site and user data evolve over the past 20 years. From the number of “hits” a web site gets and what other sites might be sending us traffic via URLs in our referrer logs, to sophisticated tracking of specific users across pages and their microinteractions within a given page.

And yes, I am a data junkie. In that picture of me, I’m sitting in front of part of my record collection, which I not only listen to, I mine for data. Here’s a better picture of that.

[Slides 3-5]
Starting at age 15, I created my first digital database of my collection – here’s a sample of the flat file containing my 7″ singles. And of course it’s not just about the data itself, but the story it tells. So I’ve used data visualization software to give me insights about my collection. I clearly like a lot of music released in the late 80s to early 90s.

[Slide 6]
So, I do that for fun. But I apply the same curiosity about data in my job. Because it’s key to not only understanding the effectiveness of my work as a designer, but in helping to set the direction of a project from the outset.

But many people still either don’t fully understand the data they are collecting, or misuse it – intentionally or not – to draw conclusions about what our users want or need. And I am here to share some of these missteps, and how to do better, how to use data to inform your designs, to create work that better serves your business and your users.

[Slide 7]
Now, Amazon is a company that highly values data. In fact, fellow speaker Jared Spool has spoken in the past about some of the ways in which Amazon gathers data and uses it to improve both the customer experience and the company’s bottom line. Most product companies today not only actively collect data, but seek to use it to improve their products in an ongoing fashion. Project methodologies like Lean and Agile emphasize customer value and validation of hypotheses, and we need data to understand both.

[Slide 8]
What about agencies, or client services companies? I’ve worked at my share of agencies, and traditionally they haven’t been able to get the same access to information that can be used to shape and improve design. Either the data is considered company confidential (not shareable with the agency), or once the agency delivers the end product, their engagement with the client ends, with no further insight into how the product performed with users.

[Slide 9]
So in some companies data is the driving force in product design, whereas others are more focused on the feature idea. Cennydd Bowles, a Design Manager at Twitter, created this chart illustrating these concepts as part of a blog post on the topic. (Though I don’t necessarily agree with the distinction between product vs. web.)

Using data to drive design would seem to be the safer route to product success. But when data is used as a starting point, we can fall into the trap of molding our ideas to fit what can be easily gathered.

And beginning with an idea, we have a clear vision for what we want to deliver. But without customer data to validate that we’re building the right thing, we may just end up creating something we (not our users) happen to think is cool.

And of course, the worst situation of all to be in to fail to gather the data we need to make smart decisions about what to design and build BEFORE we release something to our customers.

[Slide 10]
So I’d like to walk you through a few different projects where I’ve encountered these issues. These were not shining moments. I’ve anonymized these examples, as while I’m happy to share my own mistakes, these companies and clients might not be so willing.

Let’s start with a close call where we almost didn’t get the data we needed before launch…

[Slide 11]
Case study 1 – The Meaning of A Click (or Tap)

[Slide 12]
Company: A movie listings web site
Project: Create a mobile-optimized version of the movie detail page.

The project seemed straightforward enough: take an existing “desktop” page meant to show information about a specific movie – plot synopsis, cast information, showtimes, trailers and reviews – and make a mobile-friendly version optimized for smaller screen sizes and touch interaction. The goal was simple have the same great experience, same content and features, on the mobile version of the site.

As part of the process, a usability study was planned prior to launch, where we’d see how the design prototype performed with customers seeking movie information on their phones.

[Slide 12]
As the day progressed, an interesting pattern emerged: customers would tap on the reviews widget, then immediately navigate back to the previous page without seeming to read anything.

[Slide 13]
When asked by the researcher to talk about what it was they were doing and looking for, customers said that they hadn’t meant to tap on reviews; they were trying to view movie showtimes.

This was odd – the button to view movie showtimes was further down the page, offscreen. But it turns out that users thought the “Movie Showtimes” heading was a link, as that heading text was purple, matching one of the company’s brand colors. And it was the first element they saw that matched their task (find movie showtimes). Since the ratings widget was positioned so close to the Movie Showtimes heading, customers were accidentally tapping on the widget link, taking them to content they didn’t want.

[Slide 14]
So what seemed to be a tap for one thing ended up being a thwarted tap for another. If we’d relied upon quantitative data alone, which would show us link taps and navigation behavior, we’d have gotten bad data. And yes, those elements probably needed to be spaced a bit farther apart as well, to avoid accidental taps.

Now, thankfully this issue was caught prior to launching the mobile design. But it definitely happens – maybe frequently in your organization – where the product owner says “No time for usability testing: ship it and we’ll look at the data post launch” And they honestly may believe that will give them the necessary insights about what works and what doesn’t.

However, if we’d launched without testing, a couple of things would have happened:

[Slide 15]
– First, we’d have shipped an experience that would frustrate a good chunk of our users. (And I am definitely not a fan of using your customers – and their good will – as guinea pigs for half-baked products.)

– But more importantly, the data would never have alerted us to the issue. In fact, the big takeaway would have been “our mobile customers LOVE reviews!” And maybe we’d have built out even more features to cater to that accidental behavior.

[Slide 17]
How were we able to fix this?

– Team had an understanding of the importance of gathering both qualitative AND quantitative data. In addition to measuring objective things like task completion time, number of taps, and what was tapped, we asked contextual questions of users as they performed their tasks in order to round out the picture.

– The schedule included time for usability testing up-front. We had the support of the entire team not just for testing, but for the additional design and development time needed to make changes based on what we learned.

[Slide 18]
Case study 2 – Throwing Stuff Against the Wall

[Slide 19]
Company: A mobile service provider
Project: Redesign the help portal to offer personalized help content.

Customers of this mobile service provider would arrive at the site and log in to manage their account: check their plan usage, pay their bills. So we should know exactly who they are when they go to access help content.

However, that entire section of the site was largely ignorant of the specifics of the individual visiting it. While we could display content based on whether the user had a business or personal account, any personalization beyond that was lacking.

[Slide 20]
I had started work on an approach that would tailor content based on known user attributes, such as plan type and account age. However. mid-way through the project I was told we needed to incorporate a widget built by another team in our group that provided personalized help in Q&A format.

[Slide 21]
I tested the suggestions given by this widget using my own mobile account, only to find that it was providing advice that didn’t apply to me. The questions I saw were: “How do I reactivate my phone?” (my phone was working just fine, thanks), and “How do I add more data to my plan?” (I already had a largest data plan they offered).

[Slide 22]
This help widget wasn’t helping me.

[Slide 23]
I was getting a false positive result. Because the team responsible for the widget was doing two things that didn’t quite make sense:

1. Surfacing questions to users based on “other users like them” and not their specific account situation, and

2. Taking clicking on a question, as I’d done, as a “win”. The assumption was that a click equaled interest in a topic, and interest therefore equaled relevance.

But there is only a loose connection between a click and interest. After all, I clicked just because I was curious. And since the two teams weren’t aware of what the other was doing, we never had a chance to have that conversation until it was too late.

[Slide 24]
The impact on users would be significant. While we couldn’t measure it without logging into multiple accounts we were already very familiar with, a brief audit of a few other coworkers’ accounts showed the same issues. We could expect users to be confused, and perhaps be less likely to trust our ability to help.

And the cycle would continue until we came up with a better way to measure success.

[Slide 25]
Because to fix this, we’d need to come up with a way to audit real customer accounts, noting specific lifecycle events they had experienced (service disconnection, data plan overage), then validating whether or not relevant questions were surfaced for matching accounts, or were suppressed for non-matching ones. This we should do BEFORE launching new questions.

Even better, let’s use that lifecycle model as the sole way to determine relevance, not the black box of “other users like you”.

And ultimately, we’d need to have a unified team to solve personalized help, to avoid clashing approaches.

[Slide 26]
Case study 3 – Unclear Cause and Effect

[Slide 27]
Company: A TV manufacturer
Project: Update the site support search engine to make content easier to find.

We’re responsible for the web site for a product company that mades high-end flat screen TVs. But some of our customers have trouble with setup and installation and come to our web site looking for help. And a large percentage of our customers are choosing to contact customer support not long after they arrive on our site, even though many of them are using our search engine to search for support content.

So the primary business goal was to reduce contacts. How do we align that with user goals? Make it easier for them to solve their problems on our site by finding the info they need.

[Slide 28]
I discussed this with the product manager, sketching out some of the steps in the user’s problem-resolution experience once they landed on our web site:

Find article via browse or search of support section -> Read article/forum answer or watch video -> Use solution information or self-service tool to solve problem

[Slide 29]
And in order to enable a successful task flow, our content at various stages needed to be:

Findable -> Consumable -> Actionable

So to best achieve our contact reduction goal, we’d ideally want to reduce the number of people abandoning at each stage. Move people through that funnel.

[Slides 30-31]
As I mentioned, the product manager was focused on tackling the search experience first. Findability. And the simplest way to measure success would be to track how many users clicked on the “Contact Us” link from a search results page now, then see if we can reduce that percentage as we roll out the redesigned search to our users. For a user to click on contact us instead of engaging with content that should be relevant and visible is our sign of failure.

[Slides 32-33]
Problem is, the project team had already committed to measuring contacts from users who tried the new search within a 20-minute session. Much weaker connection between cause and effect. In 20 minutes, a user could read an article, try a tool… and fail at those tasks. And we weren’t yet going to address problems in those experiences. And there were problems: from videos that included outdated information, to articles that failed to provide links to self-service tools. But by measuring the contacts per session, we put ourselves on the hook for things we didn’t’ have the time, the resources, or even the plan to fix.

[Slide 34]
As a result, we couldn’t accurately measure the impact of the search redesign relative to other factors in the support experience.

So why did the product manager pick the 20-minute session measurement? Because that was the thing that was easiest to measure. We didn’t have the ability to track a user’s path through a funnel. Nor could we see exactly which UI element was clicked on.

And we ended up working on search, when perhaps the real problem was with content. But we didn’t have the data to show what was the biggest problem for our users.

[Slide 35]
We also couldn’t get the data – that was an unfortunate by-product of the launch. Without the right instrumentation on our pages to get that funnel data, to see where the drop-off was, or without at least interviewing some users, we were leading with a idea without the data to validate it. And as much as I may have felt some of the content was poor, I couldn’t help make the business case to prioritize that work without objective data.

[Slide 36]
So, how could we fix this?

Mainly by working with the product manager as early as possible in the process, to ensure that we decided what to build based on the evidence to meet that contact reduction goal. And for me to create task flows and use those to help illustrate our users’ likely behavior, and how that should connect to the type of data we wanted to collect.

[Slide 37]
What else?

So you can see how, even on teams that have an awareness of the importance of data, different factors can cause things to go wrong. In addition to some of the project lessons I’ve shared, you can do a few more things to help yourself as a data-informed designer.

[Slide 38]
Understand your company culture
– Before you do anything, get a sense of whether data is valued, and what kind. Even if you think interviewing your users might be the best way to get the data you need, in some cultures that might be considered “anecdotal”. Or perhaps someone has some marketing research they think is great starting point. Or they want to set up a focus group to ask users what they want. In these environments, it might be challenging to steer the data ship in the right direction.

[Slide 39-40]
Learn more about what you can measure, and how
You’ll want to familiarize yourself with the full capabilities of what can be measured online. It’s a lot more than you may think:
+ Referring page. The page the user visited immediately prior to landing on the subsequent page.
+ Site path. What paths are users following through your site, either page-to-page or within a page?
+ Clicks. What UI elements the user clicked on, whether or not that resulted in a new page to load.
+ Hovers and Mouse Path. Where is the user mousing onscreen? Over what elements, following what paths?
+ Scroll depth. How far down the page is the user scrolling? Combine with data about screen size (width/height), and you can get a clear idea of exactly what the user viewed.
+ Dwell or user focus time. How long did the user spend viewing a certain segment of the screen? Combined with scroll metrics, you can see where they paused to stop and read and interact.

[Slide 41]
Use what you learn to improve your designs and increase your influence
– The best way for you to get better as a designer is to see tangible evidence of the impact of your work on your users. And good data provides that. You should be able to put your best effort out there, learn what works and what doesn’t, then try again, and continue to iterate. And with data, you gain an objective tool you can use to better demonstrate your value to your business. Which means greater likelihood of getting that strategic seat at the table earlier.

[Slide 42-43]
Because we’re all very fluent in asking who are our users, what are their needs, what problems are we trying to solve for them. These questions are a standard part of every UX designer’s playbook. All I ask is we add to that: “what data do we have to support this?” or “how will we get data to validate this?” Learn from my mistakes, and use data to inform your designs to better delight your users.

[Slide 44]
Thank you.


Instagram and Online Service Principles


My final Instagram post

Another day, another free technology service choosing to sell it users. Oh, Instagram. There you go predictably following in the well-trodden, privacy-violating, user-unfriendly steps of your new daddy, Facebook.

You For Sale

And as other have already noted, in the face of ever-more-aggressive moves by online services to “monetize” their users, “if you’re not paying, you’re the product.” It’s a sentiment also shared by writer Douglas Rushkoff, who, in an alternately entertaining and sobering keynote rant at WebVisions 2011 in Portland, defined the problem as both a warning and a challenge to us all: “Program or Be Programmed.”

This Is Not Your Beautiful House

I like to think of the issue as “Own or Be Owned.” If you don’t have your own domain name, you’re putting your online identity in the hands of your landlords at every digital service in which you participate. Google yourself and see: are your social profiles on third-party sites the first results that are returned?

Your landlords may be cool and all — hell, you might even know some of them personally from the awesome early Internet times that Anil Dash recently reminisced about. But, cliche or not, Money Changes Everything. And your cool landlord can go from being cool to kind of a dick to slumlord in no time.

But you don’t have to simply sit back and become a product, perhaps complaining loudly while ultimately changing little about your online behavior. Or worse than simple inaction, becoming so jaded as to not care anymore. Nor do you need to become a digital hermit eschewing all contact with dirty, dirty commercial online services that nonetheless seem very useful indeed to people you like and respect. Sure, you can participate. Just don’t commit.

Don’t Worry About the Facebook

No matter what some politicians may say, companies are not people, and I don’t need to treat them as such. Apply the same skepticism and care when evaluating online services as you do in any other area of life where there are real values at stake that you care about.

Many of us apply personal principles when choosing food to eat based on the source or how animals involved in its creation were treated. Why not exhibit the same care towards your personal identity as your diet? Granted, this is much easier if you’re a geek, like me, because setting up your own web site is a relatively trivial task.

My Online Service Principles

In that vein, I’ve put together what I’m calling my “online service principles.” As principles, they’re things I care about and use to guide my engagement with, and usage of, the multitude of social and other online services scattered across the Internet. But they’re also practical guidelines that can (usually) be easily measured via review of such services’ online user agreements and privacy policies, as well as choices they make in their products’ interface design and PR folks’ external communications via blogs and other news channels.

Right now, I’ve come up with five principles that encapsulate pretty well the things I value online:

1. Ownership
I own my content and maintain my copyright. Anything I post on a service is used in some way as part of that service, but ultimately it’s mine.

2. Control
I control my content, as well as my likeness, and can do with it as I see fit. I can post my content in a context relevant to the nature of the service (e.g. posting a photo for it be seen by friends browsing my photo stream), but it’s not okay to use it in another context altogether — say, for advertising purposes — without my consent.

3. Portability
I have the ability to move all of my content generated using a service to another platform or my own personal archive with a minimum of friction. That means downloading or exporting a database, not cutting-and-pasting or manual “Save As..” operations. Because, as per #1, I own my content, and you should make it easy for me to get at it.

4. Openness
There are external APIs, news feeds or other input/output hooks that both enable some level of customization of my experience as well as demonstrate an interest in engaging with the larger Internet community. This principle is one that others may care about less, but I personally am not, and have never been a fan of walled-garden communities. I’m an unabashed web fan, and I seek to support those who are the same.

5. Transparency
The service is proactive about notifying me of changes that may materially affect my decision to keep using the service as per above. Notifications are done in language that is human-understandable, and user controls for making changes are labeled in a way that is clear and located in an easy-to-reach area of the site.

Walking the Walk

Now, does ever service I use meet all of these principles? Of course not. They’re not a litmus test. But they provide a compass I can use for regularly evaluating not just my usage, but my level of engagement with different sites and services.For example, I’m not a fan of Facebook, as I feel they are one of the worst offenders, in terms of privacy violations and lack of transparency to users. But it’s the site that many of my friends use to communicate, share, and send digital invites. To remove myself entirely would cut me out of a significant part of that social activity, so I maintain an account I check occasionally: when I get an email about being “tagged,” when I get an invitation. But then I follow-up, if needed, via email. And I choose my own places — this blog, my Twitter account — for conversing and sharing.

Most important is that I get to choose. I can allocate my time and energy to the services who match my values most closely. And I often pay for the privilege, as in the case of Flickr. Yesterday, I re-upped my Pro subscription when I realized that I wanted to share pictures from a recent trip of mine to India with my friends, on my terms. In light of the Instagram policy changes, serendipitous timing indeed.

So yes, I’ll pay, whether for a Flickr Pro account or my own domain name and web host. Because I’m tired of gambling on other companies to value the same things I do. Whenever possible, I’m choosing to take the safest bet: on myself.


Mobile, Desktop, Neither, Both

If you design or develop web sites for a living (and if you’re reading this, you probably do), for the past couple of weeks your Twitter stream, RSS feed or [insert news firehose of choice] has likely been filled with everything from detailed rebuttals to complete bewilderment regarding Jakob Nielsen’s latest alertbox column, Mobile Site vs. Full Site.

Probably my favorite quote on the topic is this, from Josh Clark:

When you see a “full desktop site” link on your phone, you’re looking at an admission of failure.

As much as I agree with that sentiment, admitting failure is still preferable to ignoring the problem altogether. What if the site you’re browsing on a phone doesn’t even have that link, and you know that the content you need does in fact exist on that company’s “desktop” web site?

A Mobile Browser in Disguise

Well, if you use an Android phone running Ice Cream Sandwich (4.0), there is an easy option for circumventing the dumbed-down mobile experience on such sites: toggle on the option to view the “desktop” version of any given site in the latest version of Chrome browser for Android, released just this past week.

Chrome for Android browser settings, "Request desktop site" checked

Advanced users have long been able to tweak desktop browser settings or use plug-ins to change the browser version or type advertised to web sites. The mobile version of Chrome just puts this option within easier reach of its users, perhaps anticipating (correctly) that they are more likely to need it due many of the poor choices sites have made in how to serve up mobile content. And it works the same way, by changing the user agent string of the browser — in this case, to a desktop version of Chrome. Below are examples of the UA string, both before and after:

Default “mobile” user agent for Galaxy Nexus, running Chrome for Android:

Mozilla/5.0 (Linux; Android 4.0.2; Galaxy Nexus Build/ICL53F) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19

Spoofed “desktop” user agent for Galaxy Nexus, running Chrome for Android:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.45 Safari/535.19

Switching up the user agent string foils most web sites’ attempts to redirect such requests to their “mobile site,” since all clues as to its mobile-ness — the device name, the true OS and true browser — are stripped away or obfuscated. Result? The mobile-dressed-as-desktop browser is served up the default (desktop) content.

A Carrier Collision

Unfortunately, there’s another player in the web browsing experience that can get in the way of a user, his browser, and a web site. And it’s someone with a pointed financial interest in your browsing habits: the mobile carrier.

Such was the rude realization had by by my boyfriend Josh when attempting to visit a web site a few nights ago on his Android phone, a Samsung Galaxy Nexus. Encountering a redirect to that company’s feeble mobile site, he turned on the “Request desktop site” option in Chrome and attempted to reload the page… only to receive content of a very different kind, and not from the requested site:

(“Mobile HotSpot” is just T-Mobile’s way of saying “tethering.”)

Why the upsell/road block?

Josh quickly deduced that not only was T-Mobile “listening in” on his web activity/requests, it had determined that he was using a desktop computer paired with T-Mobile phone (not broadband) data service. And that type of network activity is a definite no-no on T-Mobile unless you are paying an additional monthly charge for the ability to tether other devices, such as a laptop, to your mobile data connection.

The way T-Mobile likely decided that Josh was tethering? The same way his destination web site decided to serve him that lame mobile experience: via the browser’s user agent string. And perhaps they combined this with knowledge about the number of gigabytes of data he’d already consumed that month. (He habitually uses around 2-3 GB per month, preferring to use his phone as his primary device for accessing the Internet, even at home.) And given that the display resolution of the Galaxy Nexus is a whopping 1280 x 720 pixels, even that would not have helped determine mobile vs. desktop.

So a disguise engineered by Google to let him successfully pass as a desktop user was convincing enough to trip T-Mobile’s “unauthorized tethering” alarms and end up blocking him from the content he requested in the end.

Even worse, more tests the next day revealed that Josh was now getting blocked by the “tethering upsell” screen when using Chrome for Android regardless of whether the “Request desktop site” option was checked. Apparently, T-Mobile had decided that all of his web activity while using any variant of Chrome was suspect, even when it was accurately reporting itself as a mobile browser.

What This Means

This little episode was, of course, frustrating (and triggered a testy, 20-minute call to a front-line technical support rep including lots of explaining to said rep about how web browsers and servers work). And someone at T-Mobile has logged the issue in some database somewhere where it may or may not be addressed in some manner (someday).

But the real lesson, I think, is in how users’ mobile browsing behavior is increasingly failing to map to outdated assumptions from both web site owners and mobile carriers about how their content and networks are being used. While iPhone users are known for chugging down data at a rate that far exceeds that of their other-OS’ed brethren, more Android users are trading their straws for steins as powerful new devices like the Galaxy Nexus or Galaxy Note are introduced, with their huge screens and fast, 4G network connectivity.

And, Jakob’s recommendations aside, a growing cadre of in-the-trenches designers and developers of experiences tailored for mobile and beyond understand how mobile is a user state, not a device. That context is increasingly difficult to pinpoint as users try to do anything and everything on their mobile device they would on a desktop computer. And companies who fail to shift with their users will be far likelier to, as Josh told the T-Mobile rep, “be tempted to take my business elsewhere.”


Revisiting Adaptive Mobile UX Design


Last month I had the pleasure of returning to Columbus, OH to speak at the inaugural M3 Conference, devoted to all things mobile. I gave an extended version of my Adaptive Mobile UX Design talk from the Midwest UX conference earlier this year, adding more on core UX considerations for a mixed designer-developer audience, as well as updating much of the material to account for all of the changes in the mobile space in the past six months.

I’ve posted the slides from my talk on SlideShare, also embedded below with a complete transcript of speaking notes. Thanks to all attendees, question-askers, live-tweeters and folks I got to hang out / geek out on mobile with.

Transcript:

[Slide 1]
Hi. My name is Jen Matson and I’m an interaction designer and user experience architect. Today I’m going to talk about ways of crafting a great user experience for the mobile web.

[Slide 2]
But there are no simple directives for doing so. And insights about how to make the right design choices often come from trial and error, as well as experiences we have as users. That was my path — a winding one. So I’m going to share with you a little of not only my thinking, but my process.

[Slide 3]
Think of it as a treasure map, but instead of one big “X,” there are a number of different stops on our journey, from a visit to a Sears store, to a look inside our users’ heads, to an exploration of some myths, to an examination of some concrete technologies you can use. First stop: Sears, or how I came to be there.

[Slide 4]
At the beginning of this year, I converted a spare bedroom in my house from what was essentially a storage space into a home office. Fortunately, my desk is right in front of a nice big window. UNfortunately, it’s a single pane, old wood window, in my 100-year old uninsulated house. The room stays cold, and I realized pretty quickly I needed a space heater.

[Slide 5]
Since I didn’t want to wait for one I’d order online, I decided to purchase one from a store near me. I did go online first to research features and prices, but then I headed over to my local Sears, since I figured they’d have a pretty extensive appliance selection.

[Slide 6]
I go to the store, head upstairs to where the space heaters are, and find the one I want. The price was a bit higher than at other stores online, but still okay. Even so, I’m a savvy shopper, and I wanted to see if Sears maybe had a price-matching policy.

[Slide 7]
So I took out my smartphone, and did the following Google search in my web browser: “Sears price match policy” Great, a web page with that exact phrase for the title, at the sears.com domain. So I tapped on the link to view it. But this is what I got:

[Slide 8]
“The server has not found anything matching the Request-URL. ERROR 404 Not found”

Not good. Where was the web page that Google had tantalizingly dangled in front of me? But looking at the error page URL, I see:

“m.sears.com”

Ah, sounds like a mobile URL. So the Sears web site KNOWS that I am on a mobile phone, but it can’t use that information to provide me with the appropriate experience based on the content I’m looking for and the context of me, standing in their store. So then I went directly to “m.sears.com”, got their mobile site.

[Slide 9]
I repeated my search phrase there. But I didn’t really get anywhere there, either, just over 64 thousand results for things like jewelry. Obviously searching their product catalog, not the site.

I even tried to go to www.sears.com, but I kept getting redirected to the mobile site. There was no way I could get to that page with the price match policy info from my phone.

Overall, not a good experience. And I’m not just talking about the mobile web site. While I did buy the space heater anyway, the entire process left me pretty grumpy.

[Slide 10]
What we have here is a failure to adapt. The Sears.com site couldn’t adapt to the combination of an incoming search query from a mobile device to a page on their main web site. They actually blocked me from getting to information they did have on their main site. I certainly hope web experiences like this do become extinct.

[Slide 11]
Now this story dates from January or February of this year. So I wanted to check in and see how Sears is doing, mobile-wise. If you type “sears.com” directly into the address bar, you get taken to what looks like a new mobile site, much improved in both UI and functionality. But unfortunately, repeating my little scenario by following the top link on Google for “sears price match policy” resulted in…

[Slide 12]
This. Come ON, Sears! Clearly, there is still work to be done.

[Slide 13]
So, what is Adaptive Mobile Design? It’s an approach to creating web sites and applications that try to give each user the best possible content and experience, tailored to their device and browsing context. And the “try to give” part in there is pretty important, since we can never anticipate all of the factors involved.

As it turns out, this approach is nothing new. Another industry has been doing this for hundreds of years.

[Slide 14]
The ad industry is the perfect example. Display advertising, in particular, is a specific medium where within the relatively two-dimensional constraint of showing an advertising message, it adapts to the user context. Here’s one from a classic roadside ad campaign:

[Slide 15]
(Burma Shave ad, part 1 of 5.)

[Slide 16]
(Burma Shave ad, part 2 of 5.)

[Slide 17]
(Burma Shave ad, part 3 of 5.)

[Slide 18]
(Burma Shave ad, part 4 of 5.)

[Slide 19]
(Burma Shave ad, part 5 of 5.)

[Slide 20]
Or this print ad, taking advantage of the then-novel full color printed magazine page.

[Slide 21]
Or Boston’s famous Citgo sign, where, in its pre-digital incarnation, shown here, the canvas was thousands of illuminated tubes of neon, lit up and set to animate.

[Slide 22]
The message adapts to, and sometimes even acknowledges the medium, as well as the setting. This tour bus ad, for example, is clearly meant for locals, as the tourists unwittingly become part of the ad.

[Slide 23]
And the same campaign, adapted for taxicab and subway placements.

[Slide 24]
Here we’ve seen three major design considerations: canvas, capabilities and context. As applied to mobile design, Canvas is the varying display sizes and resolutions of phones, tablets and other devices. The capabilities of the device, from the processor speed to the data connection speed, also play a role. Finally, context: where the user is, what they’re doing, and their attention level.

[Slide 25]
So here’s where someone might say, “Great! As long as we understand these three things, let’s jump right in and create a great mobile experience. We’re ready!” To which I’d reply: “Maybe.” Because there’s one big thing, that can be summed up in one little word, that needs to be considered first and foremost.

[Slide 26]
That description of adaptive mobile design? That middle bit: “understanding user needs.” Above all else, you need to know your user, not just her place in space or gadget she’s using, but what really motivates her.

And if you already have a deep understanding of what drives your users to interact with your product, service or site in the first place, then: “Yes, fantastic, you’re ready.” Because that kind of research — really, just talking and listening to your users, whether it’s through usability testing, user interviews or even just surveys — is an essential not just as a first step, but an ongoing process. But hey, you guys are already doing that anyway, right?

[Slide 27]
Well, if you’re not, then all bets are off. Unless you’re designing something for which the target users are pretty much you, there’s a good chance you’re making some wrong assumptions about why and how people will use your site. And we know how that one goes.

[Slide 28]
Because, the funny thing is, Sears actually got bits of the “canvas, capabilities and context” piece right. They anticipated that yes, I might want to view content tailored to the small screen of my phone, and Sears.com correctly detected my mobile browser. But by fixating on the presumed object in my hand, the experience is disjointed, untethered from the thoughts in my brain.

[Slide 29]
What’s my immediate task? Get information on their site I requested by tapping on a link. And that’s actually a sub-task of getting price match policy info. And that is the same task whether using a phone or computer browser. And my actual goal, the reason why I’m standing there in the store? To buy something. Presumably, Sears wants me to do that. And wants to support me in any way they reasonably can to help me reach that goal. Define your user goals and tasks, and *then* you can start to use those mobile considerations to better shape the experiences you design.

[Slide 30]
So, considering your user’s goals and tasks is essential, but getting that data can be kind of tricky. To get it right, you ideally need to be doing your own research. And while it’s not a substitute for user research, industry research can be amazingly helpful in providing general information about user behavior and context issues. Much of this research generously shared by mobile pioneer Luke Wroblewski on a regular basis on his web site, at lukew.com. And mobile analysts like Horace Dediu and Michael Mace help make sense of this data, synthesizing, charting and reflecting upon it, to create a multi-dimensional view of what’s happening in the industry based on observing mass mobile purchase and usage behavior.

So along with insight about your site’s users, this research is valuable in helping to create a realistic picture of the mobile Internet user.

[Slide 31]
In fact, actual behavior is so varied, it can be most helpful to start training your brain to NOT jump to certain assumptions about mobile Internet usage. Mobile designer/developer/author Josh Clark has written and spoken quite a bit about some of these mobile context myths, but I’d like to review a couple of which are probably the most important — or dangerous:

[Slide 32]
Well, of course! What makes it mobile is that you’re, well, mobile. Some of the time, yeah. Catching up on email and news while sitting on the bus headed to work is a perfect example. But I’m also using a mobile device while sitting on the couch, watching an old episode of “Law & Order,” thinking “hey, that actor looks *really* familiar” and then grabbing my phone on the coffee table to hit IMDB to do a quick search. Both scenarios are equally valid.

[Slide 33]
Even if certain mobile carriers focus on the first one. Over and over again. After all, if we’re NOT on-the-go, then we’re less likely to be using their service, right?

[Slide 34]
But the research data tells that nuanced story that reflects my mixed home-and-away usage, and probably yours. In fact, an even *larger* number — 84% — report accessing the Internet on a mobile device while at home.

[Slide 35]
This myth stems somewhat from the previous one, in assuming on-the-go usage can lead to justifications for removing content or features to presumably better serve these distracted, impatient nomads. And sure, it’s much easier to chop things out from your desktop web site to fit a small screen than re-think your site structure and navigation to best accommodate mobile use.

Here on the “desktop” version of the web site for retailer Nordstrom, we see a lot of ways to shop, and links to support that activity by getting users to the right departments and brands.

[Slide 36]
Another feature that is very well-liked by users, and is an especially popular promotion around the holidays is the free shipping offer, something proven to help increase sales. Here we see it promoted in two different spots. Clearly, it’s important, right?

[Slide 37]
Well, apparently not so important to mobile shoppers, as the mobile version of Nordstrom’s site omits any mention of free shipping. For some reason, different content choices are being made here, and presumably not to optimize for the mobile context. In fact, this disconnect is likely less a choice and more an accidental byproduct of not synching sales channels. But it’s still an example of treating mobile users as second-class shoppers. And while there is a link to the view the full site at the bottom of the page, that’s only useful for shoppers who have a specific task in mind they can’t complete using the mobile site.

[Slide 38]
Oh, and I almost forgot — before I was able to see the Nordstrom mobile site at all, I had to suffer through this pop-up, one of my main mobile site pet peeves. You have an iPhone app. Congratulations! Now can I get back to shopping for a new pair of boots already?

[Slide 39]
Someone might say, well, people are more likely to shop online using their computer. Isn’t it hard to see pictures of clothing on a tiny screen? Well, 25% of U.S. smartphone owners, about 22 million Americans, say that they mostly go online using their phone, rather than with a computer. Those people will likely never see the “free shipping” message on the “main” Nordstrom site, because, for them, the mobile site *is* the main — and only — site.

[Slide 40]
So, you know your users and what they want, and you’ve got a better understanding of what’s fact vs. fiction through industry research. All you have to do now is start creating that experience. How do we apply them to the design of mobile web sites?

[Slide 41]
There are many different approaches you can take, everything from creating a separate mobile site to creating a single site to serve all devices and contexts.

Crafting a bespoke site or app is great, if you can manage it. But the fact is, it’s time- and resource-intensive. And it can be tricky figuring out how to gracefully integrate and manage a number of elements — work streams, strategy, content — across multiple sites.

It’s also a big challenge to redesign an existing site to be truly responsive. The tools and technologies are there, but to successfully implement such a site, it takes both developers and designers with intimate familiarity with all of their capabilities, limitations and differences cross-platform, -browser & -device. That’s a pretty tall order, for even the most skilled teams.

An approach that’s likely to work for a larger number of companies, especially those looking to make incremental improvements today, is to be pragmatic. Apply new technologies to your main site, where it makes sense, in order to improve the mobile experience for your users.

[Slide 42]
And these new technologies? HTML5 and CSS3. These two are just the latest versions of both HTML and CSS, used to structure and present web page content. But they are chock full of new features — too many to cover here, in fact. The following are just those most relevant to the mobile browsing experience.

[Slide 43]
First, HTML5. There are five features — four big ones, and one little one — that we’ll be looking at:

– Smart web forms, with form input UI changing based on the form field type.
– Geolocation, where the site can know your location and use that info.
– Dynamic device orientation, where the site gets motion tracking info from your phone.
– Web-native video playback, such as what Apple uses to display videos without the use of Flash on its iOS platform.
– And, semantic web markup, which is less a feature than an architectural change

[Slide 44]
Here we’ll look at smart web forms as implemented on sites viewed with the iPhone’s Safari browser. Shown is the default soft keyboard, a Qwerty one with all letters. Since space is limited, numbers and symbols requires toggling to different keyboards.

[Slide 45]
But if we go to eBay’s mobile web site, we can see one of the new input types in action. On this page, in order to bid on this Go-Betweens record, I would tap in the field for “USD” (dollars), where I want to enter an amount.

Since the field value must be a number, eBay has specified an input type attribute value of “number” for that field. So when the soft keyboard appears, the version shown is numeric, not the default Qwerty one.

And here is what the HTML code for that would look like. Since it’s a new attribute type, it’s simply ignored in older browsers without any ill effect.

[Slide 46]
Here’s another input type, on MailChimp’s web site. When you go to sign up for an account, there’s the familiar field for inputting an email address. Tap on the email field…

…and you get the Qwerty keyboard, but slightly modified, with the “@” symbol and a period sharing space with the space bar. This way, the user can enter an email address without having to toggle back and forth between the different default keyboard states.

And here is the code for that feature.

[Slide 47]
Next geolocation. This is something that is incredibly common in mobile apps, such as Google Maps, where it detects your current location to plot a course. But this is something that web sites can do, as well. On some platforms, such as the iPhone, you’ll need to explicitly turn on the ability for the web browser to use geolocation, as it’s turned off by default.

Assuming you’ve turned this feature on, Old Navy’s mobile web site has a store locator that uses geolocation. If you tap on the Find Store button…

…you’ll first get an alert asking you if you want to let this web site know your location. If you tap on “OK”…

You’ll automatically get a list of locations nearest you, without having to enter or tap on anything additional.

[Slide 48]
The next feature is dynamic device orientation. Like geolocation, this is something that’s being used in mobile apps now, primarily for games. Web applications of this feature are still pretty few and far between, but there are some demos online showing exactly how the movement of a device in-hand can effect objects onscreen.

Here is a brief video showing me using one of these web demos on my phone.

[Slide 49]
Another, more common feature, is web-native video playback. The de facto standard for video playback on the has been Flash, which isn’t a true standard at all, but a proprietary technology owned by Adobe.

Apple’s decision not to support video playback using Flash has given HTML5 video a real boost, and largely because of that, sites like YouTube and Vimeo have been adding HTML5 video support.

Here are a couple of examples showing how iPhone and Android each handle things. On the iPhone, tapping on the “Play” icon for this particular video…

…Triggers playback using the native iPhone video player. Safari hands off the request to that app.

[Slide 50]
On Android, things work a little differently. Again, seeing the same video play icon, tapping on it…

…brings up a couple of programs from which the user can choose to play the video.

Handling video natively, each mobile platform gets to provide an experience that best meets the expectation of its users, instead of applying a one-size-fits-all approach.

[Slide 51]
Finally, semantic the new semantic tagging structure that HTML5 uses, something that should warm the hearts of information architects everywhere. Instead of faceless divs and spans that need classes and IDs to give them any meaning, the new content containers *themselves* have meaning. When we specify “nav,” “header” and “footer” in a wireframe, those elements can now be coded with “nav,” “header” and “footer” tags.

This also happens to be important for findability, as search engines are increasingly looking for structure to help apply meaning when parsing web page content. Properly structured and tagged content, especially when semantically tagged, will be more likely to be indexed properly and given greater prominence in results.

Those are the HTML5 highlights. Next…

[Slide 52]
CSS3. The story here for mobile is pretty much CSS Media Queries, whereby custom stylesheets, which determine web page layout, styling, and even content, can be served up for different screen size, page orientation and resolution. This may sound like a fantastic way of tailoring your content for mobile, and it is. But media queries are not the silver bullet for your mobile solution. Ensuring you don’t serve desktop assets to devices on slower connections, for example, is something that is trickier and requires additional techniques to achieve.

[Slide 53]
A good example of a design that adapts to different screen sizes is the web site for northwest music festival Sasquatch. Here we see the full page layout, viewed in a web browser, close to fullscreen, on my laptop. But when viewed on my iPad.

[Slide 54]
…The images and other content scale accordingly, filling the entire screen in way that perfectly suits this browsing context. This, instead of presenting a zoomed-out view of the “full-size” web page. Or even worse, a page with the right side cut off and a dreaded horizontal scrollbar.

[Slide 55]
And on the iPhone, the smallest screen size, you can see how the design once again undergoes a transformation. The heading design is completely different, in order to fit into that small space, and no attempt is made to show the full navigation bar, which likewise wouldn’t fit.

And next to the screen is the bit of code that shows how a phone-specific stylesheet is served up via a media query that says: “use this design when this content is viewed on a screen, with a maximum device width of 480 pixels.”

[Slide 56]
Orientation is another media queries feature. Here we have two screenshots from my iPad of a web site that changes the design based on the dimensions of the browser window: blue if the window is between 400 and 1000 pixels wide, red if it’s wider than 1000 pixels. Above is the code that specifies which stylesheet to use for which orientation: landscape or portrait.

[Slide 57]
And the third feature is screen resolution. Here are three different phones, each with a different screen pixel density. The oldest phone here, the iPhone 3GS, can show 163 dots per inch. The Samsung Galaxy S has a 233 DPI display. The best picture is on the iPhone 4 — with it’s “Retina Display” it can show twice the number of pixels as the previous generation iPhone, at 326 DPI.

Why do these things matter? By targeting screen resolution, you could serve up an entirely separate set of high-quality images to users with displays capable of viewing them in all their fine-detailed glory. Otherwise images designed for a lower resolution display may not scale properly.

And the code for that media query.

[Slide 58]
Of the features mentioned today, it’s important to note that while not all are currently supported by browsers on the most popular smartphone platforms, the majority are. This is an excerpt of a chart from mobilehtml5.org that covers the features I’ve mentioned and many more. So you should refer to it when deciding whether or not it makes sense for you to use a certain HTML5 or CSS3 feature for your mobile site. Even so, it can only show you what the latest and greatest devices are capable of, as most Android users, for example, are stuck using older versions of that OS due to lack of updates from carriers or OEMs.

[Slide 59]
So, I know that’s a lot of information to absorb. But, to wrap up:

You’ll already be doing something right by considering any device, any context, any screen as part of your design process.

Research is essential to get a mobile site or app right, since there are so many people doing it wrong.

Modern smartphone browsers already have good HTML5 and CSS3 support, so you should start using these techniques now. And of course ensure your sites are build in a way that browsers without those capabilities are still able to get essential content and functionality.

Finally, in order to create mobile web experiences that are both adaptive and exciting, it takes close collaboration between designers and developers, with each ideally moving closer to the other’s role: designers familiarizing themselves with code to understand capabilities and create prototypes, and developers getting involved as early as possible in the design process, to help shape the discussion with their in-depth knowledge of platforms and technologies.

[Slide 60]
Here are a few great resources that have more information relating to some of the ideas presented today, including some folks I’d recommend following on Twitter for the great links and other mobile web-related content they post on a frequent basis. All the sites have tons of examples, many of them interactive, so you can see these and other techniques in action.

[Slide 61]
I hope you’ve found some tangible things you can use in crafting better mobile experiences. I also hope that you’ll find — as I did — that taking the first steps on that path by putting yourself in the minds of users, and starting to use these techniques to enhance your existing sites, you’ll be able to see how rewarding an adaptive approach really is.

[Slide 62]
Thank you.


This Is (Web) DIY

In was the fall of 1994, and I was but a lowly intern at Salem, MA-based record label Rykodisc. Specifically, I was the “online” intern, primarily responsible for maintaining the label’s forum on the Delphi ISP1. Commercial online services like Delphi were essentially separate from the then-nascent web, and only the largest services had fancy, proprietary graphical interfaces. So Delphi’s forums were text-only, though ours had downloadable cover art, plus a few rudimentary sound clips. And since I had interned for Delphi that summer, I was already pretty comfortable with my forum management duties and was looking for a new challenge.

So while my boss Lars frantically tried to get us a spot on the then hip-and-happening online place2, I decided to attempt creating a page on this new thing called “The World Wide Web.” A Delphi co-worker3 had shown me a bit of the web a few months earlier, and I, like many others who stopped by his computer to take a peek, were immediately seduced. It wasn’t just that there were pictures, though that was indisputably cool. It was the whole, hyperlinked way of interacting with information, the self-published free-flowing-ness of it, which left the green-text-on-a-black-screen, press-a-number-to-select, BBS-style interface and walled-garden paradigm we were used to in the dust.

Armed with a SLIP connection4 to a local ISP and a 14.4k Modem, I fumbled my way around creating my first page, starting by viewing the source HTML code behind the web pages I visited, and pasting the contents into my text editor to mess around with and preview in my Mosaic browser.5 Then I bought a book, one of the first on HTML6, which helped me build upon the fragments of knowledge I’d picked up by trial-and-error.

And since creating a web page was free7, my boss was all for it, even if, as I warned, I wasn’t sure how many people would be able to view this page. It might only be visited by students and academics, the only large demographic groups likely to have Internet access. The vast majority of the online population then, including anyone using the major ISPs — AOL, Prodigy, CompuServe — still did not have access to the web.

It was in this atmosphere of “what the heck?” that I created my first professional web site, in fact my first web page ever, for Rykodisc. My toolbox was limited, as there was no CSS and no JavaScript. The first iteration of HTML didn’t even allow for using different fonts, any page background color (beyond the default gray), or tables, which, in the pre-CSS era, would be used (and abused) in the service of visual design.

What I did have: a collection of links to web pages for Rykodisc artists I’d gathered via web searches, digital album artwork scavenged from the art department’s file server (including that nifty Dan Clowes piece from a Ryko sampler CD), and witty, brand-appropriate copy written by Lars. That’s right, early web pages were all about the content, if only because there was precious little design one could apply.

But I did my best with the design, as well as the code. And it all came together in the page you see below, with historically-appropriate gray background added in via CSS prior to taking this screenshot. While I didn’t save the exact first iteration of that page, this one, from the very beginning of 1995, is essentially the same design and content. (You can also view a full-size version.)

It may not be a pretty web page, but it served its purpose and started me off on a path I continue to this day. Figuring out how to build something from nothing, even when I hadn’t a clue… that was heady stuff. And it’s one of the reasons why I continue to preach the power of prototyping to other UX and design folk. That is to say: I did it, and so can you.

Rykodisc Home Page - January 4, 1995

1. Delphi’s two main claims to fame in the footnotes of Internet history: a) Edging onto the list of Top 5 nationwide Internet Service Providers, and b) Getting purchased around that same time by Newscorp, in one of their first in a string of ill-fated online forays.

2. AOL, of course.

3. Renowned writer Jimmy Guterman, who I also recall assisting in what would now be called his “liveblogging” of the Rolling Stones’ “Voodoo Lounge” tour kickoff for Delphi.

4. Serial Line Internet Protocal. Pre-PPP, yo!

5. This was just before the first Netscape beta was released.

6. Laura Lemay’s Teach Yourself Web Publishing with HTML in a Week (Sams Publishing, 1994)

7. Even — believe it or not — the domain registration at that time. Also free.