PROJECT OVERVIEW: Due to tight project timelines, our team did not have the luxury of time to perform research on our users or test designs prior to site launch. We improvised by auditing our competitors, learning from industry trends and finding inspiration in alternative media outlets such as Medium & Twitter. In one short month, our team of four designers churned out over 10 original design templates for the redesign of the LA Times. These templates were eventually incorporated across all 10 major Tribune Publishing properties. During the beta launch, the rapid pace of the design phase had slowed down enough to where we had a chance to validate and iterate upon our designs.

GOAL: Use a variety of user testing methods to:

1. Gather feedback around beta site 

2. Find areas for improvement and see how we can iterate

MY ROLE: UX Researcher. I was in charge of coming up with tasks and questions for in house guerrilla testing as well as recruiting testers. In addition, I managed a feedback tab on the site; I came up with survey questions, monitored the results & synthesized them for upper management on a weekly basis.

IN HOUSE GUERRILLA TESTING: Method: I recruited 12 users, a mix of friends and colleagues who worked across various professions - from Human Resources, to Business Strategy to Data Analytics. The participants were split into two equal sized groups - one would give feedback on the desktop article page, the other would give feedback on the mobile article page. We focused largely on the article page since that is the most trafficked template of the site. None of the users had much familiarity with the design and most had not been exposed to the beta experience before. I also conducted 1 on 1 in person interviews with each user, capturing their thoughts on the desktop experience as well as the mobile experience. Since we wanted general feedback, user tasks were kept very basic to allow him or her the freedom to navigate freely and share their thoughts openly. Notes from interviews were consolidated and the most common issues were ranked from highest to lowest for both desktop and mobile.

USER FEEDBACK TAB: In addition to in person interviews, we wanted to hear directly from our readers about what they thought of the new site. Via UserZoom, we implemented a feedback tab that presented users a short survey.

I monitored the responses on a weekly basis, dividing out the responses across desktop, tablet & mobile users to find common themes across the different devices. I also measured and tracked the NPS score across the different devices to see how iterative changes (if any) may have affected how users felt about the site. A weekly report of findings was also created and shared with upper management.

LEARNINGS: Ideally, we would have done the testing prior to launching our beta, but since we were on a time crunch, this was the next best thing. Having feedback from actual users was really helpful. As a result of our user testing, we were able to identify issues that we’d not been aware of and iterate based on our findings.

There were design decisions that we’d made that we thought were great ideas, but found out that they were confusing to users. We had a down arrow next to ads that we thought helped indicate that there was more content, but users often wanted to click on it even though it was inactionable. Our sticky article recommender at the top of the page was a nuisance to readers and took up a lot of valuable real estate. We ended up having it scroll off screen. The “TOPICS” wording for the navigation was confusing to users who were used to “SECTIONS”. We ultimately changed it back. The feedback from both the guerrilla testing and user feedback tab were incredibly helpful in helping us refine our designs as we moved out of the beta period.

User Testing Tasks

Tasks were kept fairly general to allow for natural browsing and exploration of a news site.

Notes from user interviews

Most commonly mentioned issues across desktop & mobile, as well as general feedback.

Raw data from feedback tab results

Analysis of user feedback