Research

Approach

My approach is iterative, data-driven, and human-centered.

I make actionable recommendations based on observation and analysis. I always keep people’s needs and values in the front of my mind, whether I’m seeking to understand their behavior, make their lives easier, or nudge them in new directions.

My favorite design challenges involve encouraging people to participate in familiar activities in new ways, or creating opportunities for them to make new kinds of contributions.

Methods

Qualitative Interviews, surveys, usability, digital ethnography, grounded text analysis
Quantitative a/b tests, stratified sampling, inferential stats
UX Design workflow diagrams, wireframes, paper and interactive digital prototypes

Tools

Languages Python, Lua, MySQL, HTML/CSS, JavaScript (just a smidge)
Design Omnigraffle, Axure, Adobe, Qualtrics, Pen, Paper, Scissors


Projects

Learning pattern library


2013 – 2014

Questions_pattern_infobox

a learning pattern: Asking the right questions

I am developing a collaborative design pattern library where Wikimedia volunteers can record important lessons they learn when they work on projects funded by Wikimedia grants. Learning patterns are a concise, actionable, and evidence-based format for capturing key lessons and sharing them with the rest of the Wikimedia movement. They can be incorporated into project plans and reports, making it easy for all Wikimedians to benefit from one another’s wisdom and expertise.

The value of the Learning Pattern Library increases over time as key patterns are expanded and endorsed by community members, and brand new patterns are added to address additional important considerations for the design, execution and evaluation of community-driven projects.

Artifacts


The Teahouse


2011 – 2012

caption

The Teahouse main page

A collaborative community forum that provides new Wikipedia editors with early, personalized peer support in an engaging and user-friendly environment. The Teahouse addresses several root causes of the editor decline and the gender gap: Wikipedia’s sometimes-hostile social atmosphere, complex rules, and the technical intricacies of editing.

I led an end-to-end UX research program for the Teahouse pilot. I distilled design requirements from preliminary research and previous work; developed wireframes, user scenarios, instrumented and automated the site, and evaluated user experience and the impact of Teahouse participation by analyzing survey results and behavior traces.

Th_wrapup_new_editors_per_week

Sustained engagement

The rewards of this project went beyond the positive initial results, which were that both new and veteran editors loved it, more newcomers stuck around, and lots of women participated. In addition, the Teahouse has become an integral part of the new user experience of Wikipedia. Thousands of new editors have benefited and the project has been entirely run by dedicated and industrious volunteers for almost two years now.

Artifacts


The Living Voters Guide


2010 – 2011

lvg_pro_con_separation

Considering tradeoffs

How do can we crowdsource deliberation on hot-buttoned political issues? I helped develop the Living Voters Guide (LVG) over six months prior to the 2010 State election, and worked on a redesign of the LVG interface in the run up the a second deployment for the 2011 Washington state ballot.

Prototyping the LVG

Prototyping the LVG

I was able to make many awesome contributions to the LVG design process and the final product. It was a small team and we all wore many hats. I developed feature enhancements and performed iterative user evaluations, working from wireframes, scenarios, and paper prototypes to a seven-participant lab study. In the process our team refined our ideas, unlocked new user requirements and opportunities, and developed an entirely new paradigm for online collaboration! An exciting design space to work in, for sure.

Our hard work snagged us a good deal of press and we were short-listed for a 2011 Washington Technology Association award, for good measure.

Artifacts


Reflect


2010 – 2011

reflect_screenshot_ideasforseattle

Reflect on IdeasForSeattle.org

Getting people to talk on the web is easy. But how do you make them listen? Reflect is a context-linked sidebar for threaded online discussions. Its purpose is to provide people with a more reflective way to respond to a comment: summarizing key points, rather than responding ad nauseum.

The challenge for this project was testing whether people grokked it. Did they understand this was a new mechanism for communicating? Would they use it the way we intended, and what impact would that have on the conversation as a whole? I ran a series of user studies on a live prototype of Reflect implemented on IdeasForSeattle.org, a locally-minded online idea forum, to work out the bugs. Then I worked with Reflect’s creator, Travis Kriplean, to analyze the results of a deployment on four stories on SlashDot.com. We developed a way of categorizing Reflect responses according to their content.

Did people use bulleted summaries to reflect back meaning?

Did people use bulleted summaries to reflect back meaning?

Our analysis showed that Reflect added a productive new dimension to the discussions, and that people did use Reflect to highlight key points in other people’s posts and summarize or re-phrase them. Although sometimes they did so sarcastically.

Sarcasm? on the internet? what a surprise…

Artifacts

  • The test kit for an early user test of Reflect, and some notes and quotes from those study sessions
  • slides and notes from a presentation on Reflect I gave at the 2010 Wikimania conference
  • research paper published at the 2012 Human Factors in Computing Systems conference
  • a short video showing Reflect in action on IdeasforSeattle.org

Apocrypha

All the research that fits

Consulting work


2007 – 2011
discovertext logo

Texifter I ran a 3-person user study and a 6-person focus group & workshop to gather user requirements and test end-to-end functionality of the DiscoverText collaborative text analytics platform. This two day study captured metrics and pain points related to basic usability, as well as evaluating the collaborative capabilities of the software, and generated insights into user wants and needs that informed a planned interface redesign.

infospace interface

Infospace In the dark days before smartphones, I ran a study for Infospace comparing the usability of their mobile web search interface to that of a young upstart, Google. While lab-based, the study ran participants through realistic mobile usage scenarios such as checking weather and movie times, and mapping local addresses. I measured clicks and time-on-task for both interfaces, and also captured subjective evaluations of desirability and ease of use.

bedynamic widget

BeDynamic I ran a set of user studies of a Flash widget that incorporated browsable dynamic destination content into BeDynamic’s customer and partner websites. I focused on ease of searching and browsing for events and local points of interest, map integration, and content categorization.

Artifacts


Datasets & repos

Teahouse corpus A set of 5,000 questions asked at the Wikipedia Teahouse. Provides rich insight into the lived experience of being a Wikipedia newcomer (stranger in a strange land!). Potential applications include Wikipedia editor engagement research, social Q&A research, network analysis, and natural language processing.

AAWD corpus The name stands for “Alignment and Authority in Wikipedia Discussions”. Wikipedia talk page discussions (English, Russian, and Chinese) labelled for linguistic cues and rhetorical moves related to group dynamics. The broader SCIL project dataset also contains labelled data from broadcast news program transcripts and a set of task-based group collaborations on IRC. Potential applications include corpus linguistics, natural language processing, machine learning.

Github codebase for two Python-based bots I run on en.wikipedia and meta.wikimedia, plus the odd research script. Needs a visit from the documentation fairy.