User Research & Testing

We build digital products for our users. Without user research, we won’t know if our digital products are truly serving New Yorkers’ needs, or just our assumptions of those needs.

Usability Testing

Many different research methods can benefit your projects, from secondary research, to immersive observation, to co-designing with users. Usability testing is one of the more straightforward methods. These tests measure how well your website or app is meeting objectives by having representative users use it to complete realistic tasks. By observing their experience, you can determine how well your content and design are serving them.

group of people sitting at desk while looking at computer screen together - a user research testing session in progress

You should test with multiple users representing the different groups of people who may use your site or app. A good aim is 5-10 people per user group. While it’s important to do testing before and after your initial launch, we recommend regular testing throughout the life of your product.

DigitalGov offers a Usability Starter Kit, which includes downloads for template testing scripts, plans, and reports. Usability.gov also has many useful explainers on user testing topics.

Intercept Testing

You won’t always have time to extensively plan your usability tests. Luckily, there are plenty of easy ways to observe your users and get feedback on products before (and after) you launch.

Quick and accessible, intercept, or “pop-up” testing is a good way to get basic feedback from New Yorkers on your projects. In this method, the tester will go to a public place—like a local park, community center, or the lobby of your building—and ask passersby for input on a design or product.

If your product will be used or administered by City employees, your own office can serve as a perfect testing ground.

Prototypes

It’s easier to test and revise prototypes than finished products. Users may also feel more comfortable sharing negative feedback about a project that seems in-progress rather than complete. Prototypes can be simple sketches on a piece of paper, a clickable prototype made on a program like InVision, or an “alpha” version of your product.

Finding Users

Make sure you’re testing with the right population. If your project is aimed at parents of young children, people with limited English language abilities, or people with physical disabilities, for instance, make sure to include these representative test participants.

If you’re setting up intercept tests, try to do so in a location where you can expect to find a diverse group of test participants. If you’re looking to speak with a small group of individuals, consider talking to your agency’s correspondence division, or contact the Digital Blueprint team for help finding the right people.


Other Research Methods

The Civic Service Design Tools + Tactics include some tried and true methods for conducting qualitative research beyond usability testing. These include:

 

Another great resource for design research methods is 18F’s design methods cards. Use this comprehensive inventory of methods—including card sorting, personas, and journey mapping—to pick the right exercise for your project phase.

Policies

Compensation

You’re allowed (and encouraged) to offer test participants some kind of compensation to show thanks for their time and insights. Tokens such as tote bags or mugs are good for intercept testing, while dollar amounts or gift cards are appropriate for those who offer more of their time. Check with your counsel to see if there are limits or constraints around what you can offer as compensation.

If you’re planning to record users in any way, you must have them sign a consent form that has been approved by a member of your agency’s legal team.

As jumping off points, use this sample consent form for interviewing adults and this sample consent form for interviewing minors prepared by NYC Opportunity’s Service Design Studio. Check with your agency’s counsel before finalizing.