I conducted a contextual inquiry with developers with different years of experience, familiar and unfamiliar with a government agency that owns an API platform to find out if they could successfully use the documentation, and understand the roadblocks that engineers could face. I talked to 7 developers, provided them with a hypothetical scenario and asked them to build an app to solve the problem presented using one of our APIs. I asked the developers to share their screen with me, and to talk out loud about their process.
To put them at ease because the exercise can be similar to coding exercises for job interviews, I repeatedly emphasized that we weren’t testing their ability and that we wanted to know how they used the documentation.
We found out that:
Developers will pivot fast and they need to find solutions quickly.
For that reason, they wanted examples of cURL commands, a list of required parameters, and informative error messages.
Descriptions of the data in plain language and avoiding agency-specific jargon so that they could assess whether to invest time developing against the API.
Developers work in teams and need easy ways to handoff work. This insight became relevant for a second study where developers expressed the need to handle API keys as a team.
I conducted this research study as we were setting up the platform. This first API usability study informed changes to the documentation, and laid the foundation for continued and iterative usability research on the API platform.