Case Studies
Recruiting Hard-To-Reach B2B Users for Design Research Programs
Facing the challenge
The biggest obstacle associated with B2B user research is recruitment. Identifying experts qualified to comment on niche products and services can be daunting, and convincing those individuals to spend time participating in research is tough. This is something I’ve experienced first-hand while leading B2B UX research programs in various industries.
When I worked at Vericast, I needed research participants with executive leadership experience in our target market so I could study potential use cases for a new marketing technology platform. I also needed current clients to participate in a longitudinal study so that I could collect ongoing feedback as we iterated on the design of the platform.
In my current role at Vanguard, my entire team needs routine access to senior leaders working in HR benefits and finance for research on retirement plan sponsors. Though some of our needs can be met through quick, asynchronous survey methods, we often require significant time with research participants for one-on-one qualitative sessions.
Across these roles, I’ve found that incorporating three specific strategies into research recruitment efforts can significantly increase engagement for hard-to-reach B2B users.
Building rapport before making the ask
Tactics that work well for B2C recruitment are often ineffective for B2B recruitment. B2B users tend to be busy professionals not looking for opportunities to participate in research. Inundated with communications, they are unlikely to respond to unsolicited emails about user research from people they’ve never met. Given these circumstances, it can be helpful to spend considerable time building rapport with B2B users before asking them to participate in research studies.
One option is to imbed research recruitment efforts into more general business events that have broad appeal. At Vanguard, for example, I collaborated on the creation of a new client roundtable series for retirement plan sponsors. Instead of receiving an invitation to participate in a research study, clients receive an invitation to participate in a social event where they’ll have the opportunity to network with industry peers, learn about emerging trends, and influence the development of Vanguard’s products and services.
This year we provided the opportunity to meet with several Vanguard experts over dinner and discuss How America Saves, the company’s influential annual report on retirement savings trends. The response was positive, and clients were enthusiastic to set aside several hours to attend.
The event was not focused on UX research, but my role as facilitator allowed me to establish personal connections with individual clients. Later, when I reached out with research recruitment materials, I was in a strong position to ask for a favor. I wasn’t just some anonymous researcher spamming their inboxes with requests for time. I was the friendly, thoughtful researcher who had hosted them for an evening of delicious food and thought-provoking conversation.
Making research mutually beneficial
The small monetary incentives that entice B2C users to participate in research will rarely impress B2B users. And in some highly regulated industries, researchers may even be prohibited from compensating B2B clients for their participation in certain types of studies. In this context, recruitment efforts will be more successful if research activities are crafted to benefit research participants in creative ways.
Generating unique status and recognition for B2B user research participants is one way to achieve this. For example, at Vericast, my team created a formal title for the early adopters of our new platform who agreed to participate in extensive research activities over the course of a year: Client Advisors. We did not merely invite clients to fill out a single survey or complete a single interview; we invited them to serve as trusted advisors on a significant collaboration.
Even though it’s common practice to keep the identities of research participants confidential, we understood that, in this case, clients would find value in having their identities publicized widely. This is why our offer included opportunities for Client Advisors to be featured in marketing content promoting our collaboration. Clients were eager to associate their brands with our innovative work and therefore viewed participating in research activities as worthwhile.
Getting help from trusted partners
In many organizations, efficient B2C user recruitment processes are well-established, allowing researchers to build samples and conduct studies on-demand. However, B2B user research often requires bespoke recruitment processes, making it challenging to drive efficiency through standardization. Therefore, it’s crucial for researchers to maintain a network of trusted partners who can assist in recruiting B2B users through a variety of different mechanisms.
When recruiting existing B2B clients, it’s wise to enlist help from colleagues in client-facing roles, like relationship managers and account directors. These colleagues have already built rapport with clients and can facilitate personal introductions for researchers. They can also target potential recruits faster and more accurately than a typical screener survey process.
When expanding beyond existing clients, it can be beneficial to hire external vendors specializing in B2B recruitment. Companies like GLG and NewtonX can recruit single samples or entire panels to meet even the narrowest criteria. As third parties, these vendors can also ensure confidentiality for firms wishing to gather data without revealing their identity. Although these services can be costly, they often save significant time and achieve results that would otherwise be unattainable.
Creating New Tools to Support Inclusive Design in Financial Services
Defining objectives and exploring best practices
During my time at Ally Financial, I collaborated on several initiatives to make our UX design processes more inclusive. The bank had historically attracted customers with healthy finances who wanted to park their extra cash in no-fuss, high-interest savings accounts, but many of the new products and features being developed were intended for people who struggle with money and have different needs.
As an active member of the UX Inclusive Design Group, I collaborated with designers, researchers, and accessibility specialists to implement process improvements supporting more inclusive customer experiences. Our goal was to position inclusive design as a necessary practice, not an optional area of interest, so we looked for ways to integrate structured activities into our established routines. Rather than start from scratch, we dedicated several weeks to learning about best practices from industry leaders.
Inspiration for our own ideas came from two key sources. First was The Designer’s Critical Alphabet by Lesley-Ann Noel, PhD, which provides definitions and question prompts for key terms in critical theory. We were drawn to Noel’s work because it guides design teams through meaningful dialogue in a formal but approachable way. Noel was kind enough to meet with our group and advise on how to incorporate her cards into our project.
Our second source was the Conscious Design Canvas by Ben Evans, a template that asks teams to define both the obvious and non-obvious users they are designing for. After attending an InVision event where Evans explained how the tool challenges biases, we concluded that a similar canvas could help our teams develop empathy for marginalized users.
Designing a template for critical reflection
Next we adapted Noel’s cards and Evans’ canvas for an original workshop activity called the Inclusive Design Canvas. We concept-tested our initial prototype with a pilot project and used participant feedback to refine the design.
The Inclusive Design Canvas uses the critical alphabet to help participants identify specific types of unintended users for a given design. Then a series of discussion questions prompts participants to brainstorm unique pain points those users may experience as a result of having been marginalized during the design process. (The Inclusive Design Canvas is proprietary, so I can’t share images of it here.)
The outcome of an Inclusive Design Canvas workshop is twofold. First, design teams leave with a list of specific user pain points that need to be solved for, and second, they become more attuned to thinking about unintended users of products and services more generally.
Fostering empathy for a new target market
Each member of the Inclusive Design Group also helped lead initiatives within their respective functional teams as well. One UX research project involved creating a new enterprise persona to represent customers living on tight budgets. The process began with a collaboration among researchers from multiple departments who compiled data from various sources. For example, themes from qualitative interviews my team had conducted were validated with quantitative survey results from our partners in consumer research and innovation.
Based on these synthesized findings, we crafted a new persona that departed from our existing framework in innovative ways. First, we shifted from focusing on demographic to psychographic attributes. Whereas previous personas had been defined by their respective income brackets, this new persona was defined by the unique psychological traits resulting from a small delta between their income and expenses. This approach helped us clarify that customers with widely varying incomes nonetheless experience similar struggles when their living costs leave them with little wiggle room.
Second, we formatted the persona with a gender-neutral name and collage of diverse photos. Traditional personas typically include a name and photo representing a single social identity, but we wanted to convey how our new persona represents a population that is inclusive of various racial-ethnic, gender, and age groups. (The new persona is proprietary, so I can’t share images of it here.)
Putting new inclusive design tools to use
Incorporating the persona into various initiatives at the bank helped teams prioritize the unique needs of customers who are economically disadvantaged. For example, I helped write a set of screener questions aligned to the persona so that teams designing new budget tools could get feedback from appropriate usability testers.
The persona also played an important role in designing a new checking account feature called CoverDraft, which provides customers with up to $100 of fee-free overdraft coverage for qualified transactions. I led a series of journey mapping workshops with the persona to document paint points and moments of delight that our current customers might experience once CoverDraft went live.
I borrowed elements from the Inclusive Design Canvas to set up two journey maps. The first was for intended users, customers whose needs and preferences are aligned to the new feature. Workshop participants were ecstatic to document each touchpoint on this “happy path” and identified multiple moments of delight customers might encounter.
The second was for unintended users, customers who simply do not want or need CoverDraft despite the fact that it would be switched on by default. As we began to brainstorm customer pain points and frustrations, we realized that several serious “unhappy paths” had never been considered.
For example, it would be possible for a customer to ignore marketing emails about the new feature, never become aware about changes to their account, and then unintentionally complete debit transactions with money they don’t have and can’t afford to pay back. As one customer later commented on the webpage announcing CoverDraft, “How do I Opt Out of this service? If my account has items presented with insufficient funds, I do not want them paid.”
A major takeaway was the realization that some customers may not welcome the new feature, especially those who prefer overdraft transactions to be declined so they can avoid owing the bank money. Even though CoverDraft is meant to help people on the margins, we had to admit that it could actually create new problems for them instead. Product leaders were receptive to the feedback and agreed to prioritize functionality that will allow customers to easily turn the feature off. In the end, this project illustrates how inclusive design tools can help teams move beyond their own good intentions and critically evaluate the impact of their work on end users.
Designing Interactive Learning Experiences that Improve Student Outcomes
Focusing on the unique needs of extreme users drives innovation in curriculum development.
Understanding the unmet needs of students on the margins
When I taught Intro to Sociology at Tennessee Tech for the first time in Fall 2017, I quickly realized my biggest challenge was to keep students engaged throughout the entire fifteen-week course. Approximately 25% of students displayed a troubling pattern of behavior. They frequently skipped class, failed to submit low-value assignments, and rarely connected with me one-on-one, such as during office hours or through email correspondence.
When I took a closer look at their grades, I realized there were actually two different types of students involved: low-achieving students, who earned D’s and F’s, and high-achieving students, who earned A’s.
To better understand their experiences, I conducted one-on-one interviews with five low- and five high-achieving students. I used thematic analysis and affinity diagramming to synthesize this qualitative data alongside quantitative measures of academic performance. Finally, I created personas and journey maps to guide my student-centered redesign of the course.
This research revealed that low-achieving students tended to be freshmen with modest high school GPA’s and ACT scores, indicators that college-level work might be daunting for them. Many were first-generation college students who didn’t know what to expect from the course. They explained that when course content proved challenging early in the semester, they felt isolated and too embarrassed to ask for help. Continuing to attend class caused anxiety, so they unplugged from the course and fell further and further behind.
High-achieving students tended to be upperclassmen who were already succeeding in their major fields of study. Many benefited from the support of college-educated parents and had conquered college-prep courses in high school with relative ease. They viewed the course as neither challenging nor important and were confident in their ability to earn A’s without investing much effort. Skipping class and failing to submit low-value assignments were calculated choices, and they were able to score well on exams by quickly skimming textbook chapters on their own.
I used these insights to frame my design problem as involving two types of extreme users, each with their own unique needs. Low-achieving students needed more social support to voice their concerns and get help when they needed it. High-achieving students needed more meaningful connections to the course other than simply earning “easy A’s.”
Testing a possible solution for increasing student engagement
My goal was to structure class time around activities that would increase engagement, so I organized students into diverse groups of five and required them to work together on completing formative assessments and preparing for summative assessments.
Qualitative evidence collected through classroom observation showed that low-achieving students were more comfortable reaching out for help and high-achieving students were more invested in collaborating with their peers. Quantitative measures of class attendance and missing assignments both improved as well, and perhaps most importantly, low-achieving students were much more likely to earn a C or better in the course.
This new emphasis on group work also increased participation in class discussion more broadly, but unfortunately students were too often discussing unsubstantiated claims related to their own personal opinions. They were distracted by political ideology and began to lose sight of sociology as a science. Over time, I realized that my solution to the problem of keeping certain students engaged had created the new problem of keeping the entire class focused.
Iterating on the solution with a lab science approach
So, my next goal was to place more emphasis on how social scientific claims are supported by the careful collection and systematic analysis of evidence. Students needed to experience the research process themselves in order to better understand how sociologists arrive at their conclusions. They needed to learn about sociology by doing sociology.
In Spring 2019 I determined I would try teaching the course like a lab science. Just as natural science courses like chemistry and biology provide students with laboratory settings to develop familiarity with the scientific method, I would create lab assignments that give students the chance to work with real data. My idea was awarded a competitive curriculum grant through Tennessee Tech’s Quality Enhancement Plan, which involved an intensive two-day design sprint with other grant recipients.
On the first day, faculty each established criteria and constraints for our respective courses and then cycled through rounds of divergent and convergent thinking to brainstorm and subsequently edit our ideas. The following day, we used backward design to sketch out course objectives, assessments, and learning experiences for an iterative series of prototypes that were shared during group critiques. My final design included a series of ten lab assignments for students to complete with their group members during class.
Evaluating the positive impacts of innovative curriculum
These lab assignments were implemented in Fall 2019 and effectively oriented class time around the collection and analysis of empirical evidence, which ultimately kept students focused on the methods and theories that distinguish sociology as a social scientific discipline. Students rated me 4.9 out of 5.0 as an “excellent teacher,” and the course 4.6 out of 5.0 as an “excellent course.”
Later, a comparison of pre- and post-test data from a departmental general education assessment showed that students in my sections of the course were making the most significant gains. Based on these outcomes, my department chair nominated me for the university’s General Education Award for Outstanding Teaching.
After sharing these results with a colleague who specializes in the scholarship of teaching and learning, we decided to collaborate on a journal article illustrating how design thinking and user experience research can be used to improve course instruction. Our co-authored reflective essay was published in Transformative Dialogues: Teaching and Learning Journal.