← Portfolio Download PDF
Case Study · UXR-224 · Comprehension Study · Lifeway Christian Resources

Checkout Savings Comprehension Study

How we discovered that perceived ease and actual understanding are not the same thing.

92% Core accuracy — Design 2
85% Overall comprehension
4 days Launch to exec shareout
3 Designs tested in parallel
01 — The Challenge

Would customers understand what the progress bar was telling them?

Lifeway was introducing a Savings Progress Bar to the checkout experience — a UI showing customers their current discount tier and free shipping eligibility as they built their cart.

Three design variations were created. Each communicated the same pricing system but used different visual hierarchies, labels, and progress indicators.

The question: would customers actually understand what the bar was telling them?

If users misread the pricing system, they'd face unexpected totals at checkout — eroding trust at the highest-friction point in the purchase journey. The stakes were real.

02 — Research Objectives

Three questions the study needed to answer.

Objective 01

Comprehension of the progress bar

Do users understand what the two tracking bars represent — savings thresholds versus free shipping eligibility?

Objective 02

Understanding of savings applied

Can users correctly identify what discounts are active and what they need to spend to unlock the next tier?

Objective 03

Comparative design clarity

Across three design concepts, which communicates the pricing system most accurately?

03 — Methodology

Unmoderated. Comparative. Mixed methods.

Study Type

Unmoderated Comparative

Participants reviewed one design and answered a structured set of questions without a moderator present.

Methods Mix

Quant + Qual Triangulated

Multiple choice comprehension tasks, 5-point rating scales, and open-ended questions — analyzed together for a complete picture.

Scope

3 Designs in Parallel

Each participant saw one design only, eliminating order bias. Results compared across groups post-study.

Feb 6 Request received
Feb 12 Study launched
Feb 13 Analysis complete
Feb 16 Executive shareout
4 days total
04 — The Results

Design 2 wins on comprehension — not perception.

Comprehension Accuracy (%) — All Three Designs

Core Accuracy
Design 1
73%
Design 2
92% ★
Design 3
76%
Qualitative Understanding
Design 1
62%
Design 2
73% ★
Design 3
57%
Overall Understanding
Design 1
69%
Design 2
85% ★
Design 3
69%
Design 2 — Winner
Designs 1 & 3

The catch: Design 3 scored highest on user ratings — 4.50 on ease of use and savings clarity. Design 2 rated lower on both. Users felt Design 3 was clearest. The comprehension data told a different story.

05 — The Key Insight

Perceived ease
≠ actual understanding.

Design 3 felt easiest. Design 2 was understood best — 92% core accuracy, 85% overall comprehension, outperforming on every comprehension measure. A design that feels simpler can still ship users into confusion, especially when the stakes involve pricing.

06 — Recommendation

Ship Design 2. Incorporate Design 3 refinements.

Primary Recommendation

Ship Design 2

Highest core accuracy (92%) and overall comprehension (85%). Grounded in evidence across all three comprehension metrics — not perception.

Incorporate Design 3 refinements

Design 3 excelled at perceived savings clarity. Selective copy and visual elements from Design 3 can be layered into Design 2 without compromising comprehension.

Consider a live A/B test

If stakeholders remain divided after refinements, a live A/B between Design 2 and the refined version provides real behavioral data at scale.

07 — Reflection

What worked. What I'd change.

What worked well

  • Splitting quant and qual analysis so each could tell its own story before being combined
  • Screener designed to match actual Lifeway customer segments — not a general population
  • TL;DR summary written for exec audiences, not just researchers

What I'd do differently

  • Add a think-aloud component to capture in-the-moment confusion, not just post-task reflection
  • Test a fourth hybrid design combining Design 2 comprehension with Design 3 perceived clarity before recommending an A/B
More Case Studies