Success metrics that matter
The most common mistake in reading ShopGuide's dashboard isn't picking the wrong metric — it's conflating metrics that measure different things. This guide explains what each number actually counts, using ShopGuide's real metric definitions rather than generic industry benchmarks (which we won't invent here — your own baseline, tracked over time, is more useful than a claimed industry average).
The core metrics, and what each one actually counts
- Unique users — distinct shoppers, tracked by persistent visitor identity. One shopper across multiple visits still counts once.
- Sessions — distinct browsing visits. One shopper can generate many sessions over time.
- Interactions — sessions where the shopper actually sent a message, as opposed to a session where the widget loaded but nobody typed anything.
- User messages — total message volume, a measure of conversation depth rather than reach.
- Chat orders / chat-attributed revenue — completed Shopify orders linked back to a chat thread when a purchase follows a chat interaction.
These are deliberately separate because collapsing them produces misleading comparisons — a month with more sessions but flat interactions means more people saw the widget without engaging, not more engagement. See Common mistakes for the specific version of this mistake to avoid.
Two ways to count sessions, and why they can disagree
ShopGuide's dashboard lets you toggle the session denominator between Shopify's own analytics session count and ShopGuide's own thread-derived count. They're not always available or equal: Shopify's count depends on your store having granted the reporting access ShopGuide needs, and where it's unavailable the dashboard shows the relevant ratios as unavailable rather than substituting a different number silently. Where both are available, small differences are expected — they're counting sessions via different mechanisms — and neither is "wrong." Pick one as your primary reference for period-over-period comparisons rather than switching sources between reporting periods.
Chat-attributed revenue is a direct linkage, not a model
Chat orders and chat-attributed revenue aren't estimated via a statistical attribution model — they come from directly linking a completed Shopify order back to the shopper's chat thread when a purchase follows a chat session. That makes the number defensible (it's a real, traceable order) but also conservative: a shopper who chats, leaves, and returns later to buy without a traceable link back to that thread won't be counted, so treat chat-attributed revenue as a floor on chat's actual influence, not a ceiling.
Information Gaps as a quality metric
Alongside volume and revenue metrics, the count and category mix of entries in Information Gaps Detection is a genuine leading indicator of chat quality — separate from CSAT-style scores, which depend on shoppers bothering to rate a conversation. A rising system_error count for one shop specifically is an app-health signal worth investigating immediately; a rising knowledge_gap count clustered on one topic tells you exactly what content to add next, since recurring phrasings of the same question are already stacked into one row by topic rather than fragmented into duplicates.
A/B test results as a conversion metric
Where success metrics get closest to a controlled comparison is ShopGuide's A/B testing framework, which attributes both chat engagement and completed Shopify orders to each variant of a launch message or prompt. This is the one place you can compare two configurations on real revenue outcomes rather than inferring cause from an overall trend line. See A/B testing and A/B test results — and give a test a full business cycle before treating an early lead as a real result.
Next steps
Pick metric definitions that match what you're actually trying to learn, and hold them constant across reporting periods — the comparison matters more than the absolute number.