Users Were Acquiring — But Not Activating
Diagnosing activation and retention through cohort analysis, identifying the behavioral milestone that separates retained users from churned ones.
The Result at a Glance:
Users signing up but not reaching activation milestone
Onboarding did not effectively guide users to the core product action
Cohort analysis + activation definition + retention comparison
Identified activation as the leading indicator of retention and quantified the retention gap between activated and non-activated cohorts.
Analytics / Product
Define activation → funnel analysis → cohort construction → retention analysis → experimentation opportunity
Product Context
A product with a self-service signup flow was acquiring new users consistently. However, engagement metrics — return visits, feature usage, and subscription conversions — were weakening. The overall funnel appeared healthy at the top but deteriorated rapidly after signup.
Define Activation
Activation is not "completed onboarding." Activation should be defined as the moment a user experiences the product's core value. I defined activation as the completion of the first core action — the behavior that, based on data, most strongly correlates with continued product usage. This is an important distinction: onboarding completion is a process milestone; activation is a value milestone.
- Onboarding completion ≠ Activation
- Activation = first time the user performs the core product action
- The activation definition should be validated against retention data
Funnel
I constructed the funnel from signup through activation:
Cohort Analysis
I built weekly cohorts based on signup date and tracked activation and retention rates:
| Cohort | W1 Retention | W2 Retention | W3 Retention | W4 Retention |
|---|---|---|---|---|
| Week 1 (Simulated) | — | — | — | — |
| Week 2 (Simulated) | — | — | — | — |
| Week 3 (Simulated) | — | — | — | — |
| Week 4 (Simulated) | — | — | — | — |
Retention: Activated vs Non-Activated
The most important comparison was retention rates between users who activated and users who did not:
- Activated users had significantly higher Week 4 retention
- Non-activated users churned rapidly after Week 1
- The gap between the two groups widened each week
- This suggests activation is a leading indicator, not a coincidence
Segmentation
I segmented activation rates by:
- Acquisition channel — which channels produce users who activate?
- Device — does mobile vs. desktop affect activation?
- Geography — are there regional differences in activation?
- User type — do users from specific referral sources activate at different rates?
Hypothesis
Users completing the core action within the first session are more likely to return. If this hypothesis holds, then reducing the time and friction to first core action should improve retention.
Experimentation Opportunity
Based on the analysis, I designed a potential experiment: can the onboarding flow be restructured to guide users to the core action faster? The experiment would compare the current onboarding against a streamlined version that prioritizes the activation milestone.
- Control: current onboarding flow
- Treatment: streamlined flow that reaches core action sooner
- Primary metric: activation rate within first session
- Guardrail metrics: onboarding completion rate, error rate, support tickets
Recommendation
Focus product investment on reducing time-to-activation rather than increasing top-of-funnel acquisition. The data suggests that improving activation rate will have a larger impact on downstream metrics than acquiring more users who don't activate.
Limitations
This analysis has important limitations that must be acknowledged:
- Correlation ≠ causation — activated users may differ from non-activated users in ways not captured in the data
- Selection bias — users who activate may be inherently more motivated
- Tracking gaps — some user actions may not be captured, especially cross-device behavior
- Cohort size — small cohorts produce unreliable retention curves
- Definition sensitivity — changing the activation definition changes the entire analysis
Diagnostic Code & Query Architecture:
WITH activated AS (
SELECT
user_id,
MIN(event_date) AS activation_date
FROM `project.analytics.events`
WHERE event_name = 'core_action_completed'
GROUP BY user_id
)
SELECT
DATE_TRUNC(activation_date, WEEK) AS cohort_week,
COUNT(*) AS activated_users
FROM activated
GROUP BY cohort_week
ORDER BY cohort_week;