Retention and Cohort Analysis: How to Measure Whether Users Come Back

Acquisition and conversion numbers can look healthy while the product underneath is leaking. A campaign brings in signups, the funnel converts, and a month later most of those people are gone. Retention analysis asks the question the other reports cannot: after someone arrives, do they come back?

This guide covers how to read a retention table, how to use cohorts to compare groups of visitors, and where the numbers need care. If you have not yet defined the actions that matter in your product, start with event tracking best practices.

What a retention table shows

A retention table is a grid. Each row is a group of visitors who first arrived on the same day. Each column is a later day. Each cell is the percentage of that row's visitors who returned on that day.

Read it in two directions:

• Across a row to see how one group decays over time. A steep fall after Day 1 followed by a flat line means most people leave quickly, but those who stay tend to keep returning.
• Down a column to compare the same day across groups. If Day 7 improves for visitors who arrived after a release, that is a signal worth investigating.

The table answers "did they return," not "why." Treat it as a way to find the groups and dates that deserve a closer look.

Run the Retention report in Umami

Open a website and select Retention. Choose a month and year. Umami groups visitors by the day they first visited in that month and shows the percentage who returned on Day 1 through Day 7, then Day 14, Day 21, and Day 28. Each row also shows the size of that day's group. The Retention documentation shows the current controls.

A few details shape how you should read the result:

• Recent rows are incomplete. A group that arrived five days ago cannot have a Day 14 value yet. Empty cells mean "not yet measurable," not zero.
• Group size matters. A row with a few dozen visitors can swing widely from one day to the next. Check the visitor count before reading meaning into a percentage.
• The report covers one month at a time. To compare periods, run it for each month and note the same columns.

Use Filter to narrow the report to one audience, such as a country, a device type, or a traffic source, and compare the result with the unfiltered table.

Know what "returning" means

Umami does not use cookies. It recognizes a visitor through an anonymous session identifier derived from the website, the visitor's IP address and user agent, and a salt that rotates monthly by default. Retention is calculated from that identifier.

This has practical consequences:

• A person who visits on a phone and later on a laptop is counted as two visitors.
• A person whose network or browser changes can appear as a new visitor.
• Return visits are measured within the month, which is why the report is organized by month.

The measured rate is therefore a conservative estimate of true return behavior. It is still useful for comparison, because the same rules apply to every row and every month. Compare cohorts with each other rather than quoting the absolute figure as "our retention."

If your product has logged-in users, you can set a distinct ID with umami.identify(). Umami links the sessions that share an ID, which lets you search for a user and view their combined activity in Sessions. Use an internal identifier rather than an email address, and make sure sending it is consistent with your privacy policy.

Compare groups with cohorts

Day-of-arrival rows are one way to group visitors. Often the more useful question is behavioral: do people who did X come back more than people who did not?

A cohort in Umami is a group of visitors who viewed a specific page or triggered a specific event within a date range. To create one, open Cohorts, select Add cohort, choose the date range and the page or event, and save it. To keep membership fixed, choose a custom date range; a relative range such as "last 30 days" moves as time passes.

Once saved, a cohort can be applied as a filter. Add a filter, switch to the Cohorts tab, select the cohort, and apply it. The report then describes only that group.

Useful cohorts to try:

1. Visitors who triggered your activation event, such as project-created, in their first week.
2. Visitors who viewed onboarding or documentation pages.
3. Visitors who arrived during a specific campaign.

Segments complement cohorts. A segment saves a set of filters, such as "mobile visitors from organic search," so you can reapply it consistently. Use segments for who visitors are and cohorts for what they did.

Worked example: finding an activation signal

A team running a project management tool sees that Day 7 retention has been flat for three months. They want to know which early action is associated with coming back.

They create two cohorts for the same custom date range: visitors who triggered project-created, and visitors who triggered invite-sent. They apply each to the Retention report and write down the Day 7 and Day 14 columns alongside the unfiltered values.

The invite-sent cohort returns at a noticeably higher rate than the others. That does not prove invitations cause retention. People who invite colleagues may simply be more committed to begin with. It does give the team a testable hypothesis: prompt new users to invite a teammate earlier in onboarding, then compare the retention of users who arrive after the change with those who arrived before, ideally with an A/B test.

They also build a funnel from signup to invite-sent to see how many new users reach that step at all, and use the Journey report to see what people do instead.

Turn retention findings into changes

A retention number becomes useful when it changes what you build. A workable loop:

1. Pick one column. Day 1 reflects first impressions, Day 7 reflects habit, Day 28 reflects lasting value. Choose the one that matches your product's natural rhythm.
2. Find a group that does better. Compare cohorts by early action, source, or device.
3. Form a hypothesis about why, and check it against journeys and session activity.
4. Change one thing that helps more people reach the behavior.
5. Compare later arrivals with earlier ones over a similar period.

Keep a record of what you changed and when. Annotations on the website timeline make it easier to connect a shift in the table to the release that preceded it.

Common mistakes

1. Reading correlation as cause. A cohort that retains better may differ in ways you did not measure. Test before rebuilding onboarding around one event.
2. Comparing incomplete rows. Recent groups have not had time to return. Compare only columns that every row has had the chance to fill.
3. Ignoring group size. Small rows produce dramatic percentages. Combine days or widen the comparison before acting.
4. Quoting the absolute rate as fact. Cookieless measurement undercounts returns across devices. Use the figure for comparison over time.
5. Mixing audiences. A campaign that brings in low-intent traffic will lower retention for those dates without anything changing in the product. Filter by source before drawing conclusions.
6. Measuring the wrong rhythm. A tool people use monthly will look poor on a Day 1 column. Match the column to how often a satisfied user would naturally return.