A/B Testing – improving engagement

The Problem

After being onboarded in Redkix, users fail to become engaged without understanding the value. Some users leave, others continue using the product as a typical email application and never see the real value Redkix provides – a collaboration platform for their team.

Adding this "Create Channel" modal in our onboarding improved our engagement and helped our users understand what Redkix was all about.

Adding this "Create Channel" modal in our onboarding improved our engagement and helped our users understand what Redkix was all about.

Hypothesis

We noticed a trend in our data that of the users who were creating channels, they had better engagement numbers. If we help the user focus on creating their first Redkix channel with people they know, when they land in the product, their engagement will improve and they’ll gain a better understanding of the values Redkix provides outside of typical email.

A Onboarding

Focused on a general UI tour of the application, a bot engaged the user with a quick 1:1 conversation, demoed realtime and presence and then coachmarks guided the user around various features.

B Onboarding

Focused on showing the users why channels are so important, showed them several examples of what channels can be used for and guided them into creating their first channel with teammates.

Daily Messages Sent by Unique Users – Note: chart was generated by Mode Analytics and I've obfuscated the confidential Total axis data

Total Daily Messages Sent – Note: chart was generated by Mode Analytics and I've obfuscated the confidential Total Events axis data

Data and Outcome

We saw in our analytics platforms (Intercom and Mode), when we introduced the “B” version of Onboarding that was focused on channel creation and teams to our B segment, the engagement and number of messages sent per unique user improved. We also saw that our total daily messages sent gradually started going up too.