We checked for any errors that might have sneaked in the data, but there were none.
So we took a closer look at how we analysed them and noticed that people usually interacted with the live chat towards the end of the onboarding process, when the onboarding rate was bigger anyways. It was over 50% as opposed to the first steps when the onboarding rate was around 12%.
This is where the error occurred: when calculating the overall conversion rate, it was split evenly between all the onboarding steps. In reality, the vast majority of people interacted with the onboarding at the end of the process, when the conversion rate was naturally bigger.
In the end we came to the following conclusion: the fact that people interacted with the live chat did not change their onboarding behaviour. The conversion rate was just the same, with or without the live chat.
Not the kind of news you wanted to hear if you were the CEO of that company… There were plenty of resources invested in the support team, tens of employees only for the live chat.