When Segment’s VP of Growth, Guillaume Cabane, first began automating his company’s outbound email campaigns he expected two things to happen. First, because humans would no longer be a bottleneck, he expected the volume of activity to skyrocket. Second, because humans wouldn’t be actively personalizing messages, he expected the conversion rates to go down.
For Cabane, who calls himself Segment’s mad scientist, this was a simple optimization to measure. As long as the increase in volume was significant enough to outweigh the decreased conversion rate, he’d consider the automation a success. Four months later when he ran his first report he was astounded; automation had risen into the top five sources of qualified opportunities at Segment. But he saw something else in the data that surprised him.
Rather than transition entirely to the new automated system, Cabane had decided to run an A/B test. He asked his co-worker Mark Miller, one of the company’s SDRs, to continue emailing prospects as he had for the last six months so that he could compare the results. When he looked at the conversion rates, he realized that the automated email system was actually outperforming Miller in volume and efficiency.
Not only was he able to email four times as many people as the manual process could in a week, but he was achieving a 55% open rate and 13% response rate from a cold email.
At SimpleData many of our customers asked whether or not automation was worth the investment. We'd describe the tradeoff most companies make: volume usually goes up, but conversion rates go down; generally the output is higher in aggregate. The classic quantity vs. quality debate.