论文标题
个性化免费试验的设计和评估
Design and Evaluation of Personalized Free Trials
论文作者
论文摘要
免费试用促销活动是免费尝试免费尝试该产品的,这是该软件中常用的客户获取策略作为服务(SaaS)行业。我们研究了试验长度如何影响用户的响应能力,并试图通过个性化免费试验促销的长度来量化收益。我们的数据来自领先的SaaS公司进行的大规模现场实验,在该实验中,新用户被随机分配到7、14或30天的免费试用期。首先,我们表明对所有消费者进行的为期7天的试验是最佳统一政策,订阅增加了5.59%。接下来,我们为个性化的政策设计和评估开发了一个三管齐下的框架。使用我们的框架,我们根据线性回归,套索,推车,随机森林,Xgboost,因果树和因果林制定了七个个性化的目标策略,并使用反向倾向分数(IPS)估计量来评估其性能。我们发现基于Lasso的个性化政策表现最好,其次是基于XGBoost的政策。相反,基于因果树和因果林的政策表现不佳。然后,我们将方法在设计政策方面的有效性与其在不合适的情况下对治疗进行个性化个性化的能力(即捕获虚假的异质性)。接下来,我们根据消费者的最佳试验长度进行细分,并在这种情况下对用户行为的驱动因素得出了一些实质性见解。最后,我们表明,旨在最大化短期转化的政策在消费者忠诚度和盈利能力等长期成果上也表现良好。
Free trial promotions, where users are given a limited time to try the product for free, are a commonly used customer acquisition strategy in the Software as a Service (SaaS) industry. We examine how trial length affect users' responsiveness, and seek to quantify the gains from personalizing the length of the free trial promotions. Our data come from a large-scale field experiment conducted by a leading SaaS firm, where new users were randomly assigned to 7, 14, or 30 days of free trial. First, we show that the 7-day trial to all consumers is the best uniform policy, with a 5.59% increase in subscriptions. Next, we develop a three-pronged framework for personalized policy design and evaluation. Using our framework, we develop seven personalized targeting policies based on linear regression, lasso, CART, random forest, XGBoost, causal tree, and causal forest, and evaluate their performances using the Inverse Propensity Score (IPS) estimator. We find that the personalized policy based on lasso performs the best, followed by the one based on XGBoost. In contrast, policies based on causal tree and causal forest perform poorly. We then link a method's effectiveness in designing policy with its ability to personalize the treatment sufficiently without over-fitting (i.e., capture spurious heterogeneity). Next, we segment consumers based on their optimal trial length and derive some substantive insights on the drivers of user behavior in this context. Finally, we show that policies designed to maximize short-run conversions also perform well on long-run outcomes such as consumer loyalty and profitability.