CLVTools: A powerful R package to evaluate your customers
The lifetime value of a customer can be an invaluable strategic tool for businesses
In this article, written together with Patrick Bachmann, we take a stroll through the extensive R package “CLVTools”, which includes convenient implementations of some of the most well-known models to estimate the customer lifetime values (CLV or LTV, which is introduced for example here). The article is based on the quite detailed walkthrough of the package found here. There is also an (older) youtube video available with further explanations.
We will mainly focus on the ParetoNBD model and its extensions, which in the literature on LTV is seen as a kind of gold standard. This model assumes transactions occur as a so-called “Poisson process”, and that customers are (i) heterogeneous, (ii) can purchase at any time, and (iii) can end their relationship with the company at any time. Case (iii) is assumed to be permanent, once you lost a customer, they are not coming back. Consequently, alongside the Transaction or Poisson process, a separate process is modeled representing the “lifetime” of a customer. This is a typical theme in these kinds of models and so one usual divides between Transaction Process and Lifetime or Attrition Process.
Since the model is a tad complicated, its cousin the BG-NBD model is often considered, for example in this post. The difference is that the BG-NBD model is more discrete in nature — in particular, customers can only become inactive directly after a transaction (whereas in the ParetoNBD model, they can become inactive at any time). This may be desirable in some scenarios, but less so in (many) others.
But enough about the models themselves, how do we use them with CLVTools? There are generally 3 steps to follow for a successful analysis with the package:
- Create the “clv.data” object containing the dataset and required meta-information (format of the date you are using, column names etc). After this initialization, more info can be added about potential covariates.
- Fit the model.
- Use the estimated model parameters for your analysis, i.e. plotting, prediction of customer behavior, and so on.
This is also illustrated in the Figure above. We now go through these steps with an example data set.
Step 1: Initialize
library(CLVTools)
data("apparelTrans")
data("apparelDynCov")We use the “apparelTrans” data from the package, which is printed below. This screenshot is important, as it shows the data format the package requires: There should be a unique Id number for each customer, dates of transactions (in some given format such as YYYY-MM-DD), and the price of the transaction.
Additionally, we assume to have covariates (i.e. attributes) of customers stored in “apparelDyncov”. In particular, the company knows the gender of the customer and their acquisition channel (“offline”=0 or “online”=1). These are static covariates, which means they may vary over individuals, but they are constant over time. However, there is also “Marketing”, which indicates how many direct marketing attempts were made at a single customer. This is a time-varying covariate, which means it cannot only change from customer to customer but also vary over time. Only one (very recent) model in the package is currently able to handle this time-varying structure, the extended ParetoNBD model developed here, and we are going to exploit this fully.
We now take the necessary first step to introduce this data to the package with the “clvdata” command, which produces an object that can easily be handled by all functions of the second and third steps. In this command, we need to specify the data set, which date format we use (as in the “lubridate” R package), what the time unit is and when (or if) the data should be split into estimation and validation periods. Finally, we need to specify the column names of the Ids, dates, and prices.
clv.apparel <- clvdata(apparelTrans,
date.format = "ymd",
time.unit= "week",
estimation.split=40,
name.id="Id",
name.date="Date",
name.price="Price")The summary output can be seen here:
Step 2: Estimate
We can now estimate our models on the above object. With the default choices and without covariates, this is as simple as it gets:
## Estimate the PNBD model
est.pnbd<- pnbd(clv.data=clv.apparel)
## Estimate the BGNBD model
est.bgnbd <- bgnbd(clv.data=clv.apparel)On the other hand, with (time-varying) covariates we need to specify a bit more: In particular, we need to specify the covariate data for the Lifetime or Attrition process (data.cov.life) and for the Transaction process (data.cov.trans). Moreover, we need to give the names for both processes in names.cov.life and names.cov.trans.
clv.dyn <- SetDynamicCovariates(clv.data=clv.apparel,
data.cov.life = apparelDynCov,
data.cov.trans = apparelDynCov,
names.cov.life = c("Marketing", "Gender", "Channel"),
names.cov.trans = c("Marketing", Gender", "Channel"),
name.id = "Id",
name.date = "Cov.Date")
# Estimate the PNBD with Covariates (This takes a while (!))
est.pnbd.dyn <- pnbd(clv.dyn)We can then first check the summary output of these models. This output is similar as in plain linear regression: Standard errors, as well as significance values of the parameters, are given and in addition, one has information on the convergence of the optimization, such as the final likelihood value. Notice how ‘’est.pnbd.dyn’’ has 6 parameters more than est.pnbd. This corresponds to one covariate value for each of the 3 covariates (“Gender”, “Channel”, “Marketing”) for both the Transaction and Attrition processes. One can also play with this and restrain covariates to influence only one of the two processes, or to influence both processes with the same parameter.
Interestingly, the model hints at a diverse effect of the direct marketing efforts (assuming a causal interpretation is possible): It appears to increase the transaction frequency (trans.Marketing is positive), but it has an adverse effect on the average lifetime of a customer (life.Marketing is negative). This could mean that direct marketing may boost sales in the short term but increases the chance that a customer leaves a firm earlier. As usual, one has to be careful with such causal statements, but the model at least delivers a hint towards the effects (the original paper linked above explains at length how the causal effects may be recovered using Instrumental Variables (IVs), etc.).
But enough with the dry outputs, let's do some plotting. An interesting plot to check the in-sample fit, usually done in this context, is the “tracking plot”.
The BG-NBD and ParetoNBD plots look quite alike: They nicely cut through the erratic transactions and appear to capture the mean pretty well. This gets more interesting with time-varying covariates, where we are suddenly able to match some of the patterns that at first may appear random. This is the advantage well-designed time-varying covariates can bring. In fact, together with individual, but time-invariant covariates, individual predictions can sometimes be sharply improved, beyond what is visible in this aggregated graph. Next to the wealth of classical models with blazing fast and elegant implementations, this is really what makes CLVTools stand out.
But ultimately we are interested in prediction not nice in-sample fits. So let's do that:
Step 3: Predict
This is again as easy as it gets, code-wise (if we go with the defaults):
predict(est.pnbd)
predict(est.bgnbd)
predict(est.pnbd.dyn)Unfortunately, there is nothing flashy to show here, just the convenient table that gives all the predictions of the different models (we omit est.bgnbd here):
There is a peculiar thing to mention here: While the ParetoNBD and BG-NBD can predict things like the expected number of transactions (“CET”) and the probability of a customer to be still with the company (“PAlive”), they cannot model the spending process on their own. For this, the output of the models is usually combined with the Gamma-Gamma (GG) model for spending. This models the spending process and is automatically fitted when we ask for a prediction, which is what can be seen in red. This not only leads to the predicted mean spending but also to the predicted customer lifetime value we desire. This is given in the very last column of the table for each ID.
These predictions can then be used for more informed business decisions. For instance, it might be sensible to focus on customers with Id 100 and 1000, since their LTV is estimated quite high. However, given the parameter estimates it is not clear in this example whether direct marketing is the way to go: There exists at least a hint that direct marketing can diminish the average time a customer stays with the firm.
Conclusion
This article presented a short introduction to the CLVTools package for customer value modeling. The package has a wide range of functionalities and advantages that we couldn’t possibly cover in one article, but we hope it helps interested readers to get started with the package and do useful stuff with it.
