Application of deep UPLIFT model in Tencent Financial user growth scenario

Today’s sharing mainly focuses on the deep UPLIFT model, including the main technical challenges and development context of this type of model, and also introduces the classic case of the deep UPLIFT model helping Tencent FinTech user growth.

1. Introduction to the UPLIFT model

First introduce the UPLIFT model.

A classic scenario: How to find the optimal customer group for marketing activities? Or how to find the optimal marketing incentive method?

​The common idea is to target users with high willingness (Sure Things and Persuadable quadrants in the above picture), but the biggest problem with this idea is: it ignores the natural flow of users. User conversion. First of all, some of the high-intent users (Sure Things quadrant users) can convert naturally without advertising; in addition, under the discount coupon marketing model, direct advertising to high-intention users often leads to high-cost losses. Is it possible to switch to users with low willingness (Do Note Disturbs and Lost Causes quadrants)? Increase their willingness to consume through advertising? In fact, for users with low willingness, their probability of conversion is very low. They may not convert no matter how much you invest, or they may even be lost permanently, but this will consume a lot of marketing costs.

Therefore, before solving this type of marketing problem, we need to return to the concept of marketing - "marketing gain". "Marketing gain" means that our marketing activities should focus on that part of users who can change their stance. In layman's terms, the target customer group we are looking for is the group of people who will not convert before marketing, but will convert after marketing (Persuadable quadrant group), and the UPLIFT model is to find these groups of people who are sensitive to marketing incentives. ​

1. UPLIFT model VS response (Response) model

(1) Response model

When judging whether the user will convert after marketing to the user, a model is often constructed Response model. The model will predict the conversion rate of each user. Then we will define a conversion rate threshold based on a series of data analysis, cost estimation, etc., and deliver to users who are predicted to be higher than this conversion rate threshold. . The response model is simple and efficient, and can be constructed directly using all current mainstream machine learning/deep learning models. However, its biggest problem is that it does not take into account the part of natural conversion traffic that can be converted without marketing, so the modeling goals and business goals are not the same. Not an exact match.

(2) UPLIFT model

The UPLIFT model uses one or more models to predict the user's conversion rate under marketing and non-marketing respectively. Conversion rate, the difference between these two values ​​is UPLIFT. In the end, we decide whether to launch based on the size of the UPLIFT value.

Comparing the response model and the UPLIFT model curve in the above figure, we can see that the UPLIFT model will lead to an earlier intervention time. In the user churn early warning scenario, operations need to retain users who are about to lose in advance. In practical applications, it is meaningless to predict the specific time of user loss. Applying retention measures to these users may have missed the best intervention time, and predicting users who can be retained by operational measures is of practical value.

2. UPLIFT model and causal inference

The UPLIFT model itself is in the field of causal inference. The above figure briefly summarizes the causal inference based on observational research. First, it includes classic causal inference methods. This type of method mainly studies the average intervention effect and forms an overall directional evaluation of the strategy effect. For example, some scholars studied "whether smoking is harmful to health" and concluded that long-term smoking will reduce life expectancy. The conclusions are often directional and cannot form specific strategies. This is a problem we face in practical work-that is, how to implement the causal inference method. On the other hand, the UPLIFT model is a model that studies heterogeneous causal effects. It is found that the effects of intervention on different groups or different individuals are different, thereby discovering the rules of the intervention mechanism and forming personalized strategies. Next we introduce a few definitions:

(1) Neyman-Rubin potential outcome framework: Individuals (samples) present potential outcomes based on their own inherent attributes. This potential outcome often only has one side in the real world, which will lead to Here comes a big problem: Inductive Bias.

(2) CATE: The modeling goal of the UPLIFT model.

(3) Unconfoundedness (Unconfoundedness) : Unbiased CATE can only be obtained by controlling confounding factors (and no undiscovered confounding factors) for estimation.

3. Industry application of the UPLIFT model

The UPLIFT model has a wide range of industry applications, and the application methods are roughly divided into two categories: one is to evaluate intervention Heterogeneous effects HTE. Taking A/B testing as an example, the overall observed intervention effect may not be significant, but drilling down through the UPLIFT model can find subgroups with significant intervention effects. The first category is strategy optimization, including strategy customization in the field of public services and personalized Internet marketing.

2. Technical challenges

1. UPLIFT model modeling process

UPLIFT modeling starts with data collection. Generally, we first create a randomized controlled experiment, and all experimental groups Intervention is carried out, and there is no intervention in the control group. Then samples are collected and modeled using the UPLIFT model.

Taking T-Leaner as an example, intervention samples can build a response model, and non-intervention samples can build a response model. The final score difference is the UPLIFT value.

2. Two core issues in UPLIFT modeling

There are two core issues in UPLIFT modeling:

(1) Confusion Factor bias Confounding Bias

Due to the selection bias caused by the intervention mechanism, the characteristic distribution of the intervention sample and the non-intervention sample is inconsistent, resulting in confounding factors. This type of confounding factors affects both the intervention and the outcome. Due to the presence of confounding factors, we cannot obtain a clean causal effect. Specific cases such as:

① Popularity bias: exposure is concentrated on popular interventions.

② Selection bias: Differences in item exposure among different groups of people.

Modeling based on such samples will probably not lead to a high-confidence conclusion.

Main solutions:

​① Introduce the tendency score regularization term in loss.

② Introduce tendency chain and confrontation structures into the model structure.

③ Propensity score inverse weighted sampling.

④ Dissociation representation: Try to dissociate the confounding factor terms into a vector. ​

(2) Inductive Bias

As shown in the figure below: When the sample/individual is scored under the intervention/non-intervention model respectively When the final distribution is inconsistent, when we perform distribution statistics on the intervention score-non-intervention score difference (CATE), we will find that the jitter is very severe, and it is difficult to distinguish the UPLIFT differences between different individuals/groups, which shows that the UPLIFT model prediction is basically invalid. of.

This case illustrates the problem of inductive bias: from the perspective of model architecture, estimating counterfactual results is unsupervised The signal, or the degree of regularization of potential outcome estimates is inconsistent, causing the final UPLIFT estimate to be unstable.

Returning to the Neyman-Rubin potential outcome framework, for different individuals we only know the results of intervention or the results of no intervention. It is impossible to know the results of intervention and non-intervention at the same time, which leads to two potential outcome predictions. The distribution of estimates is inconsistent; and our deep model modeling goals are often CTR or CVR, which themselves cannot directly obtain a gain score (UPLIFT), which results in our modeling goals and final evaluation indicators being inconsistent. In the figure we can see that the distribution of CATE is completely inconsistent with the distribution of potential outcome estimate scores. We summarize this series of problems as the inductive bias problem. The problem of inductive paranoia is a core problem of the UPLIFT model. At present, the academic community has proposed the following four solutions:

① Counterfactual output vector consistency, MMD and other distribution alignment methods;

② Design a reasonable shared parameter architecture, FlexTENet, S-Net;

③ Counterfactual parameter difference restrictions;

④ Introducing reparametrization into the model structure, EUEN.

3. UPLIFT model application challenges

After introducing the core issues of UPLIFT, let’s briefly talk about the application challenges of the UPLIFT model. There are two main categories in the financial field: One is multi-valued/continuous value intervention, and the other is continuous outcome prediction.

3. Development of the deep UPLIFT model

Along the two lines of the UPLIFT model just proposed Academia and industry have proposed many solutions to the big core problems - confounding factor bias and inductive bias.

The first and most basic one is Meta-Learner. The representative modeling solutions are S-Learner and T-Learner. After entering deep learning, it evolved into DragonNet, DESCN, and S represented by solving mixed bias. -Net, CFRNet, etc., and FlexTENet, S-Net, EUEN, DESCN, GANITE, CFRNet, etc., represented by solving inductive bias. ​

4. Application Cases: Credit Card Repayment Voucher Marketing

Next, we will introduce the application cases of the UPLIFT model in the field of financial technology. Application of the UPLIFT model often requires a combination of operational measures and model iteration. In operational activities, we started with random delivery of small traffic to collect intervention/non-intervention samples, and then conducted feasibility verification and experimental design, strategic customer group delivery, UPLIFT model construction and delivery evaluation.

The UPLIFT model solution has completed three iterations:

(1) First-generation solution: Meta-Learner

First The current solution is mainly Meta-Learner, which can be launched quickly to achieve business goals. The pain point of the business is to control marketing ROI and increase volume.

(2) Second generation solution: from CFRnet to EFIN

The second generation is mainly based on deep learning architecture, and its main purpose is In order to realize multi-voucher personalized operation, it is a problem of multi-value intervention. The main models used are CRFnet and the self-developed EFIN (Explicit Feature Interaction uplift Network).

Here we focus on the self-developed EFIN model. This algorithm mainly has three modules:

The first module is the self-interaction layer, which uses a Self-Attetion network to fully learn large-scale user features.

The second module is the key to the entire algorithm. The intervening variable T is explicitly represented and learned separately, which is beneficial to solving multi-valued intervention problems. The intervention T and the feature And the reparametrization method is used as the estimated output, which alleviates the problem of inconsistent scoring distribution of potential results.

The third module uses intervention constraints to make it difficult for the model to distinguish whether the sample has been intervened, in an attempt to solve the problem of confounding bias.

(3) The third generation solution: multi-objective UPLIFT model

The multi-objective UPLIFT model is designed to model Ctr-Lift and Value-Lift at the same time, and is expected to achieve ROI>1 while achieving GMV growth. The biggest difficulty is the difficulty in modeling the amount of UPLIFT, because the variance of the two counterfactual estimates of the amount itself is very large, and the variance of the UPLIFT obtained by subtracting it is even greater. It is relatively difficult to obtain a stable UPLIFT. At present, Still working on solving it.

5. Summary

In this lecture, we introduced the two core issues of UPLIFT modeling as well as two application challenges and three generations of solutions. And focused on the self-developed EFIN model. In the future, we will conduct more in-depth research in a series of directions such as multi-objective UPLIFT modeling, ROI constraints, dynamic UPLIFT modeling, and observation data correction.

6. Question and Answer Session

Q1: Is it effective to use a network like DragonNet to simultaneously model Y and T for biased samples obtained from non-random experiments?

A1: This is definitely effective. We generally choose this propensity score method to model observation data. The application of the propensity score method may use a model structure such as DragonNet. In the model structure directly add the propensity score. Another more direct way is to perform sample weighted sampling directly based on the distribution of the propensity score.

Q2: What impact will the presence of inductive bias in potential results have on UPLIFT estimates?

A2: In fact, the picture I just made made it clearer, because our deep learning model estimates two counterfactual CTRs respectively, and their modeling goals are actually two CTRs. This If you find the difference between two CTRs, you will find that the variance of this difference is relatively large.

Q3: How are the inputs of neural networks generally processed? What is the difference between continuous features and discrete features?

A3: This problem is not a UPLIFT problem. Generally, Embedding is used, and One-Hot is used for categories.

Q4: In your second-generation solution, Meta-Learner is better than other NN models except EFIN. Is the base model in S-Learner also this model?

A4: Yes, S-Learner also has a DNN inside it.

Q5: What are the advantages and disadvantages of deep learning models compared to causal forest and UPLIFT-Tree models?

A5: The biggest advantage of deep models is the use of complex network structures, such as feature intersection, attention mechanisms, etc., to fully learn features; the biggest advantage of forest and tree models is stability, and the forest model has The modeling target is directly CATE.

Q6: How to evaluate the accuracy of the UPLIFT model if there is only observation data?

A6: We mainly use AUUC and QINI to evaluate the indicators offline. Online, we may mainly convert them into a financial indicator to see the ROI brought by its marketing, depending on our mission goals.

Q7: What is the difference between the application of causal inference in financial scenarios and the application in search and promotion?

A7: There is actually search and promotion in the financial scenario, but it is true that the financial scenario has its uniqueness. This uniqueness lies in its very long conversion link, which involves its credit granting, verification The approval process for cards, etc., and how to estimate the amount in finance is also a difficulty.

Q8: What are the advantages and disadvantages of UPLIFT Model and DML?

A8: Both UPLIFT Model and DML are used to solve CATE. UPLIFT Model uses machine learning as its main tool, such as tree model and deep learning, and also incorporates the idea of ​​causal inference. The DML model uses econometric ideas to guide machine learning tools to estimate CATE. UPLIFT Model is a larger category, and some models also draw on the ideas of DML.

Q9: Is the UPLIFT model mainly used for new customers or old customers? If the number of old customers is only one million, can the model be used?

A9: The issue of magnitude is actually a very critical issue. But the exposure level is not actually the most important. The most important thing is how many positive samples you have. Positive samples are the key to determining whether you can model.

Q10: Is the EFIN code open source?

A10: The paper will be open source after review.

The above is the detailed content of Application of deep UPLIFT model in Tencent Financial user growth scenario. For more information, please follow other related articles on the PHP Chinese website!