Overview
The approach
We estimate the incremental value and incremental cost of each eligible action, evaluate candidate policies with logged randomized data, and allocate incentives under budget and efficiency constraints.
What makes this hard
The design has to be causally valid, rank well, stay within budget, and pass an online test. Improving one of these while ignoring the others gives gains that do not last.
Use randomized assignment and propensity logs to define causal labels.
Estimate heterogeneous value and cost with calibrated uplift models.
Prune invalid actions, rank by marginal economics, and optimize.
Use IPS, doubly robust OPE, and online tests before scaling.
Keep exploring so future policies can still be identified.
Concept map
Randomization prevents confounding. Separate value and cost heads keep one objective from leaking into the other. OPE checks offline claims. Pacing keeps spend on budget, and monitoring catches drift.
Heuristic campaigns
A rule-based incentive is easy to operate, but it can waste spend because it has no counterfactual measurement and does not choose an action for each user.
Heuristic policy
- Eligibility is based on coarse lifecycle rules.
- Offer depth is mostly fixed or manually tuned.
- Reporting focuses on observed redemption or conversion.
- Budgeting is separated from user-level marginal value.
Causal allocation policy
- Eligibility is paired with randomized holdouts.
- Offer choice is action-specific and user-specific.
- Measurement targets incremental value and incremental cost.
- The decision layer optimizes under explicit constraints.
Formal problem
Outcome value
Use the definition of value that matches the business decision. A training proxy is fine if it keeps the allocation quality and is reconciled to the accounting metric.
Incremental cost
Cost is the cost caused by assigning the offer. It can differ from the nominal offer value, expected liability, deposited credit, or realized spend.
No-offer action
No offer has to be one of the actions. Otherwise budget goes to users who would be best left without an offer.
Identification
If past assignment reflects intent, observed outcomes mix user state, treatment choice, and business rules. Randomized assignment is what makes causal labels possible.
Binary treatment design
For a fixed campaign variant, randomize eligible users between treatment and holdout. This identifies campaign-specific heterogeneous effects for value and cost.
Multi-action design
For a menu of actions, log the probability of each assigned action. Offline policy evaluation needs the propensity of every action. A treatment flag is not enough.
Common failure modes
- Too many actions: each action gets less data, and the variance of the effect estimates goes up.
- Long-horizon labels: outcome windows that are too long can become too noisy for ranking.
- Interference: other incentive programs running at the same time can contaminate the holdout.
Modeling
Value and cost are modeled separately because they are different statistical problems.
The optimizer takes the estimates as inputs. If value and cost differ in noise, label density, or calibration, a single shared score can hide a problem in one of them.
Uplift learners
They target heterogeneous treatment effects directly and can handle treatment-control imbalance better than naive conversion models.
Features
Segment users where their behavior and the identification are similar. A model for one lifecycle stage can beat a global model early in a program.
Model selection
Choose on incremental calibration, monotonic ranking in holdout bins, and stability across splits, ahead of raw predictive fit.
Labels
Longer outcome windows can be better for business reporting and worse for model training if they dilute the rank signal.
Learning horizon
The horizon used to train and rank candidate actions. It should match how quickly users respond to the intervention and keep the incremental ordering.
Accounting horizon
The horizon used for planning, reporting, and financial reconciliation. It can be longer, as long as it is not also the training target, where it adds variance.
Decision layer
Estimate incremental value and cost for eligible user-action pairs.
Remove negative-value or poor-efficiency pairs before optimization.
Order the remaining pairs by marginal economics instead of raw conversion.
Maximize value under budget, efficiency, and eligibility constraints.
Adjust available budget using actual spend and expected liability.
Offline policy evaluation
A model metric can look good while the policy it produces does poorly under the logged action distribution. OPE compares policies directly.
IPS
IPS needs known propensities and enough overlap. It is direct, but its variance is high when propensities are small.
Doubly robust
DR combines an outcome model with a propensity correction. It can reduce variance, and it gives a second estimate of policy value.
Online test
OPE is a screening step. An online experiment is still needed when interference and implementation effects matter.
Exploration
An adaptive policy has to keep exploring. If exploration is skewed toward promising areas, the propensities have to be logged and corrected for.
Warm-start exploration keeps the same estimand. It changes the sampling policy, and logged propensities correct for that.
Risk
Skewed exploration can bias naive averages and leave low-probability actions with too little data.
Mitigation
Log propensities, enforce minimum exploration where needed, use IPW or DR estimators, and watch overlap diagnostics.
Interference between programs
When multiple incentive programs touch overlapping audiences, the measured effect of one program can depend on what the other programs are doing.
What goes wrong
- What a holdout measures depends on which other programs are running.
- The lift measured for one program can be biased if the other assignments are not logged.
- Policies can compete for the same marginal user.
What to build
- Log every assignment that can affect the outcome.
- Add competing-treatment features when possible.
- Partition audiences when stacking cannot be controlled.
- Coordinate experiments across programs.
Evaluation
These checks ask whether the policy creates incremental value under the constraints it will face.
Cost and value separately.
Monotonic bins and split stability.
Budget, thresholds, edge cases.
IPS and DR versus baseline policy.
Experiment results and monitoring.
Lessons
| # | Lesson | Mechanism | Design rule |
|---|---|---|---|
| 1 | Long outcome horizons are hard to learn from. | The incremental signal gets sparse and noisy. | Train on a stable ranking proxy and report longer-run outcomes separately. |
| 2 | A large menu of actions needs enough data for each action. | The sample per action shrinks and the effect estimates become unstable. | Start with a small menu, then add actions near the ones that work. |
| 3 | Cost and value need separate diagnostics. | They have different label density and variance. | Use separate heads and calibrate them separately. |
| 4 | A campaign average hides differences between users. | Some user-action pairs are efficient even when the average is not. | Optimize at the user-action level. |
| 5 | No offer has to be one of the actions. | Forcing a treatment spends budget on users who should get nothing. | Include no offer in the policy's action space. |
| 6 | The exploration policy affects the estimator. | Skewed assignment changes which users and actions are sampled. | Log propensities and correct with IPS or DR. |
| 7 | Fixed thresholds go out of date. | The offer mix, user behavior, and costs drift. | Recalibrate and keep monitoring. |
| 8 | Features have to match between training and serving. | A mismatch between offline and online features makes the policy worse. | Enforce feature contracts and monitor serving. |
Equations
Common questions
How is this different from propensity-to-buy targeting?
Propensity predicts what a user is likely to do. Uplift estimates what the action changes. Users with high propensity may convert without an incentive, and users with lower propensity may respond a lot to one. So the decision should use incremental value net of incremental cost instead of conversion probability.
Why are randomized holdouts needed?
Past assignment is usually endogenous. The business chose who to treat based on lifecycle state, spend intent, or risk. Without randomization, observed outcomes mix the user's state with the effect of the treatment. Holdouts give the counterfactual labels needed to estimate incremental value and cost.
Why not maximize predicted value directly?
Incentives use a limited budget. An action with high value can still be inefficient if it costs too much. The policy maximizes incremental value subject to the budget and a marginal efficiency threshold.
Why can offline model metrics be misleading?
The metric may not match the policy's objective. A model can have good prediction error and still produce bad allocations if it orders marginal users wrongly or is miscalibrated where spend is high. Offline policy evaluation uses the logged propensities to estimate the value of the policy itself.
What should be monitored after launch?
Assignment volume, spend pacing, realized cost, value calibration, rank monotonicity, feature drift, action propensities, the no-offer rate, overlap diagnostics, and online experiment results. That covers both the model and the allocation policy.
