Personalization is important in many fields: in healthcare we try to find the “right treatment for the right person”, in online marketing we try to find the “right product for the right customer”, and in education we try to find “the right educational plan for the right student”. In each of these fields we assume that matching treatments to individuals improves their effectiveness.
Abstractly, the problem of personalizing treatments is the same across disciplines: given a feature vector $\vec{x}$ describing the current person we select a treatment assignment $\vec{a}$ to attain some outcome $r$. An effective personalization policy $\pi()$ maximizes the cumulative outcome over all treatment assignments. The computational personalization (CP) lab at JADS develops and analyses sequential allocation policies and builds software to deploy these policies in field experiments. The lab’s mission is to become the world leader in \emph{data science methods for treatment personalization.}
The lab identifies a number of concrete challenges to progress further:
- Novel policies: While a large number of policies have been proposed, the majority of these is of interest for their (asymptotic) properties under stringent assumptions. In our lab we are interested in developing policies that perform well on more realistic formalizations of the problem (e.g., finite horizon, latent rewards, delayed rewards, changing actions sets, etc.).
- Causality: In many applications we find that observational data regarding a process is already available. However, since we care about the \emph{causal} effects of the selected actions, we need to develop novel methods of estimating causal effects based on (mixed) observational and interventional data.
- Computation: In many situations treatment decisions will need to be made for large groups of people in (near) real-time; the CP lab focusses on developing scalable (online / row-by-row) methods to fit the statistical models used by different policies.
- Application: With different applications come different challenges: in e-commerce data volumes are extremely large, and the size of the action space is large and dynamic. In healthcare the number of observations is often small, but multiple outcome measures are of interest and human understandability / accountability of the policies is required.
Organization
The CP lab is led by prof. dr. Maurits Kaptein. The lab currently consist of the PI, 7 PhD students, one Post-doc student, and a varying number of (research) master students. The lab holds weekly lab meetings. Jointly we strengthen the quality of the work originating from the lab. Next to the weekly lab meetings, there are scheduled, bi-weekly, bilateral meetings between the PI and the lab members. Every year the performance of the lab members is evaluated by the PI. The research in the lab is currently structured within separate projects:
- Applied contextual multi-armed bandit solutions (CMAB): This project focusses on developing software to address cMAB problems in field studies and to perform (simulated) policy evaluations.
- Personalized eHealth Telescope (PET): The Health-Telescope focusses on evaluating the effectiveness of eHealth applications and developing novel methods for eHealth personalization. This project is funded by CZ and the TiU Impact program.
- Personalized Churn prevention (Churn): Together with DSC/t we are currently working on effective, personalized reinforcement learning approaches to fighting churn.
- Causality (Cause): Given the importance of estimating causal effects at the level of individuals we separately study causal inference and its applications.
- Medical trials (MT): In collaboration with TiU we work on developing effective Bayesian allocation methods for clinical trails that deal with multiple end-point and heterogeneity.