Hypothesis

A model-based reinforcement learning (RL) agent operating on a pharmacogenomic-augmented state space — incorporating HLA-DRB1 shared epitope alleles, CYP3A4/CYP2C19 metabolizer status, FCGR3A V/F158 polymorphism, and serial multi-dimensional disease activity features (DAS28-CRP, HAQ-DI, ultrasound power Doppler scores, serum calprotectin) — will discover biologic sequencing policies that reduce cumulative DAS28 area-under-the-curve (AUC) by ≥25% compared to current guideline-based sequential switching in moderate-to-severe rheumatoid arthritis over a 3-year treatment horizon.

Rationale

Current rheumatology guidelines recommend sequential biologic switching after failure, but the ordering is largely empirical: TNF inhibitors first, then IL-6R blockade or JAK inhibitors, then CD20 depletion or CTLA-4 co-stimulation modulation. This sequence ignores individual pharmacogenomic variation that determines both efficacy and adverse event profiles.

Specifically:

• FCGR3A V158F polymorphism affects rituximab ADCC efficiency, yet is never used to determine RTX positioning in the sequence
• HLA-DRB1 shared epitope copy number correlates with anti-CCP titer trajectory and differential response to abatacept vs. TNFi
• CYP3A4/CYP2C19 metabolizer phenotype affects tofacitinib and upadacitinib exposure, influencing optimal JAKi positioning

RL naturally handles sequential decision-making under uncertainty. By formulating biologic sequencing as a Markov decision process (MDP) with pharmacogenomic features as partially observable state variables, the agent can learn non-myopic policies — potentially identifying that certain patients benefit from early RTX (before TNFi failure) if FCGR3A VV homozygous, or early JAKi if rapid metabolizer status predicts subtherapeutic TNFi exposure.

Proposed Methodology

1. State space: s_t = {DAS28, HAQ-DI, CRP, ESR, calprotectin, US-PD score, anti-CCP titer, RF, treatment history vector, HLA-DRB1 SE copies, FCGR3A genotype, CYP metabolizer status, age, disease duration}
2. Action space: 7 biologic/tsDMARD classes (5 TNFi pooled, tocilizumab, sarilumab, abatacept, rituximab, tofacitinib, upadacitinib, baricitinib)
3. Reward function: -DAS28 at each 3-month decision epoch, with penalty terms for serious adverse events (weighted by severity) and treatment discontinuation
4. World model: Gaussian process transition dynamics learned from retrospective registry data (≥5,000 patients with ≥2 biologic switches and available pharmacogenomic data)
5. Algorithm: Model-based policy optimization (MBPO) with ensemble of probabilistic dynamics models to quantify epistemic uncertainty
6. Validation: Off-policy evaluation via importance-weighted estimators on held-out registry cohort; prospective validation via adaptive platform trial

Testable Predictions

1. The RL-derived policy will differ from guideline-recommended sequencing in ≥40% of patients, primarily by repositioning RTX earlier for FCGR3A VV patients and JAKi earlier for CYP rapid metabolizers
2. Cumulative DAS28-AUC over 36 months will decrease by ≥25% under the RL policy vs. guideline-based switching in off-policy evaluation
3. The learned value function will reveal pharmacogenomic state regions where current guidelines are most suboptimal (highest policy divergence), identifying candidates for prospective trial enrichment
4. Uncertainty-aware policies (using ensemble disagreement as epistemic uncertainty) will show superior performance in patients with rare genotype combinations by defaulting to conservative guideline-aligned actions when data is sparse

Limitations

• Retrospective bias: Registry data reflects guideline-driven prescribing, creating confounding-by-indication. Off-policy evaluation partially mitigates but does not eliminate this.
• Missing pharmacogenomic data: Most registries lack systematic genotyping; imputation or restriction to genotyped subsets reduces sample size.
• Reward specification: The reward function assumes DAS28 adequately captures treatment benefit; patient-reported outcomes and radiographic progression may diverge.
• Generalizability: Policies learned from one registry population may not transfer to different ancestral backgrounds with distinct allele frequencies.
• Regulatory pathway: RL-derived treatment recommendations face regulatory uncertainty for clinical implementation.

Clinical Significance

If validated, this approach would transform biologic sequencing from empirical trial-and-error to genotype-informed precision sequencing. For a disease affecting ~1% of the global population where biologic costs exceed $20,000/year, reducing time-to-optimal-biologic by even one failed trial cycle saves both patient morbidity and healthcare expenditure. The framework generalizes to any chronic disease requiring sequential targeted therapy selection.

RheumaAI Research • rheumai.xyz • DeSci Rheumatology