Markov Decision Process Framework (Dynamic Pricing)

The problem

Many pricing and capacity-constrained problems are sequential: today’s decision changes tomorrow’s state. That’s an MDP—but real-world MDPs often have:

• time-dependent states
• stochastic transitions learned from data
• constraints and business rules

What I built

A framework that makes it straightforward to:

• define states/actions/rewards with explicit time dependence
• estimate transition probabilities from observed data
• compute optimal policies (and compare to baselines)

Why it’s interesting

• clear abstractions (state/action/reward)
• measurable evaluation (policy performance under simulation)
• practical handling of constraints (not just textbook RL)