Package index • pomdp

POMDPs

Define, inspect, transform, simulate, and analyze partially observable Markov decision process models.

POMDP() is_solved_POMDP() is_timedependent_POMDP() epoch_to_episode() is_converged_POMDP() O_() T_() R_(): Define a POMDP Problem
update_belief(): Belief Update
simulate_POMDP(): Simulate Trajectories Through a POMDP
sample_belief_space(): Sample from the Belief Space
make_partially_observable() make_fully_observable(): Convert between MDPs and POMDPs
start_vector() normalize_POMDP() normalize_MDP() reward_matrix() reward_val() transition_matrix() transition_val() observation_matrix() observation_val(): Access to Parts of the Model Description
actions(): Available Actions
add_policy(): Add a Policy to a POMDP Problem Description
plot_belief_space(): Plot a 2-State or 3-State Projection of the Belief Space
projection(): Defining a Belief Space Projection
reachable_states() absorbing_states() remove_unreachable_states(): Reachable and Absorbing States
regret(): Calculate the Regret of a Policy
solve_POMDP() solve_POMDP_parameter(): Solve a POMDP Problem using pomdp-solver
solve_SARSOP(): Solve a POMDP Problem using SARSOP
transition_graph() plot_transition_graph(): Transition Graph
value_function() plot_value_function(): Value Function
write_POMDP() read_POMDP(): Read and write a POMDP Model to a File in POMDP Format

Define, inspect, transform, simulate, and analyze finite state-space Markov decision process models.

MDP() is_solved_MDP(): Define an MDP Problem
solve_MDP() solve_MDP_DP() solve_MDP_TD(): Solve an MDP Problem
simulate_MDP(): Simulate Trajectories in an MDP
q_values_MDP() MDP_policy_evaluation() greedy_MDP_action() random_MDP_policy() manual_MDP_policy() greedy_MDP_policy(): Functions for MDP Policies
make_partially_observable() make_fully_observable(): Convert between MDPs and POMDPs
start_vector() normalize_POMDP() normalize_MDP() reward_matrix() reward_val() transition_matrix() transition_val() observation_matrix() observation_val(): Access to Parts of the Model Description
actions(): Available Actions
add_policy(): Add a Policy to a POMDP Problem Description
gridworld_init() gridworld_maze_MDP() gridworld_s2rc() gridworld_rc2s() gridworld_matrix() gridworld_plot_policy() gridworld_plot_transition_graph() gridworld_animate(): Helper Functions for Gridworld MDPs
reachable_states() absorbing_states() remove_unreachable_states(): Reachable and Absorbing States
regret(): Calculate the Regret of a Policy
transition_graph() plot_transition_graph(): Transition Graph
value_function() plot_value_function(): Value Function

Solve finite state-space MDP and POMDP models using exact, approximate, and reinforcement learning methods.

solve_POMDP() solve_POMDP_parameter(): Solve a POMDP Problem using pomdp-solver
solve_MDP() solve_MDP_DP() solve_MDP_TD(): Solve an MDP Problem
solve_SARSOP(): Solve a POMDP Problem using SARSOP

Extract, evaluate, inspect, and visualize policies and value functions for solved models.

policy(): Extract the Policy from a POMDP/MDP
value_function() plot_value_function(): Value Function
optimal_action(): Optimal action for a belief
reward() reward_node_action(): Calculate the Reward for a POMDP Solution
plot_policy_graph() curve_multiple_directed(): POMDP Plot Policy Graphs
estimate_belief_for_nodes(): Estimate the Belief for Policy Graph Nodes
plot_belief_space(): Plot a 2-State or 3-State Projection of the Belief Space
policy_graph(): POMDP Policy Graphs
projection(): Defining a Belief Space Projection
solve_POMDP() solve_POMDP_parameter(): Solve a POMDP Problem using pomdp-solver
solve_SARSOP(): Solve a POMDP Problem using SARSOP

Create gridworld MDPs, convert between states and grid positions, and visualize their policies and transitions.

gridworld_init() gridworld_maze_MDP() gridworld_s2rc() gridworld_rc2s() gridworld_matrix() gridworld_plot_policy() gridworld_plot_transition_graph() gridworld_animate(): Helper Functions for Gridworld MDPs
Cliff_walking cliff_walking: Cliff Walking Gridworld MDP
DynaMaze dynamaze: The Dyna Maze
Maze maze: Steward Russell's 4x3 Maze Gridworld MDP
Windy_gridworld windy_gridworld: Windy Gridworld MDP

Example POMDP specifications and example files for learning, testing, and demonstrating package functionality.

Tiger Three_doors: Tiger Problem POMDP Specification
RussianTiger: Russian Tiger Problem POMDP Specification
POMDP_example_files: POMDP Example Files
POMDP() is_solved_POMDP() is_timedependent_POMDP() epoch_to_episode() is_converged_POMDP() O_() T_() R_(): Define a POMDP Problem

Example MDP specifications, including classic maze and gridworld problems from reinforcement learning.

Color palettes for visualizations and rounding helpers for stochastic vectors and matrices.

colors_discrete() colors_continuous(): Default Colors for Visualization in Package pomdp
round_stochastic(): Round a stochastic vector or a row-stochastic matrix