pomdp 1.2.7 (09/22/1026)
CRAN release: 2026-09-23
Bugfixes
-
remove_unreachable_states()now handles transition, observation, and reward specifications stored as data frames, normalized dense or sparse structures, distribution keywords, or functions. It also preserves observation names and consistently subsets terminal values and solved-model data. - Expanding an MDP reward function into a dense reward representation no longer passes the POMDP-only
observationargument. - Time-dependent dense reward specifications are now validated per episode, including dense MDP rewards with their single reward column.
- The R POMDP simulator now loads episode-specific transition and observation matrices correctly when simulating time-dependent models.
- The R POMDP simulator now reports action counts from the action counter rather than incorrectly copying state counts.
Changes
-
transition_val(),observation_val(), andreward_val()are deprecated in favor of the corresponding matrix accessors withdrop = TRUE. - Model constructors and public accessors now consistently validate state, action, observation, belief, episode, epoch, discount, and horizon arguments.
- Matrix accessors gained
drop, matchingpolicy()andvalue_function();drop = FALSEpreserves matrix or list dimensions across model representations. Simulation result containers are now consistent between the R and C++ engines when optional results are not requested. - Package examples now use bundled POMDP files instead of downloading models from the internet.
- Building the gridworld vignette no longer requires
gifski. Animations are created when it is available; otherwise, a static final frame is shown.
pomdp 1.2.1 (04/08/2024)
CRAN release: 2024-04-09
pomdp 1.2.0 (04/02/2024)
New Features
- Added functions to work with MDP policies (see ? MDP_policy_functions).
- Added MDP solver functions: Q-learning, Sarsa, and expected Sarsa.
- simulate_MDP() and simulate_POMDP() gained parameter return_trajectories.
- New functions
absorbing_states()andreachable_states()for MDPs and POMDPs. - Support for gridworlds (see ? gridworld).
- New datasets: Cliff_walking, Windy_gridworld, RussianTiger
- plot_transition_graph() now hides unavailable actions.
- Added actions() to find available actions (unavailable actions have a reward of -Inf).
- Added make_partially_observable() and make_fully_observable() to convert between MDPs and POMDPs.
Changes
- simulate_POMDP(): Better calculation of T for infinite-horizon problems.
- several functions are now generics with methods for POMDP and MDP.
- policy() lost the parameters alpha and action.
- policy() and value_function() and gained the parameter drop.
- regret(): renamed parameter belief to start. Regret is now available for MDPs.
- simulate_MDP() stops now at absorbing states.
- simulate_MDP_cpp() works now with sparse model representation.
- POMDP and MDP gained field for additional info.
- approx_MDP_policy_evaluation() is now called MDP_policy_evaluation() and gained parameter theta as an additional stopping criterion.
- rewrote all accessor code reward_matrix, transition_matrix, observation_matrix for better and faster access.
- normalize() gained parameters for more detailed normalization.
- POMDP() and MDP() lost normalize.
- model.h has now support for keywords in transition_prob and observation_prob.
- MDP2POMDP is now make_partially_observable().
pomdp 1.1.3 (12/20/2023)
CRAN release: 2023-12-21
New Features
- simulate_POMDP() and simulate_MDP() gained parameter delta_horizon and calculates now the horizon for infinite-horizon problems.
- added add_policy() and several consistency checks.
pomdp 1.1.1 (09/04/2023)
CRAN release: 2023-09-05
Changes
- plot_policy_graph(): The parameter order has slightly changed; belief_col is now called state_col; unreachable states are now suppressed.
- policy() gained parameters alpha and action.
- color palettes are now exported.
- POMDP accessors gain parameter drop.
- POMDP constructor and read_POMDP gained parameter normalize and, by default, normalize the POMDP definition.
New Features
- Large POMDP descriptions are now handled better by keeping the reward as a data.frame and supporting sparse matrices in the C++ code.
- New function value_function() to access alpha vectors.
- New function regret() to calculate the regret of a policy.
- transition_graph() to visualize the transition model.
pomdp 1.1.0 (01/23/2023)
CRAN release: 2023-01-24
New Features
- Added C++ (Rcpp) support. Speed up for simulate_POMDP, sample_belief_space, reward, …
- simulate_POMDP and sample_belief_space now have parallel (foreach) support.
- Sparse matrices from package Matrix for matrices with a density below 50%.
- Added support to parse matrices for POMDP files.
- Added model normalization.
- is_solved_POMDP(), is_converged_POMDP(), is_timedependent_POMDP(), and is_solved_MDP() are now exported.
Changes
- accessors are now called now transition_val() and observation_val().
- simulate_POMDP() and simulate_MDP() now return a list.
- reimplemented round_stochastic() to improve speed.
- MDP policy now uses factors for actions.
- estimate_belief_for_nodes() now can also use trajectories to estimate beliefs faster.
- cleaned up the interface for episodes and epochs.
pomdp 1.0.2 (05/17/2022)
CRAN release: 2022-05-17
- policy_graph() can now produce policy trees for finite-horizon problems and the initial belief can be specified.
- simulate_POMDP(): fixed bug with not using horizon.
- reward() and reward_node_action() have now been separated.
- sample_belief_space() gained method ‘trajectories’.
- simulate_POMDP(): supports not epsilon-greedy policies.
- added x_prob() and x_val() functions to access individual parts of the matrices.
- fixed converged finite-horizon case. It now only returns the converged graph/alpha.
- we use not internally NA to represent * in the POMDP definition.
- actions, states and observations are now factors in most places.
pomdp 1.0.1 (03/25/2022)
CRAN release: 2022-03-27
- Fixed rounding issue on some architectures.
- Fixed bug in write_POMDP() (reported by emile-pelletier-gc).
- estimate_belief_for_nodes() is now exposed and the code has been improved.
pomdp 1.0.0 (02/23/2022)
CRAN release: 2022-02-24
- POMDP objects now have no list element model, but are the model list directly.
- moved pomdp-solve to package pomdpSolve.
- added solve_MDP().
- transition probability, observation probabilities and rewards can now be specified as a function.
- transition_matrix et al now can also return a function.
- Improved POMDP file writer.
pomdp 0.99.3 (08/05/2021)
CRAN release: 2021-08-05
- moved Ternary and visNetwork to SUGGESTED.
- removed clang warning for lex scanners.
pomdp 0.99.1 (05/13/2021)
CRAN release: 2021-05-14
pomdp 0.99.0 (05/04/2020)
CRAN release: 2020-05-07
Changes
- Support finite-horizon POMDPs and store epochs.
- reward now looks at different epochs, calculates the optimal actions and the parameter names are improved.
- solve_POMDP not looks at convergence.
- solve_POMDP gained parameter terminal_values.
- solve_POMDP gained parameter discount to overwrite the discount rate specified in the model.
- solve_POMDP can now solve POMDPs with time-dependent transition probabilities, observation probabilities and reward structure.
- solve_POMDP gained parameter grid in parameter list to specify a custom belief point grid for the grid method.
- write_POMDP and solve_POMDP gained parameter digits.
- added read_POMDP to read POMDP files.
- plot for POMDP is now replaced by plot_policy_graph.
- added policy graph visualization with visNetwork.
- added plot_value_function.
- added function sample_belief_space to sample from the belief space.
- added function plot_belief_space.
- added function transition_matrix.
- added function observation_matrix.
- added function reward_matrix.
- POMDP model now also contains horizon and terminal_values.
- added MDP formulated as a POMDP.
- added policy function to extract a better readable policy.
- added update_belief.
- added simulate_POMDP.
- added round_stochastic.
- added optimal_action.
pomdp 0.9.2 (12/06/2019)
CRAN release: 2019-12-16