Find reachable and absorbing states in the transition model.
Usage
reachable_states(x, states = NULL)
absorbing_states(x, states = NULL)
remove_unreachable_states(x)Value
reachable_states() returns a logical vector indicating
if the states are reachable.
absorbing_states() returns a logical vector indicating
if the states are absorbing (terminal).
the model with all unreachable states removed
Details
The function reachable_states() checks if states
are reachable using the transition model.
The function absorbing_states() checks if a state or a set of states are
absorbing (terminal states) with a zero reward (or -Inf for unavailable actions).
If no states are specified (states = NULL), then all model states are
checked. This information can be used in simulations to end an episode.
The function remove_unreachable_states() simplifies a model by
removing unreachable states.
See also
Other MDP:
MDP(),
MDP2POMDP,
MDP_policy_functions,
accessors,
actions(),
add_policy(),
gridworld,
regret(),
simulate_MDP(),
solve_MDP(),
transition_graph(),
value_function()
Other POMDP:
MDP2POMDP,
POMDP(),
accessors,
actions(),
add_policy(),
plot_belief_space(),
projection(),
regret(),
sample_belief_space(),
simulate_POMDP(),
solve_POMDP(),
solve_SARSOP(),
transition_graph(),
update_belief(),
value_function(),
write_POMDP()
Examples
data(Maze)
gridworld_matrix(Maze, what = "label")
#> [,1] [,2] [,3] [,4]
#> [1,] "" "" "" "Goal: +1"
#> [2,] "" "X" "" "-1"
#> [3,] "Start" "" "" ""
# the states marked with +1 and -1 are absorbing
absorbing_states(Maze)
#> s(1,1) s(2,1) s(3,1) s(1,2) s(3,2) s(1,3) s(2,3) s(3,3) s(1,4) s(2,4) s(3,4)
#> FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE FALSE
which(absorbing_states(Maze))
#> s(1,4) s(2,4)
#> 9 10
# all states in the model are reachable
reachable_states(Maze)
#> s(1,1) s(2,1) s(3,1) s(1,2) s(3,2) s(1,3) s(2,3) s(3,3) s(1,4) s(2,4) s(3,4)
#> TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
which(!reachable_states(Maze))
#> named integer(0)