Simulating Behaviors of POMDPs

Advisor(s)

Piotr Gmytrasiewicz, UIC Artificial Intelligence Laboratory

Location

Room B1018-2

Start Date

26-4-2019 11:05 AM

End Date

26-4-2019 11:20 AM

Abstract

Intuition is an important human capability that allows us to gauge and predict the properties of an unknown environment. This study presents an approach to how a computer agent can replicate this innate intuition by using an algorithm that will learn from an unknown underlying state using Partially Observable Markov Decision Processes (POMDPs). This task is difficult because the computer will never have 100% certainty in its decisions, thus every action will be based on a belief state. To determine the transition probabilities in our unknown environment, we are currently recursively applying a learning algorithm, looking for patterns in the computer’s belief state. We are implementing these ideas and algorithms using Java in an effort to create an intelligent machine. We test the competence of our algorithms by running them in a simulation with known transition probabilities and finding percent error between calculated probabilities and true probabilities. Although a robust, error-free algorithm has not been developed, we hope to find a logical and consistent algorithm that will determine transition and observational probabilities accurately and consistently.

Share

COinS
 
Apr 26th, 11:05 AM Apr 26th, 11:20 AM

Simulating Behaviors of POMDPs

Room B1018-2

Intuition is an important human capability that allows us to gauge and predict the properties of an unknown environment. This study presents an approach to how a computer agent can replicate this innate intuition by using an algorithm that will learn from an unknown underlying state using Partially Observable Markov Decision Processes (POMDPs). This task is difficult because the computer will never have 100% certainty in its decisions, thus every action will be based on a belief state. To determine the transition probabilities in our unknown environment, we are currently recursively applying a learning algorithm, looking for patterns in the computer’s belief state. We are implementing these ideas and algorithms using Java in an effort to create an intelligent machine. We test the competence of our algorithms by running them in a simulation with known transition probabilities and finding percent error between calculated probabilities and true probabilities. Although a robust, error-free algorithm has not been developed, we hope to find a logical and consistent algorithm that will determine transition and observational probabilities accurately and consistently.