Sample problems included with SymbolicPerseus
---------------------------------------------

Author: Pascal Poupart (ppoupart@cs.uwaterloo.ca)

* POMDP problems
* Syntax to encode POMDP problems

POMDP problems
--------------

The following problems are provided with SymbolicPerseus:

* Coffee: classic coffee problem that illustrates how to encode a
simple factored POMDP.

* Handwashing: 3 versions of the handwashing problem

    1) problems/handwashing/cppo3.txt 
	- 180 states, 6 observations, 6 actions
        - reduced version of the handwashing problem
        
    2) problems/handwashing/ver26_15_unstructured.txt 
	- 50,181,120 states, 12 observations, 20 actions
	- Reference: A Decision-Theoretic Approach to Task Assistance for Persons with Dementia
                     Jennifer Boger, Pascal Poupart, Jesse Hoey, Craig Boutilier, Geoff Fernie, and Alex Mihailidis
                     In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 1293-1299, Edinburgh, Scotland, 2005

    3) problems/handwashing/cppo37c_unstructured.txt
        - 20,7360 states, 198 observation, 26 actions
        - Reference: Assisting Persons with Dementia during Handwashing Using a Partially Observable Markov Decision Process
                     Jesse Hoey, Axel von Bertoldi, Pascal Poupart, and Alex Mihailidis
                     In Proceedings of the International Conference on Vision Systems (ICVS), Biefeld, Germany, 2007.
 
* Networks: system administration network problems (cycle and 3legs).

The network problems can be generated by running the matlab commands

>> pomdp3legsFlatAct(N)
>> pomdpCycleFlatAct(N)

where N is the number of machines in the network. A network of N
machines gives rise to a POMDP of 2^N states. Alternatively,
SPUDD encodings of the 3leg and cycle problems are provided in the
problems/network directory for networks of 4, 7, 10, 13, 16, 19, 22
and 25 machines.  

References for the network problems:

VDCBPI: an Approximate Scalable Algorithm for Large Scale POMDPs
Pascal Poupart and Craig Boutilier
In Advances in Neural Information Processing Systems 17 (NIPS), pages 1081-1088, Vancouver, BC, 2004

Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes
Pascal Poupart
Ph.D. thesis, Department of Computer Science, University of Toronto, Toronto, 2005

Syntax
------

The file problems/SYNTAX describes the formal grammar used to 
encode the above factored POMDP problems. The format is a subset of
the SPUDD format originally developed by Jesse Hoey
(http://www.computing.dundee.ac.uk/staff/jessehoey/spudd/). 

For a simple example that illustrates this syntax, see 
problems/coffee/coffee3po.txt, which encodes the classic coffee 
problem.

POMDP problems encoded with this syntax can be read by the 
symbolicPerseus package using the matlab command: 

>> parsePOMDP('path/problemName')  

The class javaClasses/ParseSPUDD encodes the parser used by
symbolicPerseus.

It is possible to convert small POMDP problems to the Cassandra format
by typing the following matlab commands:

>> ddPOMDP = parsePOMDP('path/problemName');
>> convertPOMDP2CassandraFormat(ddPOMDP,'path/fileName');

Note that the Cassandra format requires the enumeration of all states
(flat format) and therefore may not be suitable for large problems.
Depending on the amount of RAM available it may not be possible to
convert problems with more than 10,000 states.