Marek Grzes

is a postdoctoral research fellow in the School of Computer Science at the University of Waterloo where he holds a postdoctoral fellowship from the Ontario Ministry of Research and Innovation and is working with Jesse Hoey on a number of projects.

---

Quick links

• Adaptive Learning Agents Workshop at AAMAS 2011
• Adaptive Learning Agents Workshop at AAMAS 2010
• Graphical Models Reading Group

Research Projects

• Art Therapy for Cognitively Disabled Older Adults (ePad), since 2010
• Cognitive Orthosis for Assisting with aCtivites in the Home (COACH), since 2010
• Sensor Planning and Management, the SIPAM project with MoD/QinetiQ, 2006-2008
• CLUSTERIX (National CLUSTER of LInuX Systems), 2003-2005

Teaching Experience

• Introduction to Machine Learning, 2009
• Logic Programming and Artificial Intelligence, 2007-2009
• Algorithms for Graphical Models, 2007,2009
• Software Engineering, 2003-2006.

Professional Activities

Chair

• The AAMAS 2011 Workshop on Adaptive Learning Agents (ALA)
• The AAMAS 2010 Workshop on Adaptive Learning Agents (ALA)

PC Member

• AAMAS 2010, 2011
• IJCAI 2011
• GAMEON 2010
• AAMAS 2009 Workshop on Adaptive Learning Agents (ALA)

Reviewer

• ICRA 2012
• ICAPS 2011
• IAT 2008

Edited volumes

• Peter Vrancx, Matt Knudson and Marek Grzes (Eds.): Adaptive and Learning Agents, LNAI, Vol. 7113, Springer, 2012.
  
• Peter Vrancx, Matt Knudson and Marek Grzes (Eds.): Proceedings of AAMAS 2011 Workshop on Adaptive and Learning Agents (ALA 2011), Taipei, Taiwan, 2011 (link)
• Marek Grzes and Matthew E. Taylor (Eds.): Proceedings of AAMAS 2010 Workshop on Adaptive and Learning Agents (ALA 2010), Toronto, Canada, 2010 (link)

Refereed Publications

1. Marcin Czajkowski, Marek Grzes, and Marek Kretowski. Multi-Test Decision Trees for Gene Expression Data Analysis. In Proceedings of the International Joint Conference on Security and Intelligent Information Systems. Warsaw, Poland, Springer, LNCS, 7053: 154-167, 2011. (link)
2. Veronika Koltunova, Jesse Hoey, and Marek Grzes. Goal-Oriented Sensor Selection for Intelligent Phones (GOSSIP). In Proceedings of the UbiComp International Workshop on Situation, Activity and Goal Awareness. Beijing, China, ACM, 2011. (link)
3. Marek Grzes, Jesse Hoey, Shehroz Khan, Alex Mihailidis, Stephen Czarnuch, Dan Jackson and Andrew Monk: Relational Approach to Knowledge Engineering for POMDP-based Assistance Systems with Encoding of a Psychological Model. In Proceedings of the ICAPS 2011 Workshop on Knowledge Engineering for Planning and Scheduling (KEPS), Freiburg, Germany, 2011. (link).
4. Sam Devlin, Daniel Kudenko, and Marek Grzes. An empirical study of potential-based reward shaping and advice in complex, multi-agent systems. Advances in Complex Systems, 14(2): 251-278, 2011. (link)
5. Jesse Hoey and Marek Grzes: Distributed control of situated assistance in large domains with many tasks. In Proceedings of International Conference on Automated Planning and Scheduling (ICAPS), Freiburg, Germany, 2011.
6. Marek Grzes and Jesse Hoey: Efficient planning in R-max. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Taipei, Taiwan, 2011.
7. Sam Devlin, Marek Grzes, and Daniel Kudenko: Multi-agent, potential-based reward shaping for robocup keepaway. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Taipei, Taiwan, 2011.
8. Marek Grzes and Daniel Kudenko: Reward shaping and mixed resolution function approximation. In Developments in Intelligent Agent Technologies and Multi-Agent Systems: Concepts and Applications. IGI Press, 2011.
9. Sam Devlin, Marek Grzes, and Daniel Kudenko: Multi-agent Reinforcement Learning with Reward Shaping for KeepAway Takers. In Proceedings of AAMAS 2010 Workshop on Adaptive and Learning Agents (ALA 2010), Toronto, Canada, 2010.
10. Marek Grzes and Daniel Kudenko: PAC-MDP learning with knowledge-based admissible models. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Toronto, Canada, 2010, ACM Press.
11. Marek Grzes and Daniel Kudenko: Online learning of shaping rewards in reinforcement learning. Neural Networks, 23(4): 541-550, 2010.
12. Marek Grzes and Daniel Kudenko: Theoretical and empirical analysis of reward shaping in reinforcement learning. In Proceesings of International Conference on Machine Learning and Applications, Miami, USA, 2009. IEEE Computer Society.
13. Marek Grzes and Daniel Kudenko: Learning shaping rewards in model-based reinforcement learning. In Proceedings of AAMAS 2009 Workshop on Adaptive Learning Agents (ALA 2009), Budapest, Hungary, 2009.
14. Marek Grzes and Daniel Kudenko: Reinforcement learning with reward shaping and mixed resolution function approximation. International Journal of Agent Technologies and Systems (IJATS), 1(2):36-54, 2009.
15. Sam Devlin, Marek Grzes, and Daniel Kudenko: Reinforcement learning in robocup keepaway with partial observability. In Proceedings of the IEEE/WIC/ACM International Conferences on Intelligent Agent Technology (IAT 2009), Milan, Italy, 2009. IEEE Computer Society.
16. Marek Grzes and Daniel Kudenko: Improving optimistic exploration in model-free reinforcement learning. In Proceeding of the International Conference on Adaptive and Natural Computing Algorithms (ICANNGA'09), volume 5495 of LNCS, Kuopio, Finland, 2009. Springer.
17. Marek Grzes, Daniel Kudenko: Plan-based reward shaping for reinforcement learning. In: Proceedings of the 4th IEEE International Conference on Intelligent Systems (IS 2008), IEEE, vol. 2:22-29, Varna, Bulgaria, 2008. (link)
18. Marek Grzes, Daniel Kudenko: Robustness Analysis of SARSA(lambda): Different Models of Reward and Initialisation. In: Proceedings of the 13th International Conference on Artificial Intelligence: Methodology, Systems, Applications (AIMSA 2008), Springer LNAI, vol. 5253:144-156, Varna, Bulgaria, 2008. (link)
19. Marek Grzes, Daniel Kudenko: Multigrid reinforcement learning with reward shaping. In: Proceedings of the 18th International Conference on Artificial Neural Networks (ICANN 2008), Springer LNCS, vol. 5163:357-366, Prague, Czech Republic, 2008. (link)
20. Marek Grzes, Daniel Kudenko: An Empirical Analysis of the Impact of Prioritised Sweeping on the DynaQ's Performance. In: Proceedings of the 9th International Conference on Artificial Intelligence and Soft Computing (ICAISC 2008), Springer LNCS, vol. 5097:1041-1051, Zakopane, Poland, 2008. (link)
21. Marek Grzes, Daniel Kudenko: Learning potential for reward shaping in reinforcement learning with tile coding. In Proceedings of the AAMAS'08 Workshop on Adaptive and Learning Agents and Multi-Agent Systems (ALAMAS-ALAg 2008), pages 17-23, Estoril, Portugal, 2008.
22. Marek Grzes, Daniel Kudenko: Plan-based reward shaping for reinforcement learning. In Proceedings of the AAMAS'08 Workshop on Adaptive and Learning Agents and Multi-Agent Systems (ALAMAS-ALAg 2008), pages 9-16, Estoril, Portugal, 2008.
23. Marek Kretowski, Marek Grzes: Global Induction of Decision Trees. Encyclopedia of Data Warehousing and Mining - 2nd Edition, Idea Group Inc., 2008.
24. Marek Kretowski, Marek Grzes: Evolutionary Induction of Mixed Decision Trees. In Wang, J. (Ed.), Data Warehousing and Mining: Concepts, Methodologies, Tools, and Applications, Idea Group Inc., 2008.
25. Marek Kretowski, Marek Grzes: Evolutionary Induction of Mixed Decision Trees. International Journal of Data Warehousing and Mining, vol. 3(4): 68-82, 2007. (link)
26. Marek Grzes, Marek Kretowski: Decision Tree Approach to Microarray Data Analysis. Biocybernetics and Biomedical Engineering, vol. 27(3): 29-42, 2007.
27. Marek Kretowski, Marek Grzes: Evolutionary Induction of Decision Trees for Misclassification Cost Minimization. In: Proceedings of 8th International Conference on Adaptive and Natural Computing Algorithms (ICANNGA 2007), Warsaw, Poland, Springer LNCS, vol. 4431:1-10, 2007.
28. Marek Kretowski, Marek Grzes: Evolutionary Induction of Cost-Sensitive Decision Trees. In: Proceedings of 16th International Symposium on Methodologies for Intelligent Systems (ISMIS 2006), Bari, Italy, Springer LNAI, vol. 4203:121-126, 2006.
29. Marek Kretowski, Marek Grzes: Mixed Decision Trees: An Evolutionary Approach. In: Proceedings of 8th International Conference on Data Warehousing and Knowledge Discovery (DaWaK 2006), Krakow, Poland, Springer LNCS, vol. 4081:260-269, 2006.
30. Marek Kretowski, Marek Grzes: Evolutionary Learning of Linear Trees with Embedded Feature Selection. In: Proceedings of 8th International Conference on Artificial Intelligence and Soft Computing (ICAISC 06), Zakopane, Poland, Springer LNCS, vol. 4029:400-409, 2006.
31. Marek Grzes, Marek Kretowski: Decision tree approach to microarray data analysis. In: Proceedings of 6th International Seminar on Statistics and Clinical Practice. Lecture Notes in The ICB Seminar, vol. 70:105-111, 2005.
32. Marek Kretowski, Marek Grzes: Global learning of decision trees by an evolutionary algorithm. In: Information Processing and Security Systems. Springer, pages 401-410, 2005.
33. Marek Kretowski, Marek Grzes: Global Induction of Oblique Decision Trees: An Evolutionary Approach. In: Intelligent Information Processing and Web Mining. Proceedings of Intelligent Information Systems (IIS 2005), Springer, pages 309-319, 2005.
34. Marek Kretowski, Marek Grzes: An evolutionary algorithm for global induction of decision trees. ACS-CISIM 04, pages 63-70, Elk, Poland, 2004.

Other Publications

1. Daniel Kudenko and Marek Grzes: Knowledge-based reinforcement learning for data mining. In Proceedings of the AAMAS workshop on Agents and Data Mining Interaction, volume 5680 of LNCS, Budapest, Hungary, 2009. Springer-Verlag.

Talks

1. Marek Grzes and Daniel Kudenko. Robustness analysis of SARSA(lambda): Different models of reward and initialisation. In The European Workshop on Reinforcement Learning 2008 (EWRL'08), Lille, France, 2008.
2. Marek Grzes and Daniel Kudenko. Multigrid reinforcement learning with reward shaping. In The European Workshop on Reinforcement Learning 2008 (EWRL'08), Lille, France, 2008.

Dissertations

• Marek Grzes: Improving Exploration in Reinforcement Learning through Domain Knowledge and Parameter Analysis. PhD thesis, University of York, United Kingdom, 2010.
• Marek Grzes: Parallel execution of SQL queries in a distributed heterogeneous database environment. MSc Eng. dissertation, Bialystok University of Technology, Poland, 2003.

---

Software

• Gaussian Mixture Model learning using EM
• 0-1 Knapsack
• Small examples which I often find useful
• Ideas for programming and debugging learned by trial and error

---

Related and Useful Links

• Documents
  • Neural Networks FAQ
  • Statistical Data Mining Tutorials
  • Encyclopedia of Machine Learning
• Software
  • APPL, a C++ implementation of the SARSOP algorithm
  • dlib C++ machine learning library
  • Fast Artificial Neural Network Library (FANN)
  • GeNIe&SMILE: Influence diagrams and Bayesian networks modelling
  • MDPSim: PPDDL Plan Evaluation Simulator
  • Netlab neural network software
  • Perseus and Symbolic Perseus
  • PyBrain
  • RL-Glue: Reinforcement Learning Glue
  • ZMDP Software for POMDP and MDP Planning
• Misc
  • Who is interested in algorithms and why?

---

Academic Genealogy

• My advisor: Daniel Kudenko
  (Ph.D. Computer Science, Rutgers University, 1998: "Feature Generation for Sequence Categorization")
• Kudenko's advisor: Haym Hirsh
  (Ph.D. Computer Science, Stanford University, 1989: "Incremental Version Space Merging: A General Framework for Concept Learning")
• Hirsh's advisor: Bruce Buchanan
  (b. July 7, 1940, St. Louis)
  (Ph.D. Philosophy, Michigan State University, 1966: "Logics of Scientific Discovery")
• Buchanan's advisor: Gerald Massey
  (b. Feb 11, 1934, Wauseon, Ohio)
  (Ph.D. Philosophy, Princeton University, 1964: "The Philosophy of Space ")
• Massey's advisor: Carl Gustav Hempel
  (b. Jan. 8, 1905, Oranienburg, Germany; d. 1997, Princeton, NJ)
  (Philosophy, Berlin, 1934: "Beitrage zur logischen analyse des wahrscheinlichkeitsbegriffs")
  (Hempel worked with Reichenbach until Reichenbach was forced to leave Germany in 1933; his final year at Erlangen was spent under the supervision of Wolfgang Kohler and Nicolai Hartmann.)
• Hempel's advisor: Hans Reichenbach
  (b. Sept. 26, 1891, Hamburg, Germany; d. April 9, 1953, Los Angeles, CA)
  (Philosophy, Erlangen, 1915: "Der Begriff der Wahrscheinlichkeit fur die mathematische Darstellung der Wirklichkeit")
• Reichenbach's two advisors:
  
  • Paul Hensel
    (Dr. phil., Albert-Ludwigs-Universitat Freiburg, 1885: "Uber die Beziehung des reinen Ich bei Fichte zur Einheit der Apperception bei Kant")
    Please email me if you know anything about him
    
  • Max Noether
    (b. September 24, 1844, Mannheim, Baden, Germany; d. December 13, 1921, Erlangen, Germany)
    (Dr. phil. Ruprecht-Karls-Universitat Heidelberg, 1868)
    Based on information Haym Hirsh received from Hans Reichenbach's widow, Haym Hirsh had previously listed the second advisor as Emmy Noether. He has recently learned that this did not match up with Reichenbach's dissertation's signature page, which has Max Noether's signature.
• Noether's two advisors:
  
  • Ludwig Otto Hesse
    (b. April 22, 1811, Kvnigsberg, Germany (now Kaliningrad, Russia); d. August 4, 1874, Munich, Germany)
    (Dr. phil. Universitat Kvnigsberg 1840: "De octo punctis intersectionis trium superficium secundi ordinis")
  • Gustav Robert Kirchhoff
    (b. March 12, 1824, Kvnigsberg, Prussia (now Kaliningrad, Russia); d. October 17, 1887, Berlin, Germany)
    (Dr. phil. Universitat Konigsberg 1847: "De criteriis quibus cognoscatur an aequatio quinti gradus irreductibilis algebraice resolvi posset / De parallaxi stellae Argelandriae")
    Kirchhoff's advisor was also Otto Hesse
• Hesse's advisor: Carl Gustav Jacob Jacobi
  (b. December 10, 1804, Potsdam, Prussia; d. February 18, 1851, Berlin, Germany)
  (Ph.D. Humboldt-Universitat zu Berlin, 1825: "Disquisitiones Analyticae de Fractionibus Simplicibus")
• Jacobi's advisor: Enno Heeren Dirksen
  (Ph.D. Georg-August-Universitat Gottingen, 1820: "Historiae progressuum instrumentorum mensurae angulorum accuratiori interserventium inde a Tob. Meyeri temporibus ad umbratione non de artificio multiplicationis")
  
• His two advisors:
  
  • Friedrich Anton Justus Thibaut
    (b. January 4, 1772, Hameln, Hanover; d. March 28, 1840, Heidelberg)
    (Dr. phil. Christian-Albrechts-Universitat zu Kiel, 1796: "De genuina iuris personarum et rerum indole, veroque huius divisionis pretio")
    (Advisor unknown)
  • Johann Tobias Meyer d.J.
    (b. February 17, 1723, Marbach, Wurttemberg, Germany; d. February 20, 1762, Gottingen, Germany)
    (Ph.D. Georg-August-Universitat Gottingen, 1773: "Tetragonometriae specimen I")
• Meyer's advisor: Abraham Gotthelf Kaestner
  (b. Sept 27, 1719, Leipzig, Germany; d. June 20, 1800, in Gottingen, Germany)
  (Ph.D. Universitat Leipzig, 1739: "Theoria radicum in aequationibus")
• Kaestner's advisor: Christian A. Hausen
  (b. 1693, d. 1743)
  (Dr. phil. Martin-Luther-Universitat Halle-Wittenberg 1713: "De corpore scissuris figurisque non cruetando ductu")
• Hausen's advisor: Johann Christoph Wichmannshausen
  (Ph.D. Universitat Leipzig, 1685: "Disputationem Moralem De Divortiis Secundum Jus Naturae")
• Wichmannshausen's advisor: Otto Mencken
  (Ph.D. Universitat Leipzig, 1668: "Thomae Hobbesii Epicureismum historice delineatum sistit")

This tree was created by Haym Hirsh and its original version can be found here.

---

Last modified Wednesday, 22nd February, 2012 09:57am