I am a post-doctoral faculty researcher in the Computer Science and Electrical Engineering department at UMBC. I specialize in machine learning, especially reinforcement learning with a focus on state abstractions for hierarchical reinforcement learning and planning. My PhD was advised by Marie desJardins and Cynthia Matuszek.

My aim is to develop decision-making agents that are more general. I'm working toward that goal by creating algorithms for learning behavior over abstract concepts and transferring this knowledge to new scenarios.


  • 11 NOV 2019 »
    Our paper on abstract model-based RL has been accepted to AAAI 2020. See you there in February!
  • 3 OCT 2019 »
    I'm presenting our research on learning abstract models at the Do Good Robotics Symposium, discussing how our work can be used in domestic service robots.
  • 1 SEP 2019 »
    I've been hired as a faculty research assistant at UMBC in the Interactive Robotics and Language Lab, advising student research groups working in human-robot interaction and concept formation for RL.
  • 20 JUN 2019 »
    I successfully defended my dissertation and passed my final exam! I'll be graduating in August.
  • 10 MAY 2019 »
    My paper The Expected-Length Model of Options has been accepted at IJCAI-19, joint work with Dave Abel and our advisors.
  • 11 AUG 2018 »
    Our NSF IIS proposal on concept formation in POMDPs was accepted.
  • 03 AUG 2018 »
    I'm joining the IRAL lab at UMBC.

Selected Publications

  • John Winder, Stephanie Milani, Matthew Landen, Erebus Oh, Shane Parr, Shawn Squire, Marie desJardins, and Cynthia Matuszek. Planning with Abstract Learned Models While Learning Transferable Subtasks. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20). 2020.
  • David Abel*, John Winder*, Marie desJardins, Michael L. Littman. The Expected-Length Model of Options. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19) [*equal contribution]. 2019.
  • John Winder, Marie desJardins. Concept-Aware Feature Extraction for Knowledge Transfer in Reinforcement Learning. Knowledge Extraction from Games (KEG-18) Workshop at the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). 2018.
  • Karan K Budhraja, John Winder, Tim Oates. Feature Construction for Controlling Swarms by Visual Demonstrations. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 12(2), 10. 2017
  • John Winder, Shawn Squire, Matthew Landen, Stephanie Milani, Marie desJardins. Towards Planning With Hierarchies of Learned Markov Decision Processes. Integrated Execution of Planning and Acting Workshop (IntEx-17) at the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). 2017.
  • Nakul Gopalan, Marie desJardins, Michael L Littman, James MacGlashan, Shawn Squire, Stefanie Tellex, John Winder, Lawson LS Wong. Planning with Abstract Markov Decision Processes. Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). 2017.
  • Nicholay Topin, Nicholas Haltmeyer, Shawn Squire, John Winder, Marie desJardins, James MacGlashan. Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI-15). 2015.