I'm John Winder, a research scientist at The Johns Hopkins University Applied Physics Laboratory (APL) working on machine learning for complex real-world systems.

I received my Ph.D. in Computer Science from UMBC, where I specialized in reinforcement learning (RL) with a focus on state abstractions for hierarchical RL and probabilistic planning. I had the good fortune to be advised by Marie desJardins and Cynthia Matuszek.

My aim is to develop decision-making agents that generalize and reason about long-term goals under uncertainty, operating in dynamic and open environments. I'm working toward that objective by creating algorithms that enable agents to learn behavior over abstract concepts and transfer this knowledge to new challenges in diverse scenarios.


  • 17 FEB 2021 »
    I've been promoted to Section Supervisor for the Advanced Artificial Intelligence Algorithms section, working on RL and autonomous systems. We're hiring!
  • 13 JUL 2020 »
    I joined APL as a Senior Professional Staff Scientist researching RL for intelligent platforms.
  • 09 JUL 2020 »
    Our National Robotics Initiative proposal was awarded a three year grant! The IRAL lab will be studying grounded language learning and concept-based knowledge transfer in deep RL for collaborative robots.
  • 11 NOV 2019 »
    Our paper on abstract model-based RL has been accepted to AAAI 2020. See you there in February!
  • 03 OCT 2019 »
    I'm presenting our research on learning abstract models at the Do Good Robotics Symposium, discussing how our work can be used in domestic service robots.
  • 01 SEP 2019 »
    I've been hired as a faculty research assistant at UMBC in the Interactive Robotics and Language Lab, advising student research groups working in human-robot interaction and concept formation for RL.
  • 20 JUN 2019 »
    I successfully defended my dissertation and passed my final exam! I'll be graduating in August.
  • 10 MAY 2019 »
    My paper The Expected-Length Model of Options has been accepted at IJCAI-19, joint work with Dave Abel and our advisors.
  • 11 AUG 2018 »
    Our NSF IIS proposal on concept formation in POMDPs was accepted.
  • 03 AUG 2018 »
    I'm joining the IRAL lab at UMBC.

Selected Publications

  • John Winder, Stephanie Milani, Matthew Landen, Erebus Oh, Shane Parr, Shawn Squire, Marie desJardins, and Cynthia Matuszek. Planning with Abstract Learned Models While Learning Transferable Subtasks. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20). 2020.
  • David Abel*, John Winder*, Marie desJardins, Michael L. Littman. The Expected-Length Model of Options. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19) [*equal contribution]. 2019.
  • John Winder, Marie desJardins. Concept-Aware Feature Extraction for Knowledge Transfer in Reinforcement Learning. Knowledge Extraction from Games (KEG-18) Workshop at the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). 2018.
  • Karan K Budhraja, John Winder, Tim Oates. Feature Construction for Controlling Swarms by Visual Demonstrations. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 12(2), 10. 2017
  • John Winder, Shawn Squire, Matthew Landen, Stephanie Milani, Marie desJardins. Towards Planning With Hierarchies of Learned Markov Decision Processes. Integrated Execution of Planning and Acting Workshop (IntEx-17) at the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). 2017.
  • Nakul Gopalan, Marie desJardins, Michael L Littman, James MacGlashan, Shawn Squire, Stefanie Tellex, John Winder, Lawson LS Wong. Planning with Abstract Markov Decision Processes. Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). 2017.
  • Nicholay Topin, Nicholas Haltmeyer, Shawn Squire, John Winder, Marie desJardins, James MacGlashan. Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI-15). 2015.