About

Hello, I'm John Winder. I research artificial intelligence and machine learning for complex real-world systems. I work as a senior staff scientist at the Johns Hopkins University Applied Physics Laboratory (APL).

I received my Ph.D. in Computer Science from UMBC, where I specialized in reinforcement learning (RL). I focused on developing state abstractions for hierarchical RL and probabilistic planning. I had two excellent doctoral advisors, Marie desJardins and Cynthia Matuszek.

My main interest and objective is the creation of decision-making agents that generalize and reason about long-term goals under uncertainty, collaborating with humans and other AI agents, while operating in dynamic and open environments, facing new challenges in diverse scenarios.

News

• 26 JUL 2023 »
  I gave a talk on Beyond Human Reasoning, my team's multi-year effort to develop an AI co-pilot, at APL's 2023 XR Symposium.
• 03 NOV 2022 »
  I just finished meta-reviews for AAAI 2023. This time was my first serving in the senior program committee, excited to see the pace of research continue to pick up.
• 16 AUG 2022 »
  Congrats to Adam Berlier and the IRAL Lab on their paper "Augmenting Simulation Data with Sensor Effects for Improved Domain Transfer" being accepted at the European Conference on Computer Vision (ECCV) Workshop on Assistive Computer Vision and Robotics (ACVR).
• 16 JUL 2022 »
  APL colleague Josh Bertram has successfully defended his dissertation on FastMDP, an extremely efficient solution for complex multi-agent planning problems. Congratulations, Dr. Bertram!
• 01 AUG 2021 »
  Our paper on the GoLD dataset for grounded learning of spoken language descriptions has been accepted at NeurIPS 2021. Congrats to the IRAL lab!
• 17 FEB 2021 »
  I've been promoted to Section Supervisor for the Advanced Artificial Intelligence Algorithms section at APL, working on RL and autonomous systems. We're hiring!
• 13 JUL 2020 »
  I joined APL as a Senior Professional Staff Scientist researching RL for intelligent platforms.
• 09 JUL 2020 »
  Our National Robotics Initiative proposal was awarded a three year grant! The IRAL lab will be studying grounded language learning and concept-based knowledge transfer in deep RL for collaborative robots.
• 11 NOV 2019 »
  Our paper on abstract model-based RL has been accepted to AAAI 2020. See you there in February!
• 03 OCT 2019 »
  I'm presenting our research on learning abstract models at the Do Good Robotics Symposium, discussing how our work can be used in domestic service robots.
• 01 SEP 2019 »
  I've been hired as a faculty research assistant at UMBC in the Interactive Robotics and Language Lab (IRAL), advising student research groups working in human-robot interaction and concept formation for RL.
• 20 JUN 2019 »
  I successfully defended my dissertation and passed my final exam! I'll be graduating in August.
• 10 MAY 2019 »
  My paper The Expected-Length Model of Options has been accepted at IJCAI-19, joint work with Dave Abel and our advisors.
• 11 AUG 2018 »
  Our NSF IIS proposal on concept formation in POMDPs was accepted.
• 03 AUG 2018 »
  I'm joining the IRAL lab at UMBC.

Selected Publications

• John Winder, Stephanie Milani, Matthew Landen, Erebus Oh, Shane Parr, Shawn Squire, Marie desJardins, and Cynthia Matuszek. Planning with Abstract Learned Models While Learning Transferable Subtasks. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20). 2020.
• David Abel*, John Winder*, Marie desJardins, Michael L. Littman. The Expected-Length Model of Options. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19) [*equal contribution]. 2019.
• John Winder, Marie desJardins. Concept-Aware Feature Extraction for Knowledge Transfer in Reinforcement Learning. Knowledge Extraction from Games (KEG-18) Workshop at the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). 2018.
• Karan K Budhraja, John Winder, Tim Oates. Feature Construction for Controlling Swarms by Visual Demonstrations. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 12(2), 10. 2017
• John Winder, Shawn Squire, Matthew Landen, Stephanie Milani, Marie desJardins. Towards Planning With Hierarchies of Learned Markov Decision Processes. Integrated Execution of Planning and Acting Workshop (IntEx-17) at the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). 2017.
• Nakul Gopalan, Marie desJardins, Michael L Littman, James MacGlashan, Shawn Squire, Stefanie Tellex, John Winder, Lawson LS Wong. Planning with Abstract Markov Decision Processes. Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17). 2017.
• Nicholay Topin, Nicholas Haltmeyer, Shawn Squire, John Winder, Marie desJardins, James MacGlashan. Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI-15). 2015.