Joshua Harris

Joshua Harris

Biography

Joshua Harris is a rising sophomore and Meyerhoff Scholar at the University of Maryland, Baltimore County (UMBC), studying Computer Science with minors in Mathematics and Entrepreneurship. Growing up in Prince George’s County, Maryland, he developed a passion for computer science that pushed him to explore how artificial intelligence and machine learning can be used to build secure software and innovative systems that benefit society at the intersection of health, education, security, and integration. At MIT’s Space Enabled Group, Josh is building and training a multimodal large language model to help NASA’s robots on the International Space Station operate more efficiently. At UMBC, he is committed to service and building community on campus, volunteering for UMBC’s Choice Program, as well as serving as the Parliamentarian for UMBC’s NSBE chapter, and the Ambassador for UMBC’s Campus Connect platform. His passion, combined with his experience, enables other students around him to excel.

Augmenting the Astrobee: Deploying Multimodal Models in Microgravity-Based
Human-Robot Interactions
Joshua Harris¹, Alissa Chavalithumrong^2,3, Scott Dorrington², and Danielle Wood^2,3
1Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County
²Program in Media Arts and Sciences, Massachusetts Institute of Technology
³Department of Aeronautics and Astronautics, Massachusetts Institute of Technology

As missions to the International Space Station (ISS) progress, astronaut time remains tightly constrained. Free-flying robots, such as NASA’s Astrobee, help optimize astronaut time by performing routine tasks on the ISS. Current robot systems, however, lack the necessary context-aware interactions to communicate with astronauts efficiently. Recent studies show that multimodal models (MMMs), which process several types of data at a time, offer a path toward more flexible interaction in microgravity. However, not enough current research confirms or validates the feasibility of deploying a MMM on Astrobee and other free-flying robots on the ISS. This study aims to determine the feasibility of how a MMM can be integrated into Astrobee to optimize microgravity-based human-robot interactions on the ISS. To conduct this study, a formal system architecture was designed to showcase the process of a MMM being integrated into Astrobee’s internal software, using NASA JPL’s ROSA agent and a Large-Language Model to build the model. A prototype model is still in the process of being developed, and it will be tested in simulation using a Linux environment and RViz software. If successful, this research provides the first concrete assessment of applying a multimodal model to increase overall efficiency in microgravity-based human-robot interactions.