Joshua Harris

Joshua Harris

Biography

Joshua Harris is a rising sophomore and Meyerhoff Scholar at the University of Maryland,
Baltimore County (UMBC), studying Computer Science with minors in Mathematics and
Entrepreneurship. Growing up in Prince George’s County, Maryland, he developed a passion for
computer science that pushed him to explore how artificial intelligence and machine learning can
be used to build secure software and innovative systems that benefit society at the intersection of
health, education, security, and integration. At MIT’s Space Enabled Group, Josh is building and
training a multimodal large language model to help NASA’s robots on the International Space
Station operate more efficiently. At UMBC, he is committed to service and building community on
campus, volunteering for UMBC’s Choice Program, as well as serving as the Parliamentarian for
UMBC’s NSBE chapter, and the Ambassador for UMBC’s Campus Connect platform. His passion,
combined with his experience, enables other students around him to excel.

Augmenting the Astrobee: Deploying Multimodal Models in Microgravity-Based
Human-Robot Interactions
Joshua Harris1, Alissa Chavalithumrong2,3, Scott Dorrington2, and Danielle Wood2,3
1Department of Computer Science and Electrical Engineering, University of Maryland –
Baltimore County
2Program in Media Arts and Sciences, Massachusetts Institute of Technology
3Department of Aeronautics and Astronautics, Massachusetts Institute of Technology

As missions to the International Space Station (ISS) progress, astronaut time remains tightly
constrained. Free-flying robots, such as NASA’s Astrobee, help optimize astronaut time by
performing routine tasks on the ISS. Current robot systems, however, lack the necessary
context-aware interactions to communicate with astronauts efficiently. Recent studies show
that multimodal models (MMMs), which process several types of data at a time, offer a path
toward more flexible interaction in microgravity. However, not enough current research
confirms or validates the feasibility of deploying a MMM on Astrobee and other free-flying
robots on the ISS. This study aims to determine the feasibility of how a MMM can be
integrated into Astrobee to optimize microgravity-based human-robot interactions on the ISS.
To conduct this study, a formal system architecture was designed to showcase the process of a
MMM being integrated into Astrobee’s internal software, using NASA JPL’s ROSA agent and
a Large-Language Model to build the model. A prototype model is still in the process of being
developed, and it will be tested in simulation using a Linux environment and RViz software. If
successful, this research provides the first concrete assessment of applying a multimodal model
to increase overall efficiency in microgravity-based human-robot interactions.