Building JARVIS: Towards Real-time Contextual Planning for Live Instructional Support for Physical Tasks

Our long-term research goal is to develop intelligent systems that can support human learning. We are particularly interested in developing an approach to apprenticeship learning which occurs during the physical context of task execution and is known to be very effective in learning procedural tasks such as equipment maintenance or artifact assembly. We describe our initial progress in building such system - JARVIS - that leverage real-time computer vision, high-level inference, and augmented reality technology to monitor and support human task learning through apprenticeship.

Citation

Mohan, S.; Ramea, K.; Price, R.; Shreve, M.; Eldardiry, H.; Nelson, L. Building JARVIS: Towards Real-time Contextual Planning for Live Instructional Support for Physical Tasks. AAAI Workshops.