Skip to main content

Skill Generalization with Verbs

Publication ,  Conference
Ma, R; Lam, L; Spiegel, BA; Ganeshan, A; Patel, R; Abbatematteo, B; Paulius, D; Tellex, S; Konidaris, G
Published in: IEEE International Conference on Intelligent Robots and Systems
January 1, 2023

It is imperative that robots can understand natural language commands issued by humans. Such commands typically contain verbs that signify what action should be performed on a given object and that are applicable to many objects. We propose a method for generalizing manipulation skills to novel objects using verbs. Our method learns a probabilistic classifier that determines whether a given object trajectory can be described by a specific verb. We show that this classifier accurately generalizes to novel object categories with an average accuracy of 76.69% across 13 object categories and 14 verbs. We then perform policy search over the object kinematics to find an object trajectory that maximizes classifier prediction for a given verb. Our method allows a robot to generate a trajectory for a novel object based on a verb, which can then be used as input to a motion planner. We show that our model can generate trajectories that are usable for executing five verb commands applied to novel instances of two different object categories on a real robot.

Duke Scholars

Published In

IEEE International Conference on Intelligent Robots and Systems

DOI

EISSN

2153-0866

ISSN

2153-0858

Publication Date

January 1, 2023

Start / End Page

5844 / 5851
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Ma, R., Lam, L., Spiegel, B. A., Ganeshan, A., Patel, R., Abbatematteo, B., … Konidaris, G. (2023). Skill Generalization with Verbs. In IEEE International Conference on Intelligent Robots and Systems (pp. 5844–5851). https://doi.org/10.1109/IROS55552.2023.10341472
Ma, R., L. Lam, B. A. Spiegel, A. Ganeshan, R. Patel, B. Abbatematteo, D. Paulius, S. Tellex, and G. Konidaris. “Skill Generalization with Verbs.” In IEEE International Conference on Intelligent Robots and Systems, 5844–51, 2023. https://doi.org/10.1109/IROS55552.2023.10341472.
Ma R, Lam L, Spiegel BA, Ganeshan A, Patel R, Abbatematteo B, et al. Skill Generalization with Verbs. In: IEEE International Conference on Intelligent Robots and Systems. 2023. p. 5844–51.
Ma, R., et al. “Skill Generalization with Verbs.” IEEE International Conference on Intelligent Robots and Systems, 2023, pp. 5844–51. Scopus, doi:10.1109/IROS55552.2023.10341472.
Ma R, Lam L, Spiegel BA, Ganeshan A, Patel R, Abbatematteo B, Paulius D, Tellex S, Konidaris G. Skill Generalization with Verbs. IEEE International Conference on Intelligent Robots and Systems. 2023. p. 5844–5851.

Published In

IEEE International Conference on Intelligent Robots and Systems

DOI

EISSN

2153-0866

ISSN

2153-0858

Publication Date

January 1, 2023

Start / End Page

5844 / 5851