TY - GEN
T1 - Word2vec to behavior
T2 - 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
AU - Matthews, David
AU - Kriegman, Sam
AU - Cappelle, Collin
AU - Bongard, Josh
N1 - Funding Information:
ACKNOWLEDGEMENTS The authors would like to thank Eve Wight and Ryan Joseph for their help in creating the physical robot. This work was supported by NSF award EFRI-1830870 and DARPA contract HR0011-18-2-0022. Computation was provided by the Vermont Advanced Computing Core.
Publisher Copyright:
© 2019 IEEE.
PY - 2019/11
Y1 - 2019/11
N2 - Enabling machines to respond appropriately to natural language commands could greatly expand the number of people to whom they could be of service. Recently, advances in neural network-trained word embeddings have empowered non-embodied text-processing algorithms, and suggest they could be of similar utility for embodied machines. Here we introduce a method that does so by training robots to act similarly to semantically-similar word 2vec encoded commands. We show that this enables them to act appropriately, after training, to previously-unheard commands. Finally, we show that inducing such an alignment between motoric and linguistic similarities can be facilitated or hindered by the mechanical structure of the robot. This points to future, large scale methods that find and exploit relationships between action, language, and robot structure.
AB - Enabling machines to respond appropriately to natural language commands could greatly expand the number of people to whom they could be of service. Recently, advances in neural network-trained word embeddings have empowered non-embodied text-processing algorithms, and suggest they could be of similar utility for embodied machines. Here we introduce a method that does so by training robots to act similarly to semantically-similar word 2vec encoded commands. We show that this enables them to act appropriately, after training, to previously-unheard commands. Finally, we show that inducing such an alignment between motoric and linguistic similarities can be facilitated or hindered by the mechanical structure of the robot. This points to future, large scale methods that find and exploit relationships between action, language, and robot structure.
UR - http://www.scopus.com/inward/record.url?scp=85081161089&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85081161089&partnerID=8YFLogxK
U2 - 10.1109/IROS40897.2019.8967639
DO - 10.1109/IROS40897.2019.8967639
M3 - Conference contribution
AN - SCOPUS:85081161089
T3 - IEEE International Conference on Intelligent Robots and Systems
SP - 4153
EP - 4160
BT - 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 3 November 2019 through 8 November 2019
ER -