Meta introduces Embodied AI Tool, which announces the five significant updates to robotics and embodied AI systems. This also encompasses making available tools such as benchmarks and artifacts for improved modeling of and engagement with the physical environment.
Sparsh, Digit 360, and Digit Plexus, the three research artifacts developed by Meta, investigate haptic feedback, robot manipulation, and human-robot symbiosis. Meta is also launching PARTNR, a new tool for measuring planning and reasoning in human-robot interaction.
Meta Sparsh
Meta Sparsh, the first entry for the general purpose of an encoder for vision-based tactile sensing. Tactile sensors that are vision-based are available in different designs concerning their shape, illumination, and gel patterns. Previous works focus on task- or sensor-based hand-crafted representations. This is difficult to scale since real labeled data such as forces and slip, may be quite expensive to acquire.
On the other hand, Sparsh naturally operates over multiple kinds of vision-based tactile sensors for multiple tasks owing to recent progress in SSL and can be label-free. They are models that are built based on a very large set of over 460 thousand images of tactile images.
Meta Digit360
Digit 360, is a morphing, artificial finger that provides high-resolution, multi-modal haptic feedback information by digitizing touch at the level of human skin. With over 18 sensing features, Digit 360 will extend the opportunity to develop new opportunities for touch perception research in combination with the ability to turn off all sensing technologies but one to deeply investigate each sign in isolation.
Meta expects that researchers will use this device to advance AI that models the physical world and physical interactions such as object physicality, human-object interaction, and contact physics. Digit 360’s on-device AI processing greatly minimizes reaction latency to stimuli such as the flex of a tennis ball or the poke of a needle. It can serve as the peripheral nervous system on a robot based on the reflex arc happening in humans and animals.
Meta Digit Plexus
Meta Digit Plexus deals with the mechanism to incorporate the tactile sensors on a single robotic hand both in software and hardware. The platform integrates vision and skin-based tactile sensors such as Digit, Digit 360, and ReSkin with fingertips, fingers, and the palm into control boards while encoding all data to a host computer. The software integration and component hardware of the platform enable the acquisition and transmission control and analysis to be accomplished over a single cable.
GelSight Inc. will produce and sell Digit 360; they plan to make Digit 360 available to the market early next year. That will help promote the relevant research of the robotics area in a community way. Researchers in the community can directly go through the listed call for proposals on Digit 360 to get access early.
Meta publishing a reference for PARTNR, Planning And Reasoning Tasks inhum@n-Robot collaboration in hous@hold activity. The difficulty here is to train and test on the physically intricate hardware; facing actual human partners these agents are not harmless. To address this, we build PARTNR upon a high-speed, realistic simulator called Habitat 3.0 that supports both robots and humanoid avatars and will in the future allow for testing in physical-world settings focused on human-robot collaboration in homes.