00:00This demo, we have built a mixed reality experience with.
00:12So in this mixed reality interface, what we get to see
00:15is what the robot is thinking.
00:17I'm going to say, let's clean up the living room.
00:23So now it's thinking what it should
00:25do about the instruction I just gave it.
00:28And the first action it decides to do
00:31is explore the living room.
00:40So what the robot is doing right now
00:42is that it's trying to look for objects
00:45on this side of the house.
00:46To look for objects, it's using the RGB and depth
00:49camera in its gripper.
00:52It turns over, starts looking at the coffee table,
00:55and now does the same thing.
00:57Now, around the coffee table and this bench,
00:59it's looking if there are other plush toys.
01:22So it should not hit the chair when it's trying
01:24to place the object, ideally.
01:27It should find a location on the dining table
01:29that is the correct height and correct width
01:32for this pineapple plush toy.
01:49It's using its cameras on its hand
01:52to find a location that is good for placing the object.
01:56And then it's going to move its arm and base
02:00so that the donut plush toy is nicely placed on the table.
02:07OK, all right.
02:09So this is essentially a glimpse into a future
02:14where we can give very high level instructions
02:16to our robots that are able to perceive their environment,
02:22adapt to changes in this environment,
02:25and then do really long horizon tasks robustly
02:28and improve our everyday lives.
02:31Our machine learning systems or robots
02:34were able to do simple tasks, but they were not
02:37able to semantically generalize to more complex tasks.
02:42And now with more powerful AI systems, what we are able to do
02:45is a much more natural way of interacting with robots.
02:49How I would interact with you, I can also
02:50interact with this robot now.
02:53You and I can't live without our phones.
02:55I think AI and robots are going to be
02:58this for the next generation.
02:59That their everyday life is entwined with AI models
03:06of one type or another.
Comments