Chapter 1: How Robots See and Talk
How Robots See and Talk 📸🗣️
Section titled “How Robots See and Talk 📸🗣️”Pictures Are Made of Numbers
Section titled “Pictures Are Made of Numbers”When you look at a picture on a tablet, you see colors, shapes, and maybe even your favorite character. You see smiles, trees, pets, and people. But a robot does not see the way you do. You have eyes and a brain that help you understand what you are looking at. A robot has a camera, but it does not truly “see” a dog or a tree. Instead, it sees something very different.
Every picture on a screen is made of tiny dots called pixels pixels: Very small squares that make up a picture on a screen. . A pixel is a very small square. If you could zoom in very close to your screen, you would see that the picture is really made of many tiny squares lined up in rows and columns. Each of those squares has a number. The number tells the computer how bright or colorful that square should be. Some numbers make a square dark. Some numbers make it bright. Some numbers make it red, blue, green, or another color.
When all the little squares are placed together, they form a picture. To you, it looks like a smiling face. To a robot, it is a big grid grid: A pattern of squares or numbers arranged in rows and columns. of numbers. The robot stores that picture as a list of numbers arranged in rows and columns. That is how robots “see.” They do not see with understanding. They see with math.
If you color a picture by number, you are doing something very similar to what a computer does. When you follow the numbers and color the correct squares, a secret image appears. The robot stores that same image as numbers in a grid. Pictures are really patterns of numbers.
Listening and Speaking Clearly
Section titled “Listening and Speaking Clearly”Robots can also “hear,” but they do not have ears like you do. They use microphones microphones: A tool robots use to catch sound waves from the air. to catch sound. When you speak, your voice makes sound waves. Those waves travel through the air. A robot measures those waves and turns them into numbers, just like it does with pictures.
When you say a word like “cat,” you are really saying small sounds: “c–a–t.” If you change one sound and say “cap,” the word changes. Robots must learn to notice these tiny sound differences. They listen for patterns in the sound waves and try to match them to words they have been taught.
But robots do not have something very important that you have. They do not have common sense common sense: Understanding how the world works, which helps you know what might happen next. . Common sense helps you understand how the world works. You know that if you drop a glass, it might break. You know that if someone is crying, they may feel sad. A robot does not automatically know these things. It only knows what it has been taught with data and instructions.
That is why we must speak clearly to robots. If you say, “Pick it up,” a robot might not know what “it” is. You have to say exactly what you mean, like, “Pick up the red book on the table.” If you say, “It’s raining cats and dogs,” you know that means it is raining very hard. A robot might get confused because no real cats or dogs are falling from the sky. Robots need clear words and clear instructions because they do not understand stories the way people do.
Bip was a little robot with a shiny metal head and two camera lenses for eyes. One day, Leo showed Bip a drawing.
“Look, Bip! It’s a cat!” Leo said.
Bip’s camera eyes zoomed in. Click-whirrr. Bip didn’t feel fuzzy fur or hear a meow. Bip saw something else. Inside Bip’s computer brain, the picture turned into a giant grid of tiny squares called pixels.
“I see numbers,” Bip beeped. “I see a grid. I see bright orange squares and dark black squares.”
“No, Bip,” giggled Leo. “Use your sensors! Look at the shape. It has pointy ears and whiskers.”
Bip scanned the pixels again. It looked for the pattern of triangles for ears. “Processing…” Bip said. “Pattern found! It is a cat.”
Bip learned that robots don’t see pictures like people do—they see math and numbers to figure out what is in the world.
Think About It! 🤔
Section titled “Think About It! 🤔”- How is the way you see a picture different from the way a robot “sees” a picture?
- Why do you think robots need clear instructions instead of phrases like “it’s raining cats and dogs”?
- Can you think of a time when you had to explain something very clearly to someone? What made it hard?