Gesture recognition technology is all around us, from the controllers we use in video games to the smart devices that respond to a wave of our hand. This project will let you explore the same principles used in robotics and automation, where machines are programmed to understand and respond to human actions, making technology more intuitive and accessible.

In this project, you’ll use CreateAI to train a micro:bit to recognize three human actions: waving, cheering, and sitting still. Then, you’ll build a robot friend with the Climate Action Kit and program it to respond to your actions in real-time.

Prepare

You’ll need the following items to complete this project:

Build

First, build your robot friend (steps 1-26). Then, attach the second micro:bit to CHARGE (step 27).

Train & Code

In this project, you will need to code two separate micro:bits to communicate via radio. The micro:bit attached to CHARGE will be the sender, while the micro:bit attached to the robot will be the receiver.

  • Sender
  • Receiver

Use CreateAI to train the micro:bit attached to CHARGE to recognize three different actions: sitting, waving, and cheering.

Screenshot of CreateAI model trained to recognize sitting, waving, and cheering. Each action has 8 data samples.

Then, edit the model in MakeCode to send out a unique radio signals based on the action detected.

Sample code that reads: On start, radio set group 3. on ML sitting start, show neutral icon, radio send string "sit", pause 2 seconds on ML waving start, show smiley icon, radio send string "wave", pause 2 seconds on ML cheering start, show yelling icon, radio send string "cheer", pause 2 seconds.
Download Model
CreateAI

Connect the micro:bit attached to your robot to the computer. Then, download this starter code.

Review the code comments on each block to learn more about how your robot friend responds to your actions.

Improve

Think about additional features you could add to your project! How would you modify the sample model and code provided, or create your own that includes the following features?

  • Sound Effects: Can you add music or other sound effects for each action?
  • New Gesture: Can you train your robot to respond to a fourth gesture like a ‘high five’ or ‘dancing’?
  • New Output: Can you swap one of the positional servo motors for a continuous servo motor to create a new output or response for your robot friend?

Feeling Stuck? Explore the Knowledge Base for common questions and troubleshooting tips.

Knowledge Base