Finally! It is possible to program your Aisoy1 using Scratch or BYOB (Snap!). If you are not familiar with Scratch we encourage you to give it a try. It’s a free, fun, visual programming environment for creating simple programs. Scratch was created by the “Lifelong Kindergarten" team at the Massachusetts Institute of Technology. The effort had the financial support of the National Science Foundation, Microsoft, the Intel Foundation, Nokia, and the Media Lab research consortium at MIT. The fundamental goal was to create a simple programming language that would appeal to children and young people from ages 8 to 16.
Since its initial release in 2003, the project has become a revolutionary educational tool and has been involved in more than five million public projects with over three million registered users. Projects such as BYOB evolved from Scratch with the intention of making it a more powerful tool suitable for more advanced computer courses. BYOB retains the aesthetics and philosophy of Scratch but allows the creation of custom blocks that encapsulate the execution of routines.
What do we need?
Currently the Aisoy1 Airosbridge is compatible with BYOB 3.1.1 although it will be integrated with Scratch 2.0 through the course of 2014. Before continuing with these instructions, be sure your Aisoy1 is updated. Please open the botserver and confirm that the the "Rosbridge" option appears.
In addition to an Aisoy1 Raspberry Pi, you'll need a computer to run BYOB, the Chrome browser and Airos Bridge, which is a chrome app. Airos Bridge (also called Scratch Bridge) is a "helper app" based on Chrome which acts as an interface between BYOB / Scratch and your robot.
If you own several Aisoy1 bots, you can communicate to them (and between them) from the same BYOB program...imagine the types of apps you can develop! ;)
Required for this Project
Aisoy1 Raspberry Pi, with the Rosbridge section in botserver.
Browser Google Chrome
Chrome App “Airos Bridge”
First we install BYOB which is available on all popular operating systems:
- windows = http://snap.berkeley.edu/BYOB%203.1.1%20Setup.exe
- mac = http://snap.berkeley.edu/BYOB%203.1.1%20Mac.zip
- Linux Debian http://snap.berkeley.edu/byob-3.1.1-4thlinux.deb
If you want learn more about BYOB before continuing, we recommend the following resources:
Next, we will install Chrome:
From the Chrome browser, you can download the “Airos Bridge” app from the link below. You will need to open a free Google account if you do not have one:
After installing the Airos Bridge app, an Aisoy icon appears in the applications section of Chrome. However, before we launch it, we have to activate Rosbridge on Aisoy1.
Within Aisoy1’s botserver, click on Rosbridge tab:
Rosbridge needs about 1 minute to start up, so we can return to BYOB and open the Aisoy’s template that contains the blocks compatible with our bot while it’s loading.
You can download this template and other examples from here:
After the system loads this template, it will display a message indicating that the sensory motors are connected. This allows us to connect to an external application. Read more here:
Ok, now BYOB is all set to use.
Good, now BYOB is ready to use. The next step is to start the Airos Bridge application. In the address bar of Chrome, type: chrome://apps/. The browser will display the apps installed in Chrome. Click on Airos Bridge to launch it and you should see something like this:
We see that Scratch is connected, so we are on track. Since we launched Rosbridge a long time ago, it should be working now and available for connecting. To connect to Rosbridge, type the IP address of your Aisoy1 in the upper text box. If the connection is successful, a new tab will appear in the application displaying information and settings for your Aisoy1.
Try touching either side of your robot or tilting it. If things are working well you should see the result of those actions reflected in the corresponding touch or accelerometer fields.
Programming Aisoy1 using BYOB
Aisoy provides a template that includes the most common blocks used by Aisoy1v3. These support the use of motor functions (moving eyes, head, eyebrows ...), capture of events (touch, accelerometer), control of the mouth, changing the color of the heart, using TTS (text-to-speech or voice synthesis) as well as the ASR.
They are listed under the “Control” group and classified on “actions” or “events”.
“Action” Aisoy blocks
Movement of eyebrows, eyes, horizontal head rotation and vertical head tilt have a similar process. They are supported by blocks with three parameters:
Name of bot that launch the movement.
Final position of movement. The value must be between 0 and 1.
Execution time in miliseconds.
Voice synthesis (TTS)
Voice synthesis is only possible if bot is connected to a Raspberry Pi.
There are two blocks for this job, one that involves moving the mouth and one that supports speaking. The format is the same for both:
Name of bot that is going to speak.
Text to say.
Draw into LEDs matrix (Mouth)
The matrix of LEDs in the mouth can be used in two ways: writing text on it or drawing a picture (eg a smile). The block has the following format:
Name of bot that we will write on mouth.
Text or image to write
If we want to draw an image, the parameter will be a string with 70 characters, where 0 turns the LED off and x turns it on.
Change light heart
This block will change the heart color using three parameters (rgb format):
Time to change the color in milliseconds
Aisoy Blocks “Events”
Voice recognition for the Aisoy1 consist of two phases:
- Grammar scope: “Set grammar” block establishes a range of words and rules for the bot (a set of words and sentences that bot will recognize). If we establish a grammar with the words "hello" and "goodbye"; the robot will only be able to understand these words. Using grammar we can specify whole sentences too.
- Voice recognition: When robot has a vocabulary loaded, it will start to listen and capture all the words and phrases that it recognizes. The message will be in this format:
We can capture these messages using a “when I receive” block. The following picture shows the steps for activating voice recognition in BYOB:
Other Aisoy blocks
This combination of blocks is necessary for the program to operate properly.
Each block uses an Aisoy latch variable that disables the execution of new instructions when the latch is closed. When the a block with a latch is used, the latch is closed preventing the execution of other blocks. When the block that closed the latch ends the “command confirmation” block will open the latch and other blocks will be able to execute.
This mechanism allows synchronous program execution and prevents invalid simultaneous block execution.
Aisoy1 Scratch / BYOB communication
Remote Sensors Protocol
A Scratch extension that allows interaction with other Scratch programs starting with version 1.3.
When remote sensors are active, Scratch listens on port 42001 to read / write messages.
To support this functionality, Scratch uses these two blocks:
It will send a message to the chrome-app with the format:
- When I receive
This block is a Trigger that executes instructions hooked to the block when receiving a MESSAGE on port 42001.
Message protocol supported
For communication between different bots is proposed a protocol for message passing. The format for the actuator messages will be:
- BOT_NAME: The alias assigned to each bot. Currently, alias (bot1, bot2…) will be assigned in the order of input.
COMMAND: The function to be executed by bot (MUST END WITH BLANK!)
ARGUMENTS: Required arguments to support execution of the command.
Example: To change the heart color to green in 300ms.
bot1.setColorf 0 255 0 300
The supported actuators are:
TEXT: Text to write in matrix led. Text should be capitalized and spaces replaced by "_".
PICTURE: String of 70 characters where '0 'is off and' x ' on.
RED: Red value (0 to 255)
GREEN: Green value (0 to 255)
BLUE: Blue value (0 to 255)
TIME: (Optional) Time in ms to change exchange LED color
bot1.setColorf 255 0 0
bot2.setColorf 0 255 255 300
SERVO: Servo’s ID [1-4]
POSITION: Final position of servo [float between 0-1]
TIME: Time in ms to change position
bot1.moveServo 1 0.65 300
bot2.moveServo 3 1 1000
Only using ROS
TEXT: Text to say
bot1.say hello human
Note: Write two vowels to accent a syllable!
Currently, the only supported sensors are the touch sensors. If bot sensor is touched, it sends two messages: one for each activated sensor (left, right, head) and another general that indicates it is being touched. (J, it doesn’t sound/look like two messages are being sent...more?)
The general is:
Example: bot will send the next messages if it is touched at head and left side:
To create a block, you have to press the “Make a block” button in the Variables tab. We can define it after naming the block and choosing the category.
We just need to attach blocks to the created block. This routine will run when you call the block that we created.
Example 1: Write in mouth
To illustrate the building blocks we can use the block to write text created in the mouth.
This section focuses on creating a message that will be sent to the chrome-app for the implementation of the action. The message to write will have the format:
<bot name>.mouthPrint <text to write>
It will produce this message:
bot1.mouthPrint TEXT TO WRITE
The latch variable support the proper execution of actions. It is a variable that prevents the execution of actions until the previous actions have finished.
Example 2: Move eyebrows
Another example is controlling the movement of eyebrows:
This block expects the bot name (“bot1” in the image), final position of eyebrows (value between 0 and 1, in this case 0.5) and the elapsed time to perform the movement (300ms).
The first step is to check if the value of position is valid (between 0 and 1).
Then the block parameters are concatenated to create a message to be sent to the chrome-app. This block sends:
bot1.moveServo 4 0.5 300
Example 3: Blink
You can create blocks to provide functionality that may not be directly supported in the BYOB template. For example, to create a blinking eye effect (which is not included in BYOB), you can send the appropriate movement instructions to the servos.
The code below defines a “Blink” block which contains “move eyes of” blocks:
And these blocks contain Aisoy commands.