12Blocks: A Simple Block Language that lets Kids Program Sophisticated Multi-core Robots

12Blocks: A Simple Block Language that lets Kids Program Sophisticated Multi-core Robots

>> BRAKES: Hello, my name is Charles Brakes,
and I brought Hanno in here to talk of, little bit more about some of his latest work with
the robots. He’s emphasizing robots in kids this time around, and he currently lives in
New Zealand and he’s sponsoring a variety of programs to get kids more interested in
electronics and robotics. So without further ado, here’s Hanno.
>>SANDER: Thank you, Charles, and thank you, Google, for having me back here. I was here
exactly a year ago, talking about my dance bots, which is a balancing robot, dances with
kids using computer vision. And this year, I’ll talk about 12Blocks which is a simple
block language–that’s kids program, sophisticated multi-core robots, very easily. So, first,
I’ll talk about a little bit, New Zealand. So, I moved there five years ago, now, and
there’s a bit more than sheep down there. I contacted a couple of local companies in
Christchurch, New Zealand, which is where I live. And so, this is what my neighbors
and friends are up to. Glenn Martin is building the first practical jet pack. A hundred thousand
dollars buys you a back pack you strap on, and let’s you fly to the air, thousands of
feet up for about 30 minutes, and it was shown at the last Oshkosh very successfully. And
Yike Bike could fit in really well here at Google. With the segways, it’s a very light
folding bike, fits in the back, strap it on, and there you go. A friend of mine is doing
a GPS Boomerang, so it flies up to edge of space with a Helium balloon and returns with
a GPS sensor right to about three hours later, hits you on the top of your head, and gets
you your sensor readings, and lots of telemetry information about what’s going on in the atmosphere.
Another friend of mine is doing the first color X-rays, so they developed a novel sensor
for CERN, and that’s currently being used at the Mayo Clinic to grab color images from
X-rays. So, they’re still using the same X-ray emitter but they’re using a special detector
to count actual photons looking at their energy level, and with that, getting the composition
of what the X-ray went through. Another guy is doing WhisperGen and that’s the four-cylinder
Stirling engine which provides residential heat and power and it’s currently in trials
in Europe. And, another guy is doing a PivotalEngine, so, instead of a piston going up and down
in an engine, this has a pivot where the explosion happens in the same chamber but it’s stationary
on one side, so it’s pivoting. And so with this, you get very high power density, it’s
quiet, and it can run on hydrogen since you able to water cool the whole engine and the
piston. And, there’s another company called Jade and I searched that it’s a very high
speed database. So, I have to keep up with these other guys and so I’ve been busy too.
I’m more busy on the hobby and electronics and getting kids into robotic side, and sought–take
a minute to talk about what I’ve done there. And, so I wrote a book with some other co-authors.
We’re all using the Parallax Propeller, it’s a high-speed, multi-core semi-conductor chip
that lets you do a lot of the things that I’ll talk about here today. So, it has memory,
it has eight processors running at 80 megahertz and it has a lot of input and output pins
to let you interface with sensors and actuators and motors. So, this is a five hundred-page
book, and I wrote about my balancing robot that I talked about here last year. I also
developed a PropScope. So, this is a $250 little blue box that reads analog signals
on one side and connects to your computer with USB and it also can be used as a function
generator, a spectrum analyzer, a logic state analyzer. So basically, everything that you
did in college with a big lab full of instruments is now in a little box that you can take home
and look at synops. Okay? And I’ve also developed a tool called ViewPort, that lets you develop
very sophisticated projects to build on this Parallox chip, and I’ll show that a little
bit on my demos. And, that includes very graphical debugging tools, so, it includes the standard
break point, and call stack, and memory map, and stepping in and out of syntax, but it
also has simulated instruments like in the celloscope, the logic analyzer that let you
look at what’s going on in your system. So, if you’ve ever wondered why your program isn’t
running the way it is, but it should, with this, you can go back in time, so you can
see about ten minutes ago, this is what happened. This is what the sensors were reading, this
is what your mathematical model was doing and that it helps you develop more complex,
more sophisticated and edit systems. Integrated in here is open cd, so a complete computer
vision library’s in there, as well as physiologic tools, so, that’s all in there. And finally,
12Blocks. So 12Blocks is my answer to my daughter when she ask me, “What are you doing?”, or,
“You’re always programming, you’re always on the computer. Can I try? Can I do something?”
She’s five years old and I didn’t think it would be a good idea giving her a C Block
and showing her, and teaching in her the complete language. At the time, she was playing with
Legos and putting them together, stacking them. There are other software out there,
MIT Scratch developed the wonderful, graphical tool. Lego Mindstorm also has a block-based
interface, but there’s a lot of problems with those and I tried to come up with the solution
to those in 12Blocks. And, 12Blocks is also a language that is completely focused on embedded
systems, so it’s a visual language, but it’s developed for robots, for embedded systems
that have actuators, that have sensors and go from there. So, I called it 12Blocks because
when everything is contained in a block, so very object-oriented, then it’s very easy
to build very sophisticated things in a small number of blocks. You’re no longer looking
at pages and pages of code. All the demos that I’ll show you today are in a handful
of blocks. And there are many blocks available, so there is a library; you can create your
own blocks and there’s about a hundred plus blocks. I’ll show you sampling of those a
little bit later. And I like to say that it has a very gentle learning curve, but it reaches
very high. So you can start out using this when you’re still in elementary school and
when you’re still just dragging a couple of blocks together. But you can keep using it
to University beyond because it’s very functional, it’s very sophisticated and there’s a lot
of power there. It has live edit capabilities so you can keep editing your program while
it’s running, I’ll show that. Graphical debugging, which lets you look at graphs instead of looking
at variables and looking at lines of code. And it’s integrated with a Wiki, so with an
Internet, you can look at videos, you can look at schematics, share everything with
a Wiki; it’s all integrated there. And, you can create stand-alone programs, so, similar
to how I built the PropScope by developing some firm ware and building a PC-user interface
that lets you, with 12Blocks, develop both the firm ware running on an embedded system
but also the user interface that other users that don’t have access to 12Blocks can use
to run the program. And I mentioned you can customize your own blocks and it’s very easy
to transition to other languages. Parallax, the company behind the propeller, developed
an object-oriented language called SPIN and it’s a very nice, very powerful language for
writing textural code for the propeller. But there’s also NCEC available. So, if you’re
used to C, you can program these types of robots in C as well and you can transition
from 12Blocks all the way to writing C code and going from there. So, I won’t read all
these off, but there is plenty of capability. There’s a standard computer science where
you have arrays, and you have state machines and events and messages, strings, user interface,
graphic capability; so plugging it into a TV and then building a graphical aim with
sprites and vector graphics, sound; so type in your own score or sound effects, play midi
files as well as reading and using actuators and sensors; so looking at a proximity sensor,
looking at a will encoder, all that sort of stuff that you typically do with robots with
embedded systems is always just a block where you drag the block somewhere connected and
away you go. So, the robot that I’ll demo today was built by Chad, and it’s a very sophisticated
platform that fits in the hand, so it’s light, small, robust, made out of metal. You can
screw a mecano and vex components to this as well as plug in Lego pieces. So, again,
going from very simple, basic Lego-type expandability to really doing a hardware expansion and,
this is really designed to get you started with robots at the simple level so driving
it like a remote control car where you’re not programming at all, but also lets you
do things like Logo. I started with Logo in elementary school where you have a turtle
and you tell it to move forward, and then turn right, and you repeat for a while, and
then, you move up to using sensors. So, some of these sensors that I talked about and integrating
that to control what the robot’s doing. And, all of these is done with 12Blocks, where
you visually program it. And, I’ll show how you can transition to the text-base languages
as well. So what I’ll do with my demos is, I’ll use 12Blocks, which has its library,
which has a worksheet where I assemble programs, and then has views. And within the view, I
can look at what’s going on in the robot. And on the TBot’s side, there’s shared memory
that’s shared between the PC and between the robot, so both have access to it. It’s actually
in the TBot but I can read and write it from the PC. And later, I’ll show how that same
concept of shared memory can be extended when you have multiple robots. So, again, there
is memory on each TBot, but the TBots and 12Blocks can read and write that memory remotely.
And on the TBot’s side, that memory controls how the motor moves, how the speaker sounds,
what the microphone is doing. So it’s all a very simple architecture that is very functional.
I keep saying that–I keep focusing on this benefit of being able to easily transition
from grade school to very advanced things. But now, there’s a lot of robots out there
that are targeted at one specific item and one specific challenge, one specific competition,
and I think that this lets people use the same platform with expanding it either through
software, through hardware, through electronically changing things. And on the software side,
starting with 12Blocks but then growing into Spin and MTC code at the end. Little Ts for
the advanced people, so what I’ll show at the end is using wireless to compete, to connect
multiple robots, and using sensor information from one robot to control another robot while
monitoring the whole system on the PC and this can be extended to programming remotely,
wirelessly, as well as connecting to Internet, so causing the robot to move or to play a
sound of something on the Internet, a stock of whether it changes, as well as sending
data measurements from the robot if it’s video, if it’s sensor reading, sending that back-up
to the Internet. And of course, Internet mobile devices so Bond movies always have a scene
where you take a robot and you drive a car with a iPhone or Android, so that’s all possible
here to. So, demo-wise, these are some things that people could do right of the box with
us where they start with remote controlling a car. This is what my daughter would do starting
out with or even my son. Then moving on to drawing logo shapes, moving up to integrating
sensors, and using those to control the robot. Then, line follow is a very common challenge
where it is a black line on the white paper and you try to stay on the line, go as quickly
as possible. Communicating with a shared memory, mail terminal, so a very functional communication
networking protocol for sending data back and forth, and then trying C code and Spin
code. So, moving on to demos–I’ll clear little bit of an area here. Here’s a robot and I
will switch over to 12Blocks. So, 12Blocks is a Windows application and it has this interface.
I can start a new program and as I mentioned, it has this library section where you have
multiple types of blocks to let you do graphic manipulation, to let you do sensor measurements
and even to build user interfaces. And here in the middle is a work sheet and for this
first program, I’ll do what the simplest thing that we can do with the robot just to have
it move a little bit. And so, we start with a start block, so I’m just dragging it from
the library onto the worksheet and I take a block from the TBot section, the move block
and it shows me that these are able to connect, of course, I cannot connect it to the top,
and I can’t connect it to the library, so this is the one place where I can connect
it. And now, I can hit the run button and this will load the program into the robot
and cause it to move backwards for one second. So, it’s a very subtle program but it’s something
where I’ve just taken two blocks, connected them and right away it did something. So,
next demo that I’ll show is doing some basic input, output. So, there’s a proximity sensor
on here, there is line sensors and there’s will encoders. What we’ll do in this demo
is look at the proximity sensor to see if there’s something in front of it. And there’s
a full color LED on there and right now, it’s showing a blue light so, I’ll be able to control
the color of that light. So, I’ll take away the move block, I’ll just throw it back into
the library and I have a start and what I’ll do is I’ll set the LED to a color and I will
get the proximity sensor value. And this is a specially shaped block which fits into another
block. So, here is a variable assignment block. So, set x to one, that’s how it starts out
but I can drag this and dock it in there. So, now I have, three–four blocks. A start
block, a block that sets the color of LED and a block which sets x, the variable x,
to the value of a sensor, just to keep this going, I’m going to use a repeat block. So,
this is a loop and I’ll dock these inside of the repeat block. So, this is pretty straight
forward and I’ll hit the run button and the LED starts out red. You can see its red there
and I mentioned that I can edit things as its running so I can click here on the red
and pull down from these choices and select a different color. And, so now I have a blue
light shining on the robot, and so I can do that with all of these items that are in yellow.
These are parameters and these are changeable while the program’s running. The value of
x is supposedly being set to the value of the proximity sensor. Typically, with embedded
systems, we have this problem where it’s in the processor but there’s no way of looking
into it. So typically, what you do is you hook up also with some other gadgets to it.
The nice thing here is that we’re connected to the PC and on the PC, I have an interface
for looking at these values. So, here is x and here is a value of x. And, it’s right
now at a certain level and if I–if I put my hand in front of it I can change that value.
And, I can plot this over time so right now, I’m plotting just the X value and I can plot
this over seconds there. And–so, I can look at the value as it changes overtime. What
I’ll do in this case is move on to—-showing how to build a user interface based on this.
So, instead of having to go in here, and clicking these values and changing from blue to purple
or some other color, I’m going to build a simple user interface for that. So, I’ll go
into the user interface section, and there’s a background, and I can use a textbox, and
just drag that on top of there, and this text box will set the variable end to whatever
is type into it. So, instead of setting the LAD to the color purple, I’ll set the LAD
to the number end. And now when I run the program again, I have—-I can type into this
number, into this textbox, a different number so zero is red and now when I type a 120 into
here, it changes the LAD to a different color. I can save these 12Block files, so I’ll just
give a file name, and I can also save it as a stand-alone executable. So, I can save as
and in here I can call it Google, and in this case, a Google EXE file. And, I can hit save
and that file is now saved to this directory, so I will go there–12Blocks tutorials. And
here is a new file, just created right now, 254. I will close 12Blocks so I’ve now given
this to a colleague of mine and they want to change the LAD on my robot, and so, they
could just run this megabyte file, and what this first does is it loads the firmware to
the robot, and then it gives me this little user interface that I’ve designed in 12Blocks.
So, it just shows me N equals this, I could type a value in here, so 120 is what I have
last time, and then it changes the color of that LAD. So, it’s building the firmware inside
of 12Blocks as well as a user interface to work with the sensors, work with actuators,
and bundling all of that into an executable that you can then run somewhere else. So,
that’s with 12Blocks, and with 12Blocks we can do more. I’ll now move into transitioning
from the 12Block visual language to Spin. So, when you’re more advanced or when you
want to go into building your own blocks, or when you want to use a more powerful debugger,
then you can do that with the text language. So, I’ll take the same code that I had before.
So opening up 12Blocks again. Here we go. Okay. And, I’ll take our Google One that we
just had. And, what I’ll do now is just like you do on the Internet, you have an Internet
browser, you have view code button, and so you have right here. And what view code does
is bring up a tool, it’s called ViewPort, that has a syntax highlighting editor in there
as well as lots of graphical tools to let you debug that program. And inside of here,
I have the Spin equivalent code of my program so you can see down here that it’s doing a
repeat loop and it setting the HSL to end. And in here, I can make changes, I can do
that. But first thing—-what I’ll do is, I’ll just load this program, so from a different
environment, I’m now loading this to the robot. And, I have a tool like the DSO viewer, so
in here I can look into the values or I can change a value, so I get a textbox again for
setting the color of my LAD, so it started with zero and 120 is a nice color blue. So,
I can do something like that as well as graphing that over time and so, is that and an overtime,
it goes down. And, what’s also in here is I can set break points, so here is 989, and
I can now compile this with debugging information, and load this to the propeller to the robot,
and this will now stop on that break point and give me all sorts of debugging tools for
this. So, I can look at the files, I can look at the code but I also have a memory map of
all the information that’s in the propeller as well as a watch list of variables, a call
stack, a profiler, seeing word that time is being spent, a view of the pins the sort of
actual hardware interface of what’s going on with the propeller. And I have a command
interpreter so I can for example take 10 steps and it’ll take 10 steps in there. I can take
one step at a time, so sort of going between those two. So, the traditional visual studio-type
of debugging is integrated in here. Okay. And the last thing that I’ll show here is
that I can make changes in here. So, I can change the intensity of the LAD so I’ll make
it very dim, I’ll set it to five. And I can save this file and now when I go back to 12Blocks,
it says, “Do you want to reload it?” So, it notice that the file that I was viewing had
change and now I can say yes to this, and then the values are changed, and I can go
from there. Next demo. So I can transition to C. So, in the same ViewPort tool that I
showed earlier, I can look at a C project and this is the standard code blocks project
file. So, I can change the “Compile to” options, the “Link for” options, Environment options.
And, I can look at the standard NCC that I can compile for the same robot. And this–this
is very simple, it blinks the back LAD on here, so, there’s a little LAD. Right now,
it’s blinking very, very fast but also for the NAC part, there is a compiler–a debugger.
And so, I can compile up with debugging information. And now, in the C compiler–C debugger, I
can send a break point and run up to that point and keep stepping around here. And now,
you can see the LAD is blinking every time I go pass that instruction. So far, we’ve
seen at the very high level using a visual programming language and using–and interfacing
with the sensors, a higher level, which is the Spin language, and then I’ll going all
the way down to C code, and stepping line at a time to blink a LAD. Okay. Okay. Now,
shared memory requires two robots, so I’ll have two robots running here. And, what I
can do here is I can open a single program. So, this is a slightly longer program but
it’s still usually fits on the page. And what this program does is it runs on two robots
and uses a shared memory structure that I can monitor with 12Blocks to set the LAD color
on one robot to the proximity sensor reading of the other robot. So, I’m going to change
the proximity value on one to change the color on the other. And I need to load this into
both robots, so I’ll start by loading into one of them. Let’s see which one. First, I
have to…okay. Both robots are on. So, this robot on the right now, I can look
at what the sensor reading is doing here. And what’s–what this one is doing is in I3,
it’s putting the value of the proximity sensor. So, you can see that it’s going from 30ish,
when I’m close to it to a 180 or so when I’m further away, you out-graph it so we can look
at it. So, you can see on the graph that as I’m moving my hand back and forth, that’s
the sensor value, is changing. And this is–the sensor reading is put into the shared memory
of I3. So, I3 is where this one is putting its reading. I can now connect to the other
robot. That was–and load this robot with the same program, but that robot is looking
at I3 to set its LAD and storing its sensor reading into I0. Okay. So, look at both I0
and I3. And now, if you look at the other robots as I’m moving the–I saw them, you
know, disconnect the–so, I make this one truly wireless, and so, what’s going on is
it’s taking readings with the proximity sensor, and I can look at those readings on my graph
over here. So, you’ll see the red line goes up and down as I’m moving my hand back and
forth over here, and the other robot is receiving that information, and using that to change
its state. So, we’re taking readings from one and affecting the behavior of the other,
and it goes both ways so I can change the reading over here to change the LAD over here.
Okay. So, that’s what I was here to show, 12Blocks going all the way from starting a
program, adding a move block, moving the robot all the way to using shared memory to control
a robot remotely with wireless and looking at the result on the PC. Any questions?
So, I noticed your blocks was–looked a lot like a flow chart rather than a dataflow program.
Did you think about a data flow model at all?>>So, there are many different visual ways
of creating a program. This one is still somewhat linear, so it starts at the top and goes down.
It is in two-dimensions so if–so some of these programs that I have or I can sort of
show you so, of course, you have two dimensions and so you can put it together. This is what
I thought would be the easiest for a beginner to get started with reading from top to bottom
and naturally reading what’s going on. So…>>So, I have a couple of questions. One is–I
know this is very different from the Arduino microcontroller but could you say any comments
on how this compares? And with your programming language, could that be ported to the Arduino?
>>SANDER: Yeah, okay. So, yes, at the moment, everything I showed here used the Parallax
Propeller multi-core processor. The nice thing with that architecture is that it has eight
processors in that chip and so I’m using one of those processors to establish the shared
memory connection. The connection is just a serial interface and there is a documented
protocol of how that information is shared between the two. I currently–so, on this
device list over here, I currently support several robots as well as a simulator, as
well as Java for the Lego Mindstorm NXT. So, already I’m taking this visual programming
language and outputting different codes depending on what device you want to interface with.
So, if you want to interface with one of these robots, then it would output the Spin code
that runs natively on the device. If you want to run in a Python simulator then the same
program that you have here on screen, can be saved and run inside of a Python environment,
or it can be saved as Java code. So, you’re just mapping blocks to the native language
that you want to support. So, in terms of Arduino support, I don’t have it right now,
but the interface–that communication link is serial so that’s straight forward. And,
instead of outputting whatever commands I use here to move the robot back and forth,
I would have to or someone else would have to write the library to do that and map that
back to these blocks.>>Okay. And I notice that you have primitives
like setLED, which is different from your set, you know, variable. Is that setLED–how
is that tied to the hardware and…?>>SANDER: So, there is a block manager under
Tools and what this lets you do is map the blocks and what they look like and how many
children they have. So if you’re looking at nested items like an F or repeat and it lets
you map those to your language that you want to support, so if you want to go to spin code
what you have is you have object files that you include and you need to set up some global
variables or variables to pass information and you need to set something up at the beginning
of a program and do something in between. So, when I’m blinking an LED this is driven
by i2C. So there’s an i2C chip in there and so when you set the LED to a certain value
and the driver that’s actually doing that at the end of the day is written in assembly
and doing some bit banging to set the i2C device to a certain state. Whereas the variable
assignment in this particular language is straightforward like it is MO2setX(21) does
not require any other variables, it does not require any other objects, it’s just sitting
a variable to one. So, some of these blocks are very complex and some of them are very
simple.>>And how was the robots where they communicating
over Bluetooth or…?>>SANDER: So, we have different ways that
they communicate. Today they’re communicating over XP. It’s a $19 solution–different ways
and you can have multiple robots up to thousands and this particular one has a range of about
a kilometer and works very nicely 100 to 110 kilobyte per second. And what we did is we
developed a very flexible communication tool that lets you share memory but also send messages
back and forth, like, male or your force to respond and to where you’re able to command
other robot to do something as well as terminal emulation so all robots running on the same
hardware.>>Very nice.
>>So, I might have missed this but is any of these for sale or is this just a project
right now?>>SANDER: Getting very close. So, about a
month or so away from selling Tbots and they were initially designed for summer camps for
students go to summer camp [INDISTINCT] and at the end of a camp there’s a competition.
So, that’s what these are built for. 12Blocks has been for sale I’ve been selling out for
six months now.>>Okay.
>>SANDER: And the props called parallax [INDISTINCT] and view port I sell that as well.
>>Uh-hmm.>>SANDER: If you go to handover.com then
you’ll get a link to all of those.>>Okay, thanks.
>>SANDER: Okay, thank you very much.

6 thoughts to “12Blocks: A Simple Block Language that lets Kids Program Sophisticated Multi-core Robots”

  1. I love innovation and there are some great ideas here… But how is this not a reinvention of LabVIEW?

  2. They should introduce this in all US elementary schools. Our school systems are broken and we are losing to the rest of the world. Something like this would inject some greatness back into the broken system. Good stuff!

  3. @cageorge76 I lament that 12Blocks could be used as a teaching tool to introduce students to the concept of information flow that ultimately leads them to traditional typed coding. Once students experience the joy of systems design they are led to the "professional" concept of text-based syntactical state machines. Too many students bail along the way. Traditional coding should be evolving into something like 12Blocks, not the other way around…

  4. You should name variables automatically based on the connected block. For example, your LED app would name the variable "color". A second color variable would be called color2, etc.

Leave a Reply

Your email address will not be published. Required fields are marked *