Bret Victor The Future of Programming

Good afternoon I’m really excited to be a i’m talking
to so many programmers of automatic computing machines as you know computing is becoming more
and more important our society this now literally thousands of computers around the world
and they’re being used from everything from business and accounting to scientific
experiments and who knows what they’re going to be used for. so more more computers and coming down in price coming down
size a computer used to be the size of this entire hall and now they’ve shrunk to
really tiny proportions so is this this time a really rapid change
in the field of computing so I thought to be interested have a
look ahead a little bit to the future programming and think about given what we know now what programming might be like say forty
years from now so what I’m going to talk about today there’s
been some really interesting research has been done just the last ten years or so and i think is have implications for what program he might be like in the
future so there is 4 big ideas kind of 4 big topics that I
want to talk about um that come out of her recent research
but before I get to those for big ideas I
want to talk about the nature of adopting ideas in the first place so basically what we’ve we notice is
that technology just quickly people’s minds
change slowly so it easier to adopt new technologies it can be hard to adopt new ways of
thinking so technology wise Gordon Moore He has this company called Intel and he
observed about ten years ago that computing capacity was a increasing
exponentially over time because extrapolate is out to about now and he’s been right on target and who knows
how long will keep going but it seem pretty reasonable so the lesson to be
drawn from this is that we can kinda take this for granted
because if we just wait our computer get faster to get more capable we can just wait and that’s just gonna
happen what won’t just happen if we wait is peoplee changing people adopting new
ideas so as an example that I’m sure you all
remember this guy the old IBM 650 Steve you know IBM’s first kind of general
purpose mass-produce computer a lot of us cut our teeth programming on
this guy and and in the beginning right we all
programmed in absolute binary when we code it was
literally writing numeric codes for each instruction and that those what
we did that was programming and then afters some years that Stan Poley came along and he invented
this thing they called an assembler so that was the symbolic optimizing assembly program this language where you could
write in words if you wanted the computer to add
something about the word add you could use symbolic variable names instead of hard coding memory addresses it is this much more powerful way of
thinking about programming you were much more productive made much fewer
errors assembly was shown to these guys the
guys coding in a binary and they just weren’t
interested at all they just didn’t get it they didn’t see any value in doing this stuff so there can be a lot of
resistance to new ways of working that require you to have unlearn what
you’ve already learned and think in new ways and there can even be like
outright hostility so John von Neumann the great scientist who invented the von neumann computer
architecture that we use and so many other things he said I don’t see why anybody woud
need anything other than machine code and he one-time so he had a bunch of students and
students were all kinda coding along and in binary and one time one of his
students took a little time out to write out his own little assembler so he could write in assembly and by von neumann was furious at him furious
that he would waste precious machine time doing the assembly that was clerical work
that was supposed to be for people right and so we saw the same story happened just a
little bit later when john backus and friends came up
with us idea they called fortran this is so call high-level language
where you could write out your formulas as if your writing mathmatical notation you
could write out loops and this was shown to the assembly
programmers and once again they just they weren’t interested they don’t see any value in that they
just didn’t get it so um I want you to keep this in mind as I talk about the four big ideas that I’m going to talk about today that it’s easy to think that technology
technology is always getting better because Moore’s law because computers are getting always more capable but ideas that
require people to unlearn what they’ve learned and think
in new ways there’s often enormous amount of resistance people
over here they think they know what they’re doing
they think they know a programming is this programming that’s not programming and so there’s going to be a lot of resistance to adopting new ideas the four ideas I want talk about today
this is all coming out a very recent research the first one is today we writer
programs in code. We write a basic list of
instructions for the computer to do there’s been some really interesting research on
direct manipulation of data where you directly manipulate the data
structures and that implicitly builds up a program for the computer to follow second I want to talk about today we write procedures basically here’s a
procedure for computer to do um interesting research on go on
programming using goals telling the computer what you want not how
to do it the and the computer itself figures ou how to do it number three today we program using lists lines of text in text files um people are doing
something really remarkable just like in the last five or ten years they’re hooking
video displays up to computers and when you do that
everything changes like start thinking about spatial representations of information and
the fourth thing is we program in a sequential
programming model is basically here’s a bunch of instructions computer
does them one after the other but hardware changing soon we’re goona see massively parallel hardware and we’re gonna need a sound parallel
program model to program on a hard work so the first thing i wanna talk about that
direct manipulation of data and I’m gonna show this project that Ivan Sutherland did PhD thesis about ten
years ago the system called sketchpad and sketch
pad was a system that allowed you to draw pictures on a
video display so he took his light pen and put on the
screen and so here try that draw the line drew that line drew more lines drew this little top he’s trying to draw a rivet here and he’s drawing it really sloppily it’s kinda tilted off to the side is kinda misshapen so what he does is he then he holds down a switch and indicates a couple these lines to the computer
system indicating that he wants these lines to be mutually perpendicular so the system runs an iterative solver kinda
wheels the lines around and figures out how to turn them into something that mutually perpendicular how to turn
them into a rectangle so basically he the system doesn’t know anything about rectangles he was able to get it to draw a rectangle by directly applying the set of constraints and what makes this a program as opposed
to just a picture is that these constraints are
dynamically maintained so he’s got his rivet that he drew and he
resized this corner of it and kinda resize some other things the solver kicks back in turns back into
a perfect rectangle so essentially he’s created a program that draws a rectangle but he didn’t do it by
writing code he did by directly manipulating the data and
directly applying a set of constraints to them and so this kind of a simple example and then he went off and did fancier
things like here’s a bridge simulation so it actually simulates the physics for
bridge he drew this by hand the um the numbers here represent the
tension in those particular spans of the bridge and he can vary the weight that the load that’s hanging off of the bridge and it kinda deforms and um the system the sketchpad system doesn’t know anything about bridges he creates this bridge simulation program by
directly drawing and by directly applying asserts that a um very general constraints so I definitely see this as something that’s going to be really important in um 30 40 years from now I can imagine
programming by directly manipulating data structures
and let that build up the the code but especially for
things that are visual or physical like this so if say in a
few decades we get some sort of document format on some sort of web of computers I guess um I’m sure we’re gonna be creating all those documents by direct manipulation there won’t be any markup languages or stylesheet languages
right? that would that would make no sense Ivan Sutherland showed us how to do it back your 1962 so it’s all gonna be
direct manipulation in the future and it’s going to be
fantastic so the second thing I want to talk about
is programming using goals so we saw a little bit of this um with sketch pads constraint system so Ivan Sutherland wanted to draw a rectangle
he didn’t write a procedure to like draw each side of the
rectangle he apply to set of constraints and the
system itself kinda figured out how to draw that rectangle so he kinda said
what he wanted I want things to be mutually
perpendicular he didn’t say how to do it the solver figured out how to do it so
another great example of that just came up a few years ago Carl Hewitt is doing this system called planner which is really great it actually goes
in both directions so it can reason forward procedurally it can reason backwards from goals so if you
tell planner that apple red then if you get an apple knows
ahah it’s red but you can also say I want
something red and I’ll say oh lets try an apple so you can
express your program in terms of the goals the results that you want from
the program but you can also provide procedural
strategies for meeting certain types of subgoals really um really
interesting way of thinking a programming and this led to another system a few years
later called prologue which just kept the backwards part of
planner so in prologue you can express your program as goals and the system itself uses a search or
whatever to try to figure out how to meet those goals so
this is leading to a genre of programming that’s called logic programming but that’s that’s not really important
part here what’s important is expressed your
program as what you want to not a setup instructions on how to do
it letting the computer itself figure out how to do it another example of that same sorta concept is pattern
matching so I’m sure you remember on you all remember snobol it’s the a text manipulation language
give a bunch of text you want to shoot it throw a snow ball script or program at it and um snobol had a built in features for
pattern matching so you can express some patterns that you want to a match against a text a little bit later Ken Thompson so is over at bell labs working on this system they call Unix um I know right? Unix but um he adopted ?’s notion of regular
expressions to do pattern matching on text so when you have pattern matching if you
want to digest a big bunch of text you don’t go and write a parser that
kinda goes procedurally through it you provide a pattern this kind of template this is the sort of thing i’m looking for. and the system itself figures out how to
match the text against that pattern so all these examples a sketch pads
constraints planner prologue pattern matching again
they’re all all examples of giving the computer a
high-level goals saying here the sort of thing i’m looking for letting the computer itself figure out how to do it and we’re seeing a little bit of that sorta
thing in optimizing compilers but I think it’s gonna be really prevalent and few decades from now and the reason
that this is going to be so important this goal directed stuff has to do with this this idea that look litres kicking around so as you all know Licklider is heading up
up ARPA the government funding agency and he’s been pushing this idea of a global network of computers just taking all the
computers in the world hooking them up to each other and he calls it the intergalactic
computer network because he knows the engineers always
deliver the minimum so if he asks for a network that spans the galaxy he’s hopes to get one that at least spans the world and people are calling this the ARPANET now its no turning into some
sort of inter net now it’s kinda cute idea might work and when you have this kind of global
network of computers you run into what Licklider calls the
communicating with aliens problem so he put it here the promise is essentially the
one discussed by science fiction writers how do you get communication started among
totally uncorrelated sapient beings I’ll explain what he means by
that so say you’ve got this network of computers and you’ve got some program out here
that was written by somebody at some time in some language its speaks some
protocol you’ve got another program over here written by
somebody else some other time speaks a totally different language
written totally different language these two programs know nothing about
each other but at some point this program will figure out that
there’s a service needs from that program that they have to talk to each other so you’ve got these two programs don’t know anything about each other
written totally different times and now they need to be able to communicate
so how are they gonna do that well only one real answer that scales that’s actually going to work which is
they have to figure out how to talk to each other right they need to negotiate with each
other that the probe each other if the dynamically figure out a common
language so they can exchange information and
fulfill the goals that the human programmer gave to them so
that’s why this goal directed stuff is gonna be so important when we have this
internet is because you can’t write a procedure
it’s because we won’t know the procedures for talking to these
remote programs these programs themselves have to figure
out the features for talking to each other and fulfill higher level goals so this if we have this worldwide
network I think that this is the only model that’s gonna scale
what won’t work would be a total disaster is I’m gonna make up a term here API this
notion that you have a human programmer that writes
against a fixed interface that’s exposed by some
remote program first of all this requires the programs
already know about each other right and when your writings this program in this one’s language now they’re tied
together so the first program can go out and hunt and find other programs that implement the same service they’re tied
together if this one’s language changes it breaks this one there it’s really
brittle it doesn’t scale and worst of all you have it is basically the machine code problem you have
a human doing low-level details that should be
taken care of by the machine so I’m pretty confident this never
happened we’re not gonna have API’s in the future where we are going
to have are programs that know how to figure out
how to talk to each other and that’s going to require programming
goals the third big idea that I wanna talk
about is spatial representation of information so today our programs are basically lots of lines
of text a big file full of lines of text and that makes sense when your program is a on a stack a punch
cards or it’s a paper tape or a magnetic tape this very linear media makes sense to have your
programs have linear form if you’re using a teletype then a teletype is
made for spitting out lines of text that’s what it does so of course your programs are going to be in lines of text but as I mentioned people are doing
something really wild and crazy right now which is hooking video displays up computers and when you have a video display
hooked up to a computer you can start thinking of your computers this kind of very dynamic sheet of paper where you can represent things spatially so Doug Engelbart over SRI has this system that he calls on online
system that he calls NLS He gave a big demo five years ago you might have seen it and there’s a lot of really remarkable things about this
system one the most remarkable is this notion of displaying
information over a screen over a video screen so he has this device this um this device called a mouse where you
kinda rolled around the table it’s hard to explain but you can use this to point to
different parts of the screen and indicate that you want more information
about something that you’re pointing to and it also has this notion of
different views of information so you can see here we have some data that’s in a list and then you can flip that over and look at that same data as this kind of 2 dimensional diagram so really start to think about how can we represent information dynamic information spatially another great system kind of about about the same time coming out of the RAND Corporation called GRAIL and this is a system for programming using flow charts on a video
display and the input device here is a a stylus
on a tablet and that you can drop draw these flow charts and let me show you how that works the programmer is drawing this box and just
totally free handing it and he draws a box and the system
recognizes that it’s a box and turns it into a flow chart box so assigns semantic meaning to these drawings that he’s doing he wants
to give it a label so he just start writing letters the system recognizes his handwriting 1968 system recognizes handwriting turns it into text here he connects
up this box to that one with
the line and so on so it’s all very direct manipulation if he wants to get rid this line he just kinda
scruples it out and it goes away ands so really thinking about were
programming means when you have a video display when you can express things in two dimensions but when i’m talking about spatial representation of information I’m not just talking about things like flow charts so um xerox has a little research center in Palo Alto with
some kids over there working on something that they call Smalltalk and in Smalltalk the source code is expressed in text but um there’s no like big long text file with
whole bunch of code in it it’s organized in a
spatial fashion so here’s what they call a browser so in this list here here’s all the
collection of classes here’s all the class is that collection here’s
all protocols in that class here’s all the method in that protocol and here’s the source code for that
particular method so the method definitions are text but
they’re not a one huge line of text they’re organized
spatially so you can get around the system very
quickly and see what’s going on so between Engelbart’s NLS Grail small talk is very different ways of
representing information spatially so I’m totally confident than 40 years we
won’t be writing code in text files right we’ve been shown the way and as a side note all these system I just showed you and Engelbart’s system Grail Smalltalk on this thing that’s
going on at University of Illinois call Plato also really interesting system these are
part of this new wave of interactive computing where you sit
at the computer and you’re actually interacting with the computer in
real time and these guys know that they’re trying
to prove out this new concept and so they’ve designed the system from
the very bottom to have an immediate response the user interface is always immediately
responsive you interact with anything you
immediately get a response so it’s kinda of simulating a physical object and so if interactive computing takes
off and I think it will then i think is pretty
obvious that in forty years our User Interfaces if you interact
with them you’ll never experience any kind of delay
or lag right because these guys proved how important is to have immediate
responsive UI and they were doing this in the in the
sixties so as our computers get a million times faster obviously there’s no reason to have any kind of delay or lag in the operating system in the in the user interface so that’s gonna be really exciting the forth thing that I want to talk about is um parallel programming so today our programs are basically a sequence of instructions computer do
this then do that and do that and do that and one of the reasons that we program in
this sequential model has to do with the hardware so we’re we’ve been using this computer architecture called the von neumann computer
architecture where you have a processor and that’s hooked up to a
big memory fetching words from memory and um so the sequential program model
makes sense when you just have one processor the processor can only do one thing at
a time you give us list the things for the
processor to do and it just kinda does each one of those um one characteristic of the von neumann architecture though
is that most to this memory is idle most of the
time you’ve got this little processor over here
and it’s kind of processing as fast as it can but only one word memory is ever being
accessed the rest the memories just kind of
sitting there waiting and that works when when your processor is
made out of vacuum tubes or relays and it’s kinda of big an expensive and your memory is made out of core or a rotating drum is also big and
expensive different then you can kind of get away
with that but we’re starting to see an incredible
invention coming into the field of computing right
now which is I think gonna change everything and that is the integrated circuit
semiconductor integrated circuit so this is a thingthat the a company
called Intel made it’s called the microprocessor and it’s
an entire processor on a single piece of silicon so the
entire processor is just made out of transistors and a transistors just a little picture you
etch into silicon and the entire circuit just like one big
complicated picture that you etch into silicon so our processors are just made out of transistors and silicon our memories as well are also going to
be made of transistors on silicon it’s all the same
stuff so when you look at the von neumann architecture from that perspective you’ve got these transistors over here
that are working really hard to process things it got this huge array of transistors
over here most of which are just kinda sitting waiting they’re not
processing not doing anything and so if you want to put those transistors to
work you actually want to maximize um the
amount of processing that you can get out the piece of silicon you need to start looking at things
that are more like this right so what computers want to be on
silicon is they want to be lots of little
computers like a huge array of tiny little computers with their own processor their own little state doing their own
thing communicating with each other that’s how you maximize the amount of compute
per area of silicon and it scales so when the transistor gets smaller when the silicon die gets bigger you have all this extra space you just
filled up with more processors right done real easy this is the kind of architecture we’re going to be programming on the in the future unless you know unless Intel somehow get
a stranglehold microprocessor market and pushes this architecture forward thirty years but that’s not gonna happen
we’re going to programming on these things and when you have this hardware you have
to start thinking about how do how do we program on that what’s our
programming model for this sort of hardware and the way that we do programming today is with threads and locks right you have a few sequential thread for control and you can
pretend that they’re going in parallel by multiplex the microprocessor and they try to lock each other out from
shared resources and like this is never gonna work right this does not scale you can’t reason about
hundreds of threads all banging on the same shared memory at the same time threads and locks they’re a dead end right so I think in forty years if we’re
still using threads and locks we should like pack up and go home cuz we’ve clearly failed as an engineering field if it’s not threads and locks then what’s
going to work and Carl Hewitt and that’s the same Carl Hewitt that did planner came up with this idea that he called the
actor model so the actor model is a model
computation that’s inspired by physics so in physics you’ve
got all these particles and all the particles are just independently
doing their own thing and they have their own little states and
they interact with the ones that are around them and Carl Hewitt was thinking of computer
processes in the same way got a whole bunch processes and they’re all kind of asynchronously doing their own little thing have their own state and they’re sending messages to each
other so really interesting really new and
exciting way of thinking about programming so a lot is kinda heating up right now
Gilles Khan and France has some ideas I think Tony Hoare getting
into it with something he’s gonna be calling Communicating Sequential Processes and maybe even Robin Milner’s going to
join the party so really exciting stuff happening here now
for this talk the details of these particular
models don’t really matter um I do think it would be kinda cool if the actor models was like picked up by the Swedish phone company
or something that would be kind of weird but but what matters here is we are gonna
have massively parallel hardware we need a sound parallel programming
model that fits the hardware and something like
this is is going to be what we’re gonna be
using so those are the four things I want to talk
about direct manipulation of data something like sketch pad where you’re
drawing pictures dynamically adding contraints to those pictures directly manipulating the data
structures instead of writing instructions for the program programming using goals and constraints
things like sketch pads constraints planner and prologue um regular expressions other types of
pattern matching where you’re telling the computer what you want to do and the computer itself
has solvers the figure out how to do that spatial representations of information we’re
not gonna have text files anymore we’re going to be representing information
spatially cuz we have video displays and fundamentally parallel ways of
thinking parallel hardware parallel program
models um no more threads and locks no more sequential thinking so those are the four things I wanted to talk about and you know I’ve tried to make some
predictions about the future and you can’t really predict the future right so these are some
good ideas I don’t know what’s gonna happen to them
ideas kinda split and merge in go in and out of fashion so you know
anything could happen but I do think that it would be kind of a shame if in forty years we’re still coding in
procedures in text files in a sequential programming
model I think that would suggest we
didn’t learn anything from this really fertile period in computer science so that would kinda be a tragedy but even more of a tragedy than these ideas
not being used would be if these ideas were forgotten if if anybody were ever to be shown this stuff and actually be surprised by it right but even that’s not the biggest
tragedy that’s not the real tragedy the real
tragedy would be if people forgot that you could have new ideas about programming
models in the first place so let me explain what I mean by that the here’s what I think the worst case scenario would be is if the next generation programmers
grows up never being exposed to these ideas the next generation the next generation programmers grows up
only being shown one way of thinking about programming so they work on that way
programming they the flesh out all the details they you know kinda solve that
particular model programming they figured it all out and then they teach
that to the next generation so that second
generation then grows up thinking Oh it’s all been figured out we know what
programming is we know what we’re doing they grow up with
dogma once you grow up with dogma it’s really hard to break out of it do you know the reason why all these
ideas and so many other good ideas came about in this particular time period the sixties early seventies why did it all
happen it’s because technology it was late enough that technology had
kinda got to the point where you actually could do things with computers but it’s still early enough that nobody
knew what programming was nobody knew programming was supposed to be
and they knew they didn’t know so they just like tried everything their minds were totally free and they
just like said maybe we could program like this maybe we could program like that they just you know tried anything they
could think of so the most dangerous thought that you can have as a creative person
is to think that you know what you’re doing because once you think you know what
you’re doing you stop looking around for other ways
of doing things and you stop being able to see other ways of doing things you become
blind you become like like these guys over here coding
along in binary someone shows them assembly language
someone shows them fortran this they can’t even see it it just goes
right over their head because they know what they’re doing
they know what programming is this is programming that’s not programming and so they
totally miss out on this much more powerful way of thinking so the message of this talk you know it’s not really it’s not really
the stuff right the message on his talk is if you don’t
want to be this guy if you want to be open and receptive to new ways of thinking to invent new
ways of thinking I think the first step is you have to
say to yourself I don’t know what I’m doing we as a
field don’t know what we’re doing I think you have to
say we don’t know what programming is we don’t
know what computing is we don’t even know what a computer is and once you truly understand that and once you truly believe that then you’re free and you can think anything thank you

Leave a Reply

Your email address will not be published. Required fields are marked *