Advanced Python or Understanding Python

Advanced Python or Understanding Python


>> Hello. Yes, hi. Welcome to the latest in our
series of talks in advance topics on programming languages. Today we’re very lucky to have
Thomas Wouters whose here to talk to us about features that are currently available in Python
but are advanced features. Last week, we had–we were very lucky to have Guido, that awesome
talk about upcoming features; they will be talking about existing features, right? Yes?
I wanted to–as always, this is the latest in our series of talks. I always want to make
a pitch so that people will actually give talks of their own. It’s a really useful thing
to do. If you have specialized knowledge or if you have interesting knowledge, to share
that knowledge with as many Googlers as possible and potentially even the rest of the world.
So, please come and see me and we can set up a talk and you can give–yet another, in
our series of talks here. And with that, I will turn it over to Thomas who shall give
a very interesting topic. Thank you.>>WOUTERS: Is this on? Right. Although there’s
an echo. So I’m going to give a talk about Advanced Python which is not an easy subject
in that there’s a lot of Python that can be considered advanced. I’m not sure what your
level of advancement is so I’m going to cover basically everything from the start until
the end and we’ll see how far we’ll cover. These are the subjects I’ll be covering. I’ll
certainly be explaining what the Python design, objects, iterators and generators and hopefully
decorators. Everything after that is a bonus. I’m going to keep the pace pretty high, so
if there’s any questions, just wave your arms and I’ll stop and explain. But if everyone
can keep up then we can probably cover the more interesting topics at the end as well.
And if there’s specific interest in any of the topics especially, for instance, Unicode,
I can skip over the advanced stuff and jump right to Unicode. So, first about Python,
Python was developed by Guido as a middle system language between shell scripting and
system programming. So it was intended to be easy to use by normal programmers but still
allow more complex structures than a shell script–scripting language. It turns out that
it’s also convenient for library programmers. People doing the actual programming for end
users because you can hide a lot of smarts inside of objects or modules and just expose
a very simple API to end users. This has gotten progressively better with later releases of
Python. There are multiple implementations of Python; CPython is the one everyone uses
except for those that are developing the others. Jython is–has been around for quite awhile
it has–is actually in use by a lot of people but not as much as CPython. IronPython and
PyPy are still actively being developed although, IronPython is apparently very usable. It is
designed by implementation in that if you don’t come up with an implementation of a
feature that you want implemented, it’s not going to happen. But it’s not a slave to the
implementation and it doesn’t mean that if you implement an idea, it will get in. It’s
still a very designed language. We have feature releases every one to two years. It used to
be every year or so and now, the last one was in the works for two years and there’s
a lot of bugfix releases as well. An important note about the feature releases, if you do
not get any warnings in using a particular minor release, upgrading to newer minor release,
the one following that minor release will almost certainly work. Any–anything that
changes semantics or possibly breaks code will be warning for at least one release.
And bugfix releases have a similar requirement that you can always upgrade to a newer bugfix
release of the same minor version and nothing will break that wasn’t already broken. Important
in realizing how Python works is that everything is runtime, even compiletime is actually runtime.
As you may know, there are cached files, .pyc files that are cached bytecode, that’s basically
borrowed runtime, someone else’s runtime that you can skip. Also important is that all execution
happens in a namespace and that namespace is actually a dict, it’s a dictionary object
with keys and values. Modules, functions and classes all have their own little namespace
and they don’t affect other namespaces unless they explicitly do so. And modules are executed
top to bottom. Script start running at line one and they run all the way down to the end
or until loops or whatever make them wait. So in that regard, it’s very much like a script.
When you import a module, it does the same thing, it just starts executing line one and
runs on down until in the end and the import is done. And another import aspect is that
the def and class statements don’t define a function or at compilation time, it’s a
runtime thing. It actually happens when bad statement is reached. So some more about functions
and classes as well because they’re important features. At compiletime which is runtime,
the code for the function will be compiled into a code object. It’s a separate object
from whatever the rest of the code is compiled into. And then at def time when the def statement
is executed, it will be–the code object will be wrapped into a function object along with
whatever arguments it takes, any defaults to those arguments. And essentially def is
an assignment operation. If you look at the bytecode, you’ll see that it just uses the
same bytecodes as normal assignment operations. So here’s an example of a function. The red
part is the actual def statement that gets evaluated when it’s reached including the
part in blue which is an argument default. That part gets evaluated at the moment of
def. So the Foo object is constructed or the Foo function is called, if it’s a function,
at the moment the def is being executed and the green part has been compiled at that time
because compiletime has been in the past but it’s not being executed until later which
is when you call the function. So this is when you call the function, there’s an inner
function in there, so what happens is that the green lines you see here, they’ve been
compiled into their own code object and when you call func, the inner func gets created
at that time and again the default argument of the defaults for the three argument is
evaluated at the moment you call def. If you look after the definition of inner func, you’ll
see that arg1 and arg2 are reassigned. That means that whatever arg2 was, will be assigned
to inner func’s arc3 and not whatever it is at the moment you call inner func. However,
inside inner func, we also use arg1 which is using [INDISTINCT] scope which I’ll be
mentioning later as well, which means that it will be using the outer functions arg1
and that will be whatever arg1 is at the moment you call inner func. So there’s a very big
difference between those two uses.>>Question?
>>WOUTERS: Yes?>>That very final line, was that evaluating
inner func or [INDISTINCT] function?>>WOUTERS: No, it’s just returning function
object.>>Okay, thanks.
>>WOUTERS: So it’s not calling anything, only the parenthesis do actual calling. Yes?
>>[INDISTINCT] could you repeat the question…>>WOUTERS: Sorry? Yes.
>>[INDISTINCT]>>WOUTERS: The question was is the last line
actually evaluating inner func? It is not, it’s just returning the function object that
is inner func. So class statements are very similar but also very different. A class statement
is followed by a block of code, a suite, that’s executed when the class statement is reached
right away. It is however executed in separate namespace, a dict again. And then after that
code of block has been executed, the namespace is passed to the–or is used to create the
actual class object. Any functions that you define inside the code block then get turned
into methods by magic, basically. And–well, as people are already programmed in Python
will know the self argument of this, for Python, is passed implicitly when you call a method
but your method still have to receive it explicitly as the first argument. And an important thing
to realize is that the code block in a class is actually a normal code block. You can do
whatever you want in here, for instance, at the top I create a class attribute which is
just something stored in the class, result of calling func. I define a method which is
just a normal function but it takes self [INDISTINCT] argument and it assigns to an attribute of
self when it is called. I also define a helper function which just did some silly arithmetic
and call that helper function inside the class block itself to calculate or recalculate the
class header, and at the end I delete helper because otherwise, it would end up in the
class and since it doesn’t have self [INDISTINCT] first argument, it wouldn’t do very helpful
things. Other important parts about Python, variables aren’t containers like in other
languages. They’re not allocated memory that you can use to store data. People often say
about Python that everything is an object and they also sometimes say everything is
a reference. That’s both true but both are not true when you apply them to variables
because variables are not objects and variables are not references. Variables are only labels.
Everything concrete, everything that allocates memory, everything you can toss around is
an object and whenever you hold an object, whenever you store it anywhere, you’re actually
storing reference. You’re not actually own an object, you only own references to an object.
Variables don’t have type information, they don’t have information on the scope or the
variable or how many objects live around or when they were created, the only thing they
do is refer to an object and that’s it. If you want to look at it from an implementation
standpoint, it’s a dict. It’s a dictionary mapping names to objects, that’s it. So, scopes
also related to namespaces. Python has a very simple two scope rule which is actually three
scopes. Names are either local or they are global or built in. Local means it’s–it exists
in a current function or class or module but it doesn’t exist outside. Global means it
exists in the current module, it doesn’t mean exist everywhere, just in a current module.
And builtin is a special namespace that adduced by builtin functions that you can actually
modify. It’s just a special module that you can modify yourself if you want to. When the
Python compiler examines a function and compiles a function, it keeps track of whatever names
you assign to and assumes correctly because by definition, those names you assign to are
local. You can change this by using the global declaration which is the only actual declaration
Python has. But don’t do it too often, it’s usually a sign that your code is incorrect.
All other names that the compiler finds are either global or builtin and the lookup is
it looks in global and then it looks in builtin. So if you have a global of the same name as
a builtin, all the code in the module will find the global of that name instead of the
builtin. You can actually use this on other modules as well. If you import a module and
then assign to an attribute of the module, you’ll be assigning to a global name in that
module and you can mask whatever “itthings” is a builtin name by assigning to that attribute.
There’s also a trick with nested scopes which were added later in Python. I think in Python
2.1 where you can–as I pointed out before, you can refer in an inner function to a variable
in an outer function but that is only read-only. You cannot assign to a name in an outer function.
This isn’t really a mechanical problem in Python. It would be possible to assign if
we added it but there’s no syntax to say and one I assigned to another–outer scope variable.
Global only assigns to a global name and not to outer scopes. Apparently, Python 3 might
be getting–Python 3.0 might be getting syntax for this. So I mentioned modules. Modules
are there to organize code. They’re very convenient because they have their own namespace with
global names. They also keep the rest of the module tidy. They’re always executed on first
import and then cached in sys.modules. This cache is just about everything that you import
in your program and it’s also the only thing that keeps track of what you have imported.
So if you toss something out of sys.modules and import it again, it will get executed
again. Import is just syntactic sugar, just like–basically everything in Python is syntactic
sugar. It calls the __import__ builtin function. If you want to do an import over module whose
name you have in a string object, you use __import. If you want to replace __import,
you can just write your own function and replace the one in the builtin namespace. Another
trick is that sys.modules does not have to contain modules. It’s a mapping of name to
whatever object you want import to return. So you can toss in any–any all object in
there and then importing that name will return your objects. Storing none in sys.modules
just means this module couldn’t be found. So, if you want to prevent some module from
being imported, you can insert in sys.modules “none” under that name and then it’ll raise
ImportError whenever you try to import it. Python objects in general are described under–in
various terms, Mutability is a common one. That means the object is changeable. List,
for instance, are mutable, Tuples are not. Mutability is very important in Python because
everything is a reference. If you have a mutable object and you end up changing it by accident
and changing it for everyone, that’s rather inconvenient. So you have to keep in mind
whether objects are mutable or not. Fortunately, this mostly happens by accident correct, anyway.
Related concept is Hashability, whether you can take the hash of an object. In normal
operation, mutable objects are not hashable at most, immutable objects are hashable. For
instance, Tuples are hashable but they’re only hashable if all their values are also
hashable. Hash–hashes are being used just for dicts and sets right now, so any dict
key has to be hashable in any set item, anything you want to store in the set has to be hashable
as well. But it’s not inconceivable that more stuff uses hashes. And then there’s the imaginary
abstract base classes that are often referred saying that an object is file-like or it’s
a sequence or it’s a mapping. Those are abstract concepts that are somewhat like protocols
or interfaces or whatever you want in another language but they’re just informal. They’re
just saying whenever an object is sequenced; it has the sequence operations implemented.
You can look in and you can iterate over it, etcetera. Some of them are overlapping, for
instance, sequences, mappings and files are all iterable so the iterable interface applies
to all of them. And actually all of the objects you define, all that you’re defining is implementation
of syntax. Python defines syntax and you can plug in whatever implementation you want in
that syntax. And you don’t do it by–in other languages you say you implement the less than
operator, in Python you say you implement the less than operation so that it can be
used even when you’re not actually using the less than operator. Here’s a short list of
hooks that Python supports. I’m not going to go over all of them. Most should be–should
be common enough. The one thing that’s worth mentioning is conversions. You can define
a few hooks that define how your object gets converted to other types. That just works
for builtin types of course, because the hooks don’t exist for arbitrary types. But most
arbitrary types that want to do something like this can just say if you have an __Foo
method, I’ll call it to turn you into a Foo object if I want to. But it’s not very common.
These things are not Python hooks. These things you cannot configure on your object. Assignment.
Assignment is an operation that changes the namespace and not the object. Since an object
is just the reference to an object, you have no control over where your references are
going. Another thing you cannot control are type checks. You cannot control when people
are comparing your type or doing instance checks, you cannot control what they get as
a result. Related identity comparison, the “is” operator, it checks whether two references
are pointing to or referring to the same objects, you have no control over that. “And” and “or”
and “not” are Boolean Operations and they just call the one truth value test that you’re
operated–your object can implement and you have no control over what they actually return.
Some may know “and” and “or” in Python return, one of the two values were as “not” returns
always a Boolean. And Methodcalls, you cannot define in a single operation whatever will
happen when someone calls a method on you because methods are getting an attribute followed
by calling that object. They’re two separate steps. So in order to implement methodcalls
in your object, you would have to implement getattr to return some in go-between object
and then have that go-between object do something when it’s called. I’m sorry. So on to some
implementation details in C. This is just–applies to CPython of course. Python objects are implemented
in struct that holds various bookkeeping information like ref counts, the actual type that an object
is, as well as arbitrary C data. It can be pointers, it can be in C, it can be an array,
whatever you want. Types are what describes what an object is and how it behaves. There
are separate struct which is also a PyObject struct. The arbitrary data in a PyType struct
is the function pointers and various other things that describe the type. The PyObject
structs are not relocatable. You cannot move them around once you’ve given them out to
anyone else. It’s a blog of memory that’s allocated and that’s it. If you want to move
it around, you have to destroy the object which means you have to get rid of all the
references to it. That also means that variable sized objects like lists that have a portion
that needs to be reallocated and might move around, are–just use an extra layer of indirection.
It’s just a pointer stored somewhere in. And of course because it’s C, it doesn’t have
ref counting by nature so ref counting is on manually, whenever you receive a reference
you incref and when it’s over, whenever you toss one out, you decref. It sounds simple,
it can get quite complicated but the Python C API is mostly clear enough that it’s not
too hard once you get used to it. Another feature that is done reasonably manually is
weak references. Weak references are references that get informed when their object goes away
so that they can do some cleaning up and not crash. The weak references are just callbacks
basically in the PyObject struct arbitrary data. One of the major problems with ref counting
is reference cycles. That is two objects that refer to each other causing them both never
to be cleaned up. Two objects referring to each other as of course to the simple problem
and the complex problem is a huge loop of object referring to objects and everything.
Python has a cyclic garbage collector which keeps track of objects that might participate
in cycles, for instance, lists and dicts and walks them every now and then to find object
groups that are not reachable through anything but themselves and then it cleans up the entire
cycle. This is all done in C, if you write a C extension or a C type, you don’t usually
have to bother with it too much. There are some methods you have to implement if you
want to participate in cyclic-GC, when you think you might be participating in cycles.
So in Python, when you have an object, what do you have? Well, you have an object that
has two special attributes, the __dict attribute which is a dictionary holding per object data,
and you have the __class attribute which refers to the class. And like in C, the class defines
behavior, and in newer Pythons the class is actually a PyType object and it all matches.
The way to define the behavior is not through function pointers because Python doesn’t have
function pointers. It’s with specially made methods. Methods that start and end with __,
those are specials Python. They’re not special because they get allocated differently, they’re
just special because some of them get called by Python automatically in some situations.
You can define your own __ methods and use them whenever you want. There’s nothing special
on that regard, it’s just the–some of them signals a Python, this needs to be called
whenever that happens. In general, you should not be calling another objects __ methods
yourself, not directly. You should always use them through whatever mechanism wraps
them. So for instance, if you want to have the string representation of an object, you
shouldn’t be calling object .__str__, you should be just be calling StrObject. Another
feature of Python is that class attributes, that is attributes of the class object, are
also reachable through the instance. If you do the instance.attr and it’s not defined
in the __dict, it would be fetched from the class and the class might have pair of classes
so they–those get searched as well. And of course in Python, the whole point of Python
is that you don’t have to do stuff that Python can do for you, so refcounting, weak references
and cyclic-GC are all done automatically, you never have to worry about them. Typical
Python code does not use type checks because–partly because it was never possible until a couple
of years ago to sub class builtin types. So pretending to be a builtin type meant that
other people would not have to use type–must not use type checks or you could never pretend
to be a builtin type. It’s also very convenient, it just means you can pretend to be whatever
type you want, implement the right methods and it’ll just work. The C layer sometimes
needs specific types, if you want to multiply a string by an integer, it needs to have,
actually have a string and an integer or there won’t be any multiplication in C. So the C
layer has conversions. When it wants an integer, there are special methods that say I need
an integer at this argument, it will convert whatever gets passed in–and it will convert
whatever get passed in to an integer or a string and do the multiplication that way.
If you really must use type checks, for instance, because you’re faking C code or you’re wrapping
C code or whatever, use isinstance instead of type. Checking type for a specific value
means you could–you only accept objects of that type where isinstance does proper instance
checks so that someone who does subclass whatever type you expect works the right way. Functions
become methods by magic in Python, it happens when you get an attribute off of an object
or rather when you get an attribute of a class through an object. Whenever you get an attribute
of a class that is a function, it get turn—-gets turn into unbound method which is a method
that knows it’s a method and it knows which class it belongs to but it doesn’t know which
instance, so when you call that method it knows that the first argument you passed must
be an instance of that class and then it’ll complain if it’s not an instance of the right
type. Of course that type check uses isinstance, so when you have the methods on bound–or
if you have an on bound method you can pass a subclass of whatever class it express–expects
and it works. Bound methods on the other hand are methods that know they’re instance and
they will be passing that argument automatically so you call them without the first argument,
you start at the second argument and it all looks normal. So any questions so far? No?
All right, on to iterators then. [pause]
>>WOUTERS: So iterators in Python are really simple, they are helper objects for iteration;
they encapsulate, if you will, the iteration state and they’re created and used automatically.
If you don’t want to bother with them, you don’t have to and it all just happens by magic.
If you do want to bother with them, you can do a lot of fun stuff with them, even more
so if you combine them with generators which I’ll be talking about next. Iterators are
two methods, __iter__ and next, notice that there’s no __ around next, because next is
actually intended to be called directly sometimes. So there’s no __ around it or people would
think that they shouldn’t be calling it. Because they’re simple, iterators in Python are not
rewindable, they’re not reversible, they’re not copyable, none of that, they’re the lowest
common denominator in iterators. There are however ways to write reversible iterators
if you want, you can just write your own iterator class and add methods to rewind or reverse
or whatever. You can also just nest iterators. And iterators themselves are iterable, they
are just—-actually an iterator is required in its own __iter method to return itself
or it wouldn’t be an actual iterator. So in the example I have here, I create—-exclusively
create an iterator over range which is a list of number from 0 to 10, not including 10,
and then I call zip on it which takes a number of iterables and takes one element of each
iterable and wraps it in tuple and stores that in a list which it returns. So I created
a single iterator pass the same iterator to zip twice, and as you can see zip takes one
element of each iterator consuming the iterator as it goes and ends up with having two elements
at a time basically from the original list of zero through nine. So generators are a
convenient way to write iterators, they are lazy functions that get executed as you iterate
over the result basically. Generators use the yield key keyword instead of return, it
works very much the same way except after a yield statement execution continues they’re–the
next time you call it or the next time you iterate. The function with yield in it will
return a generator, it will return an iterator that you can loop over and there’s—-this
is terribly convenient for __iter__ methods because you can just write what you would
expect in it’ll just work. In Python 2.5 generators were complexified and they can now also receive
data and–and you can use them to build co-routines very easily, nothing you cannot also do with
2.4 and earlier iterators, just more conveniently and with extra magic. There’s a lot of generator-like
iterators in the iter tools module which is I think new in Python 2.2 or 2.3, there’s
a whole bunch of stuff in there you can use to chain and filter and whatever with iterators
that are just really convenient for chaining their various combinations. So here’s a generator
example, the bottom function is map, as you may–may know it accept–it only takes one
iterable that goes in map function, it creates a list of results from applying the function
to every item in the iterable–iterable. In the bottom, there’s a two line function that
is the generator version and then there’s a one line function that is the original map
implemented in terms of the generator. As you can see you just basically lose half the
function if you use a generator because you generate items–each item on the fly and you
return generator instead of an actual list, any questions?
>>[INDISTINCT] now you can do it like that?>>WOUTERS: Can I define what a co-routines
and how would I do it in Python? A co-routine is a routine that is basically like the generator,
it stops execution, passing data to someone else but where a generator returns results
to its caller, a co-routine returns results to someone else, another function. So you
can have two routines where they both consume the output of the other and then the end result
is handled data. How would you do them in Python? Well as I said you can do them in
Python 2.5 with the new sending data to a generator thing, before 2.5 you would write
a class that was itself an iterator and just write it in bits and pieces and–it wouldn’t
be as convenient as co-routines in actual languages that have co-routines because you
don’t have a single blog of code, you have a whole bunch of separate pieces of code that
get called at the right time, I wouldn’t bother implementing them in Python right now. Maybe
with 2.5 outer 2.5 out and people getting used to the new stuff you can do with generators
will get actual coroutines in Python. Yes?>>[INDISTINCT]
>>WOUTERS: The question was if generators are lazy, can you write a generator that loops
infinitely and just keeps on yielding results as long as you ask for it? Yes, yes there
are…>>Then you just ask for finite number?
>>WOUTERS: Well it wouldn’t—-do you ask for a finite number? You can if you use the
iter tools module, you can slice iterators, you can say up to so many items in this iterator
and it’ll return in iterator that starts consuming the original iterator until the slice ends.
But you don’t have to do that. The iterator is something you loop over, so if you have
a loop that loops over two things at once and it stops whenever the condition is met,
if you don’t–and you can just loop over an infinite object and rely on the fact that
your loop will end itself for other reasons than that the iterator stops. There’s actually
an infinite number of infinite iterators in the iter tools module like itertools.count
which starts counting in zero and just keeps on counting forever and ever, unless you stop
for some reason. [pause]
>>WOUTERS: So on to decorators. Decorators are syntax in Python 2.4 for a feature that’s
been around forever which is function wrapping. Decorators are just funcs—-are functions
or callables rather, that take a function as argument and return replacement callable
or they can return the original function if they want. However, the syntax means that
it can get confusing if you have a decorator that also has to accept its own arguments
because now you have a function that should return a callable that should accept a callable
and then return the original callable, so we’ll see some examples of that. Another problem
is that because functions are special when they’re attributes of a class they become
methods, when you have the decorators that returns something that’s not an actual Python
function but something that acts like a Python function for–in most regards, it won’t get
turned into method unless you implement the same magic that methods are–that functions
have that turn them into methods which is __get which I’ll maybe explain later. So if
you write decorators make sure their methods or make sure they’re functions or they won’t
get turned into methods. And anyway, as I said simple concept but the devil is in detail
and we’ll see about that. Here’s a simple decorator, it’s a spam decorator that says
whenever the function that you decorate, the same function at the bottom is called, it
actually calls the wrapper function at the top which loops for ten times calling dif–original
function. So in this piece of code the original function get a called–gets called ten times
and there’s no return value of the actual function call which is [INDISTINCT] then the
original either. Here’s an example of a decorator with an argument, the decorator is spam ten
which is not–no longer–spam is no longer the decorator, it’s rather the decorator generator
that takes a number which is a number of times to repeat. And then in spam, we define the
decorator function which accepts the function as an argument and then has a wrapper function
which codes the original function and then return–return wrapper and then of course
return the decorator from the decorator generator. So that will–looks–looks okay, I mean, it
takes some getting used to, the nesting and everything. But there’s another problem, what
about interspection? Maybe Python users don’t care about the name of their function but
some tools do, like Pydot for instance, which is the Python documentation tool, it actually
looks at function objects and looks the other name and their signature and their DocString
and whatever. And because we replaced the original function, when you asked for the
documentation of sing, it’ll actually say, it’s called the function wrapper and it has
a signature of *args and **kwargs and it has no DocString. That’s probably not what you
want. Another thing is that if you have another decorator in there, you can chain the decorators
that changes attributes of the function; those changes will go away because you’re not doing
anything with the attributes of the function. So, some people write code like this which
is the original spam with the repeats argument, with the decorator function in there, with
the wrapper function in there. And then after defining wrapper, we assigned the _name_,
the DocString, the _module_ and–and the dict of the original function back to wrapper so
that all those things will actually be the same for the new function and the old one.
And assigning dict like that actually works, you can–you can copy—-its not a copy it’s
a reference assignment, the original function—-all of the original function dicts or attributes
will be accessible in the wrapper function and when you assign to either one of them
it’ll–it’ll appear in the other one as well, they just share their attributes space. Now
this is not the easiest way to write a decorator, so in Python 2.5 in the func tools module
there is a decorator for decorators that does this for you. So you have a decorator that
you apply to decorator or to decorator generators and then that decorator generated decorator
gets applied to whatever function you have. So as you can see, the devil is in the details,
they can get confusing somewhat quickly, any questions? All right, how are we for time?
>>You got [INDISTINCT].>>WOUTERS: All right. New-style classes.
When I say new-style classes, when anyone says new-style classes they actually mean
all old new-style classes because they were added in Python 2.2 which was released I think
six or seven years ago something like that, they’re old. And it’s a unification of the
C types that I explained and the Python classes because before or actually still in classic
classes, instances and classes are distinct C types. There is a C type called class object
or class obj—-that implements all the magic that I talked about–about turning methods
into or turning functions into methods and there’s the C type instance which make sure
that instances work as expected with the __dict and everything. So they’re separate types
and if you ask for the type of an Int it will say it’s an Int, but if you ask a type of—-about
the type of any class that tries to be an Int, it’ll say “it’s an instance”. So you
have no way of checking all that. And another—-a problem with the original approach was that
you cannot sub class builtin types, so Guido worked on–in Python 2.2 on unifying C types
and Python classes and the end result is pretty good. You can mix them or match them and everything,
it worked good. But it only works because a lot of new general mechanisms were added.
They were necessary to bridge divide between C objects and Python types and things that
you assigned from Python have to be inserted in as a C data type in a C struct rather than
as a Python object pointer. Classic classes are still the default so if you write a new
class and you don’t specifically make it a new-style class, it’ll still be a classic
class and that was for compatibility reasons because there’s a lot of stuff that’s slightly
different between—-well a lot of stuff. There’s a couple of things that are slightly
different between classic classes and new-style classes mostly with multiple inheritance and
you can check if any class or–actually any instance of a class is an instance of a new-style
class because it inherits from object instead of nothing. So you can do actually do is isinstance,
my thing object and you’ll know that it’s an instance of a new-style class. The first
of the mechanism that I am going to explain is descriptors.
[pause]>>WOUTERS: Descriptors are a generalization
of the magic that happened with functions to turn them into methods. A descriptor is
any object that lives in the class namespace that are class attribute space, so it’s an
attribute of the class and that has __get__, __set__, __delete__ methods. It doesn’t have
to be–have all of them I think, you can just do it __get or __set or __delete for specific
operations. Whenever an attribute is retrieved–attempted to be retrieved from an object whose class
has an attribute with a descriptor in it, those methods on the descriptor will be called
and the result of those calls will be passed back to the object. Same for setting, it’ll
call the set method no result occurs and deleting they call the delete method. The delete method
is not called __del__ because that was already taken for some other hook apparently. It’s
now the method behind–the mechanism behind methods. So if you want to have a function
like object that behaves the same way as functions do becoming a method, you can do that by defining
__get__ and it’s also the mechanism behind property which is a trick of hiding accessories
behind normal attribute access. So here’s an example of properties, I’m not going to
give an example of actual descriptors because it’s too boring and you won’t be using it
anyway, but here’s a property. We have a class, we define the function get prop, it takes
a self argument even though it’s not going to be a method, it takes a self argument,
it doesn’t stop there and to return whatever the value is of the extra property. And then
we wrap it in property and store it in a local—-in a name that’ll eventually be a class attribute.
Oh, I see I have an error right there. So we instantiate the class, I should have had
Foo instead of X there and then we do foo.prop, foo.prop calls to the get prop and because
it’s a property, even though it’s not a method because it’s just a function inside the class
log, the property type knows that it needs to pass the instance for convenience, the
instance that it actually–is called on, onto the function that wraps it. If you look at
this you can say, “Oh, this can be a decorator, too,” and it’s true you can just say at property
at the top of def get_prop, except that property takes multiple arguments you can also pass
a set prop and the del prop if you want it, I didn’t do it in this example for brevity,
but if you just have a get there, you can just say at the top at property instead of
prop=property(get_prop) at the bottom, any questions about this? All right. So the other
general mechanisms, kind of related, they’re also descriptors, classmethods and staticmethods.
Before Python 2.2, Python only had “instance methods” that is normal methods, methods that
take self as the first argument, they got called in an instance and if you try to call
them on a class without having an instance, you get an error. So in the Python 2.2, we
got classmethods and staticmethods. Classmethods take class as the actual class objects as
the first argument and that’s terribly useful, I’ll show why in a moment. Staticmethods take
no magic argument and their not very useful even though Java and C++ programmers come
into Python often say, “Oh I need a staticmethod for this.” Generally not, they’re only useful
for one particular thing and I’ll show that in a minute. So here’s a classmethod. Again
it’s–you can–if you’re using Python 2.4, you can use classmethod at classmethod at
the top for the decorator syntax, if not you’ll have to use it at the bottom. So say we have
a FancyDict, that sub class of dict, and we define a method to create a dict from a set
of keys with the single value, so we don’t have to generate this list of key value pairs,
we can just say “generate it from keys”. So what I have here is we create a list of that
key value pairs and pass that to the class and because it’s a class method and gets the
actual class passed, we can call it on any sub class for FancyDict without anything,
in particular happening in the sub classes, and it’ll create a sub class of FancyDict
instead of a FancyDict itself. So whenever you think, “Oh, I should have a staticmethod
and I’ll do something with my own class,” you should actually use a classmethod and
do something with the first argument. Now, this is a rather silly example because dict
already has this exact thing implemented. There’s already a fromkeys method that is
a class method in the dict type and it’s very useful whenever you sub class dict which is
not too often. Anyway at the bottom it’s shown what happens when you use it. So staticmethods,
they’re not very useful, the main use is protecting dependencies from becoming methods. When you
use dependency injection as I do here in the example, you don’t know what you’re actually
injecting into your class. If it happens to be a method or–if it happens to be a function
or something that does something magical when used as an attribute of a class, this won’t
do what you want it to do; it won’t do the same thing as calling sendmail.sendmail where
I’m now calling self.sendmail. So you can wrap it in a staticmethods to prevent it from
becoming an actual method. That’s the only thing I’ve ever seen that makes sense for
using staticmethod. Although as we’ll see later, Python actually has a staticmethod
itself which is a good example of something that should have been a classmethod. Another
new feature, __slots__ which is for omitting the __dict__ attribute for arbitrary attributes,
it basically prevents you from assigning arbitrary attributes to another object. It reduces memory
use because the dict isn’t allocated and it’s a more compact form to store the same number
of attributes as a dict, but it’s not going to be much faster than a dict even for a few
attributes. The main reason to have it is when you want to emulate or actually implement
immutable Python classes like we add immutable types where you don’t want attributes assigned
to them arbitrarily and then your well-—there’s a tiny, tiny, tiny class for showing slots,
they’re right there. If you actually try to assign to something other than self the value,
either in init or anywhere else, it’ll actually throw an exception, except for stuff that’s
already in the class but, I mean, __init the def statement won’t be throwing an exception
of course, because Python knows that it’s already there. Another new thing in Python
2.2 is the actual constructor. Before Python 2.2 there was just __init which is an initializer
and other constructor, it gets called after the object has been created and it’s your
job to initialize it and set the right attributes and whatever, but the data is already there,
the objects are already there. So __new is called actually to construct an object, allocate
method–or allocate memory for it, make sure it’s alright. In Python, it’s used–actually
just for implementing immutable types because if you have an __init to set attributes, it’s
too late because the types who are to be created so it can be immutable if you can assign to
it in __init. So you need to do it in __new. And this is the example of a staticmethod
that shouldn’t actually be a staticmethod. It cannot be an instancemethod because its
job is to create the instance, so there’s no instance to be a method of it. So, Guido
made it a staticmethod before he added classmethods in the development cycle of Python 2.2, I
think. It could have been a classmethod but, well it’s too late now. It’s the staticmethod
that takes the actual classes’ first argument and you need to pass it around explicitly
whenever you call the super classes new. When you want to actually implement __new, you
generally always call object on __new or your super class __new to do the actual location
because there is no other way to allocate memory in Python. However, your __new method
or staticmethod can also return existing types, existing objects, you can just return any
object you want from __new whereas __init, either has to return self or return none because
there is nothing else–you can’t actually change whatever it returns, __new you can
return whatever you want and that’s the result of instantiating your class. There are–there’s
one caveat there, when you return an instance of your own class, whether it’s an actual
instance of your class or a sub class of your class, the __init is still called, even if
it’s an already created object. That’s because python can’t know that you’re __new is returning
an old object or a new object. So it always calls __init. Of course, if you return something
that’s not your type, that’s not a sub class of your type. It knows that it’s already been
initialized, so it doesn’t call under __init. So, here’s an example of an __new, WrappingInt
which is an immutable type in Python. We set slots to the empty list so it doesn’t get
arbitrary attributes. And then in __new we take the value which is whatever our constructor
was called with. We modular it so it doesn’t go pass 255 or 256 or whatever. And then we
create–we actually create self by calling the parent class method. As you can see here,
it’s a staticmethod because, even though we’re calling it on a class, we’re passing the class
we were passed as well exquisitely. Any questions so far? Yes.
>>You do you make a—-an object of view [INDISTINCT]. How do you define class? [INDISTINCT]
make sure if it is immutable.>>WOUTERS: How do you create class and make
sure it’s immutable? By not providing any of the things that are–that mutate to the
object. So, for instance, this is an easy example because Int is its own value, so you’re
not storing anything in the value. We don’t have—-we don’t accept arbitrary—-attributes
and we let our parent create the object and it’s done. If you want to not sub class ability
to immutable type, you have to do a lot of more work because you need somewhere to store
the data and then provide properties to read the data but not write the data. That’s basically
how you create. So, you do the same thing as here and you have some magic in here that
sets a hidden variable basically, that then properties can’t get out but nobody else can
get right access to. It’s not easy and it’s not, usually not worth it. Mostly Python classes
just are–just implemented in terms of existing types and if you want an immutable type you
either want something that is int like, string like, or tuple like and you can just sub class
in string or tuple and be done with it. Any other questions? Alright, is there any interested
in Metaclasses? So I mentioned them–alright. So, Metaclasses are these theoretically, satisfying
and mind boggling concept where you can draw this graphs between what the classes and it’s
Metaclass and what the class of the Metaclasses and then what instance-—where object its
an instance of the–the general ideas that the Metaclass is the class of a class. It’s
the type that are classes, it’s whatever implements a class. And of course the Metaclass is an
instance of something, so the Metaclass has a Metaclass and they are all in Python, the
base Metaclass is called type. And type’s type is type. And type is actually an object;
the parent of the type of—-parent class of type is object. All objects have a base
class that’s object, so you can see how it gets confusing. Of course, the type of object
is type, so you can draw very funny graphs that way, but it’s all, you know, it doesn’t
matter because in python everything is simple and you can just–you can just say, the class–the
Metaclasses, the class that creates the class. So, of course it doesn’t apply to type or
object because they are created secretly in python by magic, but every other class you
define is Metaclasses that whose job it is to create the class from that name space that
is the cold walk of the class. So, we go [INDISTINCT], yes well, if we go back to all the way up
to the class, here, this is all done before the Metaclasses is even looked at. And then
when this piece of code, the blue parts and the green parts are all compiled and nicely
wrapped up in the name space executed, nicely wrapped up in the name space, then that dict
is passed to the Metaclass along with a name and the parents whatever you want to sub class.
And it says, you know, “Create me a class and return it.” And then the result of that
instantiation is your new class.
So, it’s practically simple. And whatever you want to use it for in Python you can use
it for the stuff that you normally define in a class to define how an instance of the
class behaves. You can do the very same thing with a Metaclass and it’ll define how the
class behaves. So the magic that creates, for instance, functions which is hidden in
the class and stuff that calls descriptors which is hidden in the class is actually called
by get at or get attribute which is—-I should probably should have mentioned. __new and
__init are called to construct the class in to initialize the class, that all happens
to same way that you would expect. So you can overwrite them and you can do as many
things as you want, the thing there most useful for is post processing a class just doing
some wonderful renaming or mangling or other stuff with a class after it’s been defined,
before it’s visible to any other Python code without requiring an explicit post processing
step. As I said, you can do evil stuff in get at or get attribute if you want. It’s
probably not worth it. It’ll just make your code much more complex. So here’s a Metaclass
example in Python. We–sub plus type because it’s convenient and I think it’s necessary
as well. You define in __new which is the staticmethod as usual. I forgot to mention,
you don’t actually have to explicitly make new staticmethod but you can, if you want
to. The under–the Metaclasses __new gets passed—-the class that is actually ourselves
because it’s a staticmethod of ourselves or a class method of ourselves. The name of the
class that needs to be created, the basis which is the tupule of the basis that are
named in the class statement and the dict that is the name space that–of the code object,
it’s just a dict with attributes. So what we do here is we go over all the attributes,
we skip any of that start with __ because you don’t want, by accident do the wrong thing
with the magic methods. And then we name mangle whatever attributes are leftover by appending
or prepending Foo under–to them to make them look funny, we delete the original attribute
and then we–at the end we call the base class new with the same arguments but with the modified
attributes. And then to use it, to use the Metaclass you have to explicitly tell Python
to use a different Metaclass than whatever is the default which is type. You do __metaclass__
is MangledType in the class dict or if you want at the top of the module. Now Metaclasses
is getting heritage, so if you sub class mangled class you automatically get the mangled type
as Metaclass. If you want to sub class and have a different type your—-or different
Metaclass–your Metaclass has to be a sub class of your super classes Metaclass. So
if I wanted to have a more mangled class with a more mangled Metaclass, I have to sub class
mangling type to get a more mangling type and have that as my Metaclass. So, any questions
there?>>Are mangle type [INDISTINCT]?
>>WOUTERS: Yes, sorry that’s a typo. The—-it should say mangling type at the bottom and
not mangled type. Yes.>>I think I remember that django is a Metaclass
at the bottom [INDISTINCT], is that true?>>WOUTERS: Yes.
>>Do you know how would it do?>>WOUTERS: Yes. I don’t have an example right
now. I have some django code and it’s very interesting how it works in django. Sorry.
The question was django uses Metaclasses and how is that done. Django has an API where
you define a class with various attributes to describe your data model. And then you
can have some extra classes inside the class to describe admin options and display options
and whatever. What it does is it actually–just like this, it goes over all the attributes
that were the result of executing the code inside the class statement and examines them.
And the order doesn’t matter to django for the most part but where it does order it abuses
the fact when compiling a class for executing a class, it executes top–top to bottom. So
it calls a field, let me see, you do–a field is a particular type and the type is a–is
an instantiation of a class. So you say, “field is in field” and “field two is car field”.
And it keeps track of–in what order are those type were created by keeping a global counter
for instance, so I’d know which order the fields are in the class statement. That’s
about the only magic thing that the django thing does. And then, for the rest is just–is
just examining whatever’s in the others dict that gets passed in the Metaclass to write
down the sequel that’s whatever the database backhand needs to store and whatever options
there are, etcetera. Does that answer your question?
>>Yes.>>WOUTERS: Alright. Any other questions?
All right. So, I’m going to cover multiple inheritance unless everyone here goes, “No,
no, don’t ever use multiple inheritance,” alright? So, multiple inheritance in Python
and in other languages is something that’s frequently argued whether it’s good enough
or sane or insane. Well, it’s generally not a good idea but occasionally, especially in
python, it could make stuff a lot easier. New style objects have a different way of
doing multiple inheritance in that they usually C3 method resolution order which is an algorithm
described, I think in a Dylan paper, describing how to do when you have multiple inheritance,
in particular, involving diamonds where multiple super classes inherit from the same super,
super class, how to handle that correctly. And the algorithm is pretty simple, it just
goes a depth-first, left-to-right search through all the classes, but then it removes all duplicates
except the last one. So, if a class appears two times, it’ll be visited, not the first
time it appears, but the last time it appears. And in Python, we also have a convenience–contingency
object called super which can help you continue method resolution… All your parent classes
are visited after you’re visited are therefore visited before you’re visited. But, you might—-they
might not be the next phase class, if you’re super class might not be the next super class
to be visited and your sub classes might not have been visited right before you. That’s
rather important to realize. So calling your base class method directly saying “my parent.foo”
whatever, is never right. Because there’s always going to be some situation where that
will do wrong thing and skip classes when visiting methods. So, the way to do it in
Python is you have a single base class with a full interface of whatever your tree is
going to implement, so that any object can always call whatever method they want to call
within that tree on any other class within that tree. But–in usual cases those–those
functions will probably be do nothing functions. They shouldn’t raise errors because then you
can’t safely call them all the time but they should—-if anything, they should do nothing.
The signature of those methods should never change because you cannot know which order
of classes will be called in. If you have to change a signature of a method in a particular
part of the multiple inheritances tree, you should have a second master basically, in
the multiple inheritance tree and make sure that it’s a separate section of the multiple
inheritance tree. And then you should have-—you should use super everywhere you want to have
anything from a base class, anywhere. And that can be annoying because all of your code
has to follow it in all of the classes and you’re usually not the only one developing
all those classes, so you have to convince everyone to use super everywhere. And as I
can show here, using super is not actually convenient right now. You call super, passing
this class, the current class the class you’re coding in, and the instance or class if you
want-—if you have class methods that you were passed, you were called on. That’s–that
returns a proxy object and then you can, you can use that as if it was self to call the
original method. It’s not too convenient and I hope we’ll have syntax for doing super calls
in python 3.O, maybe sooner, but, I’m not holding my breathe. Any questions about these?
All right. I’m going to cover Unicode then if we have time.
>>[INDISTINCT]>>WOUTERS: This [INDISTINCT]
>>[INDISTINCT]>>WOUTERS: So, Unicode, it’s somewhat longer
topic though.>>[INDISTINCT]
>>WOUTERS: No, just twice as long as the previous topic or whatever. So, Unicode is
a standard for describing characters. The way byte strings describe bytes, you say,
“A is represented by 65”, a Unicode you say, “A is code point 65” and–there’s no relation
between Unicode as such and bytes on the disk. In Python that means the old store type that
string holds bytes, it’s a byte string, we call it a byte string nowadays. And Unicode
object or Unicode strings they hold characters which is for ASCII is actually the same thing.
But for non ASCII, it’s entirely different. A Unicode has no on disk representation as
such, so if you want to store Unicode in disk or even in memory, you need to use an encoding.
Python internally uses either UTF-16 or UCS-4 encodings or UCS-2 and UCS-4 depending on
how you exactly define what Python does. But it uses either 2 or 4 bytes to represent each
character, whereas, bytes strings they always use 1 byte for every character or byte. When
you had a byte string and you want to have a Unicode string, a Unicode object, you have
to decode the byte string into Unicode. And if you–if you have a Unicode string and you
want to store it somewhere, you want to pass it over to the network or when you write it
to disk; you have to in code Unicode into bytes. A convenient encoding is UTF-8, and
some people confused UTF-8 with Unicode, for instance, Postgres, the database, has an encoding
code Unicode which is actually UTF-8 and not Unicode. It’s one of those mistakes many people
make but UTF-8 is not Unicode. UTF-8 is a Unicode encoding and it’s Unicode encoding
that looks like ASCII to all Americans who are people who don’t care about accents or
funny characters. But it can actually encode all of Unicode and it does so by basically
screwing Chinese and Japanese people by having all of their characters be like 7 or 8 different
bytes. So, in Python, Unicode is pretty convenient, except for–it mixes the strings. You can
have Unicode literals which look just like string literals except you have a couple of
more escapes besides the normal backslash X escapes and backslash O, and backslash 0,
I mean. You can have backslash U which is a short Unicode escape and a backslash capital
U which is a long Unicode escape. The short U has, as you can see, 2 bytes and then the
long U has 4 bytes. And the long U isn’t very necessary until you start working with the
higher code planes that were added last to Unicode and anything. Also, instead of Chr,
the heart to create a single character, you have Unichr which creates from a number or
any character, any Unicode character. And we support in the compiler at compile time
Unicode names. You can use backslash and then in the curly braces, the name for any Unicode
code point. Then the code defines all these names, we have them all in the interpreter
at compile time, that actually results in a single character Unicode string with a euro
sign in there. It’s a single character Unicode, but when you encode it in—-in an encoding
that supports the euro sign, it’ll look actually an entirely different character or multiple
bytes or whatever. If you want to do this name look ups in reverse, if you have a character
you want to look up in the Unicode name, the Unicode data module does that for you. If
you have the name and you want the actual character, Unicode data does that, too. So
Unicode objects, they behave exactly like strings, it’s very convenient, you can slice
them and you actually slicing characters instead of encoded data. The length is right, you
can iterate every character, everything is great, except when they mix with non-ASCII
byte strings. When they mix with ASCII byte strings, Python will automatically upgrade
the byte string into a Unicode object which is with the ASCII encoding. So that works
for ASCII bytes strings, but if there’s a funny character in the byte string that’s
not ASCII, it’ll blow up, because it tries to interpret it as ASCII and it sees that
it’s actually not ASCII and it doesn’t know what you want to do with it, so don’t do that.
Another problem with the python approach is that the decode and encode methods of strings
and Unicode objects are generalized. They don’t just do encoding to byte string or decoding
into Unicode object, you can actually convert strings into strings and byte strings into
byte strings and integers into whatever you want or two integers. It’s–it’s inconvenient
and I’m not entirely sure if that should be fixed or not, but it’s inconvenient when you
only care about Unicode. On the other hand they do have convenient features. So, using
Unicode in python is very simple. Never mix Unicode objects and byte strings. It’s a very
simple rule, if you do that everything would be great, except of course it’s not always
easy not to mix byte strings in Unicode. If you write libraries, you might get pass the
string when you don’t expect it or you get–might get pass a Unicode object when you don’t expect
it. If you have your application–if you write your application, you have to make sure that
anywhere you get a string; you get it as Unicode object or you get it as a byte string and
you translate it yourself. So decode–the best way to do it is decode byte strings into
Unicode when you get them and in code Unicode into whatever output you have when you’re
outputting. And of course you have to remember to use the right encoding, so you have to
remember what the encoding would be like when you get input or should output and there’s
no way to detect and encoding. Because it’s just bytes and there’s no marker in there
that says “this is UTF-8” or “this is a latin-1” or whatever, or UTF-16 for that matter. And
it all looks vaguely familiar when you’re actually looking at the bytes, but that might
not mean that’s the correct thing to decode with that encoding. Fortunately, if you can
figure out which encoding to use Python does have some conveniency functions and modules,
the codecs module, in particular. Codecs module has an open function that behaves like the
builtin open function, except it takes on encoding and it’ll automatically decode data
as a—-as if you just read from it. So when you read from codecs with open objects, you’re
actually reading Unicode and you write Unicode to it it’ll automatically encode it into whatever
encode you passed in. If you want to wrap existing streams, like sockets or files you
partially read, you can use codecs to get reader or get writer to transform the on the
file transform the string from byte string to Unicode or the other way around. And lastly,
when you do write stuff like it and you’re debugging your code and there’s some problem
with mixing Unicode and byte strings, pay attention to the exceptions because there’s
two exceptions you could get; there’s Unicode decoder which you get when decoding a byte
string into Unicode goes wrong and there’s Unicode encoder which is the other way around.
And if you use str.encode, so you’re trying to encode a byte string into a Unicode encoding,
what it’ll actually do is silently try to decode the string first into Unicode and then
encode it with the encoding you gave it. So, that actually tries to apply the default encoding
which is ASCII to str, and even though you call str at encode, you will be getting a
decode error if it ends up that str is not an ASCII string. So, that was it. That was
all my topics I’m glad we went over all them. Here’s some more information if you want;
descriptors, metaclasses and super all describe in Guido’s original descrintro tutorial for
python 2.2 which is still very relevant. Iterators and generators are well described in a–under
kooklinks tutorial on functional programming, you don’t have to follow the whole thing if
you don’t like functional programming, but the parts about iterators and generators are
very good. And if you want to learn about writing C types, the standard documentation
is a great source, as well as the Python’s source, the actual Python modules are all
in the same API and they’re very readable source even if I do say so myself and it’s
highly recommended. And always the python standard library and the Python C code are
all great sources. That was it. Any more questions?>>[INDISTINCT] somewhere, that we can get
up?>>WOUTERS: I can put it up somewhere.
>>How about the previous presentation about the upcoming [INDISTINCT].
>>WOUTERS: Sorry?>>The previous presentation, I guess, it
was [INDISTINCT] wheel about the feature of Python? Is there any record of that somewhere
that we can look at up?>>WOUTERS: There’s like a four or five different
movies, sorry.>>It’s on my laptop you can upload it.
>>Okay great.>>WOUTERS: Any other questions?
>>What’s a [INDISTINCT]-—what’s the good resource
for a sort of module import resolution? You know, like, when you’re changing—-when you’re
moving Python [INDISTINCT] from one place to another. [INDISTINCT] and is there, like
a, standard way of how you do of all that.>>WOUTERS: So, you mean from one string to
another?>>From, you know, one string to another or
what [INDISTINCT] code base [INDISTINCT]. You start mixing it [INDISTINCT] and things
like that or in like [INDISTINCT] libraries. It’s got to be like when you do [INDISTINCT]…
>>WOUTERS: Usually a byte store…

100 thoughts to “Advanced Python or Understanding Python”

  1. @ThunderAppeal

    I've seen perl programs which look like someone banged there head on the keyboard. I've actually SEEN them. now you *can* violate python to make it do the same thing, I fully agree, but the language doesn't practically beg you to do it. older style perl did. the newer perl stuff doesn't but it's still not as nice as python, though I do enjoy lua for some of the similar work, but python is gaining ever more packages for my job so I fully expect it to be the way I continue on.

  2. @addmoreice You've contradicted yourself so you cant say with any sort of integrity that Python is more readable than Perl. Perl doesnt 'practically beg you' to do anything and you've already admitted to the fact that anyone can write ugly Python code. What you've 'seen' doesnt mean anything.
    We wont talk about your horrible grammar, which by itself explains why you have no idea what youre talking about.

  3. @ThunderAppeal

    wonderful argument.

    I point out that perl can be easily abused…you ignore that.
    I point out that python can be abused as well…but usually isn't, even by newbies. You ignore this as well.
    My personal experience (which, I admit is purely anecdotal and so not worth much) also doesn't count at all.
    You ignore my point about packages (you could have brought up perls huge package listing but no, evidence is too hard), and finish it of with an ad hominem.

    I now MUST agree =-P

  4. @addmoreice You continue to contradict yourself and at the same time pat yourself on the back for having made a 'strong argument' that backs up your position. The only thing you just barely rise up to the occasion for is that you admit that your personal experience are highly subjective. But thats all you say that is true, everything else are talking points du jour that you've been fed and have gobbled up.

  5. @addmoreice Python is a nice little language when everything already has some kind of shape or form, however in order to message that data with python to prepare it to be processed python is a nightmare.
    Every little thing a module has to be imported, there has to be a regexp module imported a strings module imported.While its very nice that scope is enforced automatically in order to manipulate truly manipulate data you have to get comfortable with list comprehensions.

  6. @addmoreice Even Mark Lutz admits that list comprehensions are ugly, frankly I'm quite fond of them. But again they only work when the data is nice and pretty and neat. Otherwise you have to labor under how python interprets the data and work with it in a very structured way. Additionally python does not have types, it has containers, types are an after thought. Containers take precedence over everything. Perl on the other hand was designed specifically with text processing in mind and

  7. @addmoreice And despite what you might have led yourself to believe all of the internet is based on text processing. At the core of data processing is text processing, hence perl has types and with those types you build your own containers. So based on these observations I've made I know that python is no more readable than perl and in many cases much less so. Which leads me to conclude that perl is not unreadable, you have a hard time reading. The problem is not perl it is you.

  8. @ThunderAppeal

    I was going to post a snippit from a standard module. a snippit that looks like keyboard barf. it literally looks like someone smashed there face along the number row….but i cant because it causes an error in the text handling in youtube.

    i can write 'readable' code in the whitespace language, but no one would claim it's readable or understandable. The language is ugly for it.

    Perl CAN be done in such a way that is clean and understandable. but it's EASY to make a cluster.

  9. @addmoreice You clearly have not actually looked at third party software written in python. You can look at Django, Satchmo, any other number of 'plug ins' to find 'ugly code'.

    Again, the problem is you mistake style for substance.
    A more intelligent way to judge if something is 'ugly' is to first understand what is happening underneath the code. Something you are clearly are unable to do, with python everything is a guessing game. For example whilst

  10. @addmoreice Whilst reading python code you happen across a function (ie func()) and if it isnt predefined in the code somewhere you have to hunt down its definition somewhere in the python module directory. It may be as easy as help(func) at the python prompt but you have to do is so often. Even then the python docs are crufty and crusty, too much is taken for granted. Something that python presumably was supposed to move away from when they copied elements of Perl.

  11. @addmoreice 'Its easy…' That is possibly the lamest lie to hide behind, its something I would expect from a junior in high school to come up with. You might be one, I dont know, if thats true then you should learn some more before you start yacking.
    As for this statement: 'I can write 'readable code in the whitespace language, but no one would cliam it's readable or understandable.' What 'whitespace' language are you talking about? Python comes to my mind, where whitespaces 'count.'

  12. @addmoreice But again, lets judge 'ugliness' and 'pretty' in terms of functionality. As I said before, with python, whitespaces and 'containers' reign supreme. To me that is ugly because data does not present itself in a pretty structured way. But thats the problem with computer 'scientists' today, they expect data to be pretty because they were trained in a data clinic. Where there are on surprises and data always shows up 'pre-fabbed'. Perl however takes data in and leaves the parsing up

  13. @addmoreice And leaves the parsing format up to the programmer. Handing control back to the programmer like that is beautiful.

  14. @ThunderAppeal

    Yes, it is beautiful. Just as lisps 'data is code' paradigm is wonderfully both powerful and simple in concept and structure.

    It's also possible to be abused in ways that make me shudder considering how to debug/correct. Yes, it allows for wonderful nearly poetic constructs with simplicity and use.

    But when I'm programming my VCR I don't want Haiku. I want plain simple English. I *really* don't want *bad* Haiku when programming my VCR.

  15. @ThunderAppeal

    My complaint deals with what I have to work around and fix every day, all day long. Not with how wonderful and detailed the structural constructs I can produce with this system will be. I would rather read a badly documented but clearly coded hack that takes up ten pages and gets the job done in a slow and plodding manor, then a ten line haiku that does things with wonder and panache….but also takes me 2 hours to dig through to find the misplaced '

    Noobs don't doc >.<

  16. @addmoreice No docs may be the noobs fault, they get 40-45% of the blame, the rest of the blame falls on their managers.

  17. @addmoreice Again, the problem is not language the problem is *you*. You are mixing up programming languages and documentation. I was trained by my boss to document everything I did, I had to take responsibility for all of my work. Today, 'programmers' like you snivel and whine and bitch and moan like a bunch of little babies. Programming languages dont play nearly as much of a role in 'making a programmers job easy.' as much as you imagine it does. Either someone knows the subject or they

  18. @addmoreice or they dont. Your complaint keeps coming back to the fact that you are simply incapable of understanding any code that isnt your own. Perls or pythons readability have no bearing on your ability.
    You've never had to wade through 3rd party software written in python, so you cant definitively say that you know what youre talking about.

  19. @ThunderAppeal

    "Again, the problem is not language the problem is *you*."

    Yes, in the discussion on 'is the language easy to abuse' part of the equation is the language, part is the user. Very good. here is a cookie.

    Is binary easier for a human being to understand then perl?

    If NO then go produce your programs in binary directly. tell me how well that works for you.
    If YES then you agree that language x can be more difficult for people then language Y.

    Now, are there ways to asses this?

  20. @addmoreice

    If NO then you are arguing that 'more difficult' has no metric and is a meaningless term, in which case again go back to programming in binary.

    If YES then you agree that there IS some metric. One such metric is to take a moderately trained programmer (define this how ever you like) and ask them to program a moderately difficult task (again however you like).

  21. @addmoreice

    Then simply ask these same groups of moderately trained programmers and ask them to modify those previous programs in whatever way you think is appropriate.

    How easy is it for any of those moderately trained programmers to read the resultant code and understand it? This would seem to be a meaningful (though fuzzy) metric of ease of language understanding.

    Perl bombs this test, and badly. Lots of bad perl, some good.

    I use perl, I've debugged 3rd party software in it. I know it.

  22. @addmoreice

    Python doesn't do this perfectly by any means.

    The 'oh it's a script and lets add some more' problem is rampant in python (and perl, and bash scripts).

    But some of the many OTHER issues are far less.

    Your responses are pretty much devolving into Ad Hominem and false assertions my argument is based on some reasoning.

    Use the right tool for the right job. If you have to work with a lot of monkeys lean towards python. If you know YOU are doing it, use perl.

  23. @Testerer004
    Well, you see, most people don't wait till there 20 an in college to learn programming. I know most of my friends started learning C and creating games in middle school. So, I would say a lot of us are most likely extremely young.

  24. @addmoreice The version I'm using is "Youre a fucking imbecile if you have to program to remove white spaces"

    This shit is tedious with you. I've already explained the drawbacks in python and they are pretty big.

  25. Strange, you didn't address any of the core of my comments. You ignored the points I made about ease of use for both experienced and beginner programmers (providing a trade off on skills of employees), ignored my point about 3rd party issues…and then deleted a bunch of spurious ad hominem attacks. The only point you DID make I fully agreed with you on….and pointed out how it's an unusual circumstance (though a valid one).

    Joust at another windmill, tell me how that goes.

  26. @addmoreice Youre just repeating everything I told you earlier on. You dont even bother rereading any comments except your own.
    And now youve regurgitated everything I wrote to you.

    Please repeat yourself some more, I'm sure you feel proud of yourself when you do.

  27. @ThunderAppeal

    I repeated the salient points of the discussion yes….and notice your contribution has been 'nuh uh' and 'your stupid!' comments? I agreed where you where right, and showed the places where you where wrong with evidence.

    It's called an argument. try it some time.

  28. @addmoreice Dear Moron, you dont have any 'salient' points the only thing you brought up was how to remove white spaces using perl and python. If that is the bulk of your programming experience with both languages, which judging by the way you are stalling and ignoring everything I said *prior* to your childish little example, then you are in fact an imbecile. Since it seems that you are willing to agree with that I'm willing to call it a draw if you are.

  29. @ThunderAppeal
    ad hominem ad hominem….

    I said, yes. perl is easier for parsing. i agree. but perl can look like keyboard headbang spam, python does not. perl makes it real easy for a new programmer to do a lot of damage. python less so.

  30. what the fuck, i want to learn about PYTHONS not fucking computer programing

    shit that cock fucks can copyright the name PYTHON for a computer program

    why cant programmers be original, wtf, poison search engines is all this kinda trademarking does… bullshit

  31. @HOORVT what so because your some weirdo who wants to look at pythons a long-standing highly powerful programming language is not allowed?

  32. @marshmalloney you waste my fucking time, yes ofcoarse someone searching for PYTHONS wants to LOOK AT PYTHONS

    unless they are some little programming shit head

  33. @marshmalloney this group of cock fucks copyright the name PYTHON

    for that any snake lover should be very angry

  34. @HOORVT why not get pissed that apples have been copyrighted, and so has the feeling of glee, or the humiliation of rick santorum

    and what about the monty python flying circus, which is what python programming is named after

  35. So by your logic, James Gosling had no right to name the language he created "Java" simply bc its a type of coffee? Or Yukihiro Matsumoto was wrong to name his language "Ruby"? You, sir, are a dumbass.

  36. You sir, are very confused. You can still use the name Python in all kinds of contexts. Nothing has been stolen from the English language. You can open a Python Cafe, or launch a Python breakfast cereal, or start a Python political party. You just probably shouldn't start a new programming language and call it Python, case Guido already did that.

  37. You are to write a program that calculates the ascii numbers when given any decimal number between 0 and 255. The program is to contain a function call deciToAscii(number) that accepts the decimal argument and returns an ascii number as an eight character string.

  38. at least the wiki for snake comes up first in google for me when i search.. when i search python I get the wiki for python programing language…

    and no knock on the actual language, I guess we all use it if we use linux…

  39. *code for two hours while listening to this*
    *come back and see that the video has only been playing for 6 minutes*

  40. Noticed how the audience burst out laughing when the instructor said everything in Python is simple (while he was explaining metaclasses)?
    That's exactly the problem with many books and instructors. They use a lot of jargons that were invented when the particular programming language was invented. They should rather explain things from a user's point of view. For example, if so and so is the behavior you want, this is what you have to do.

  41. I use LInux, Blender, Gimp, Inkspace… they are all powered by Python no? is it ironic? I didnt say Python programming sucked, was just commenting about the name… and my comment is a true fact! the fact that the bunch of bigots here cant stand it, is not my fault, it is just proof of the narrow minded POS's I have to deal with on a regular daily basis.

    I like the programming, I was not fond of the name, UNDERSTAND????!!!!????!!!????!!????!

    U R Proof you dont need a brain to program!!

  42. I don't see your point. "i want to learn about PYTHONS not fucking computer programing" Is irony, because it's nested. Think Inception, if you must. Some dislike the name, that's perfectly acceptable. But there is no logical motivation behind your post, and you lack of formal English begs me to question whether it is I that is struggling in the area of mental competence.

  43. No, you cursed the creators for "making it property". You're absolutely wrong, though, because "Python" is not copyrighted. It's an unregistered trademark, which is explicitly defined as NOT property.

    So, in our wonderful world, biologists, British comedy troupes, and Dutch programming language designers all have equal freedom to share the same name for their concepts.

    But then there's HOORVT, shitting all over their freedom and trying to give monopoly rights to snakes. Shame on you.

  44. It's remarkable how a failed google search a year ago keeps you raging about intellectual property to this day. What I don't understand is how you don't see the hypocrisy and contradiction. You say words are "free", yet want them to refer to only the one thing that you happened to be looking for.

    "early bird gets the worm" — in other words, words are the early bird's property, not free. You curse "content creation controllers", yet you're the one advocating the elimination of derivative works.

  45. 13:00 I was thinking… What about the nonlocal statement?! Then I realized this talk was in 2007 haha.

  46. Python is by far the most interesting language ever created! Sure there are high end langs like Scala and the Lisp family but Python is just soooo…. FUN! Is it wrong to have fun while working??

  47. Thank you for bringing back the BASIC simple type syntax and leaving out the damn { } crap. BASIC structure as well, for someone that is a VB programmer this will be second nature. I get enough torture from the other { } languages.

  48. lol. some dumbass pearl programmer, talking about how much python sucked, actually made the claim earlier in this thread: that machine code is really character sets, but not base-2.

  49. Is this tutorial outdated? I haven't watched it yet. I'm currently learning Python and wanted to learn some Advanced stuff. Since this video is made in 2007, I'm not sure how much is outdated. Can someone help me on this, please? Thanks.

  50. Computer Science people really do not seem to get or learn the lessons of Computer Science. For example, this is an advanced Python talk … so why does he waste time at the beginning explaining what Python is? We know or we would not be here or watching this. Everyone does this, so subtract 5 minutes from every talk ever done on every subject and realize what an insulting waste of time this is. People to do these talk for themselves. They want to be seen as knowing something, but they don't really want to get to the point, they just want the status. That's my hypothesis anyway. Just start in with an overview of what you are going to do … and then get to it in an ordered manner.

  51. 1:07:00 seven or eight bytes per character?
    That is non-sense.
    The farthest character in the Unicode table (0x10FFFF) can be encoded with just 4 bytes.

  52. Please anyone help. I lost my password to my ETH wallet and am trying to install/run this python program. When I try to click on the downloaded file with the python icon, it opens then quickly closes. When I try to run the file through python ver 3.0 (command line), i keep getting errors. Am I doing something wrong or is the file corrupted? Please someone help me will give you some ETH if I succeed in cracking my wallet. Here is the python file: https://github.com/Isaacdelly/Plutus

Leave a Reply

Your email address will not be published. Required fields are marked *