Python Review Surprises

I had to use Python for work again after a long time off, and during the time off from Python I had gotten used to Go.  A few of the things I had forgotten or never known about Python surprised me when I returned to it.

No Assignment Operator in Closures

It’s not like I make closures all the time, but I do take lexical scoping for granted when I’m in a high-level language.  This is a habit left over from perl.  In perl, you can create a closure without any special knowledge or techniques, just because lexical scoping works the normal way.

bash$ ./closure
3
4
bash$ cat closure
#! /usr/bin/env perl -w

use strict;

sub closuretest() {
    my $n = $_[0];

    return sub {
	return $n += 1;
    }
}

my $fn = &closuretest(2);
print $fn->() . "\n";
print $fn->() . "\n";
bash$ 

If you try the same in python, though, it won’t work, because the innermost subroutine gets its own namespace—so far so good—using an outer-scope variable as an R-value works as expected—OK—but because assigning to a variable inside that namespace creates a completely new variable, you cannot just use the assignment operator on a variable from the outer scope.

Using the += operator means you’re assigning (creating a new variable) but also reading (a variable in the new namespace that hasn’t been created yet), so you get an error. So because the variable appears as both R-value and L-value, Python won’t assume you’re trying to use the variable from the outer scope that already does exist, even though it would have done that if the variable only appeared as R-value.

bash$ ./closure-nope.py
Traceback (most recent call last):
  File "./closure-nope.py", line 10, in 
    print fn()
  File "./closure-nope.py", line 5, in inner
    num += 1
UnboundLocalError: local variable 'num' referenced before assignment
bash$ cat !$
cat ./closure-nope.py
#! /usr/bin/env python2.7

def closuretest(num):
    def inner():
	num += 1
	return num
    return inner

fn = closuretest(2)
print fn()
print fn()
bash$ 

There are ways to use the assignment operator and create a closure, but not without using some Python techniques that would likely baffle your peers. I doubt this is a big deal, but it was a surprise about Python for me.

User-Requested Name Mangling

There is a name-mangling feature you get when you use two leading underscores and at most one trailing underscore in the variable name. The Python Tutorial states that this feature allows a class to protect itself from its descendants when there’s a method it needs to rely on for its own integrity. There’s no other way in Python for the base class to prevent derived classes from overriding it. (Strictly speaking, it doesn’t prevent it. But it does make accidental overrides less likely.)

bash$ ./mangle.py 
hi B
hi B
hi A
{'__module__': '__main__', '__doc__': None, 'fn': <function fn at 0x1004960c8>, '_B__fn': <function __fn at 0x1004967d0>}
bash$ cat mangle.py
#! /usr/bin/env python2.7

class A():
    def fn(self):
        print 'hi A'
    def __fn(self):
        print 'hi A'

class B(A):
    def fn(self):
        print 'hi B'
    def __fn(self):
        print 'hi B'

b = B()
b.fn()
b._B__fn()
b._A__fn()
print b.__class__.__dict__
bash$ 

See how Python has a special case for method names that start with two underscores and end with at most one underscore? It prepends ‘_’ + self.__class__.__name__ + ‘__’ behind the scenes. Surprise!

One-time Default Keyword Parameter Evaluation

What do you think this default.py will print? Maybe three lines of ‘[0]’?

bash$ cat default.py
#! /usr/bin/env python2.7

def fn(mylist=[]):
    mylist.append(len(mylist))
    print mylist

fn()
fn()
fn()
bash$ 

You might think that because each invocation of the function creates a fresh namespace, you’d get a new empty list assigned to mylist each time it is called with no parameter.

That’s not how it works, though. There’s persistent state. See:

bash$ ./default.py 
[0]
[0, 1]
[0, 1, 2]
bash$ 

Where’s the state? It appears to live in an attribute called func_defaults. Here I am inside the interactive REPL:

>>> def fn(mylist=[]):
def fn(mylist=[]):
...   mylist.append(len(mylist))
  mylist.append(len(mylist))
...   print mylist
  print mylist
... 

>>> fn()
fn()
[0]
>>> fn()
fn()
[0, 1]
>>> print fn.func_defaults
print fn.func_defaults
([0, 1],)
>>> 

This surprise also appears in the tutorial in the form of a warning: Default Argument Values.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s