Hidden features of Python [closed]

Question

What are the lesser-known but useful features of the Python programming language?

• Try to limit answers to Python core.
• One feature per answer.
• Give an example and short description of the feature, not just a link to documentation.
• Label the feature using a title as the first line.

Quick links to answers:

• Argument Unpacking
• Braces
• Chaining Comparison Operators
• Decorators
• Default Argument Gotchas / Dangers of Mutable Default arguments
• Descriptors
• Dictionary default .get value
• Docstring Tests
• Ellipsis Slicing Syntax
• Enumeration
• For/else
• Function as iter() argument
• Generator expressions
• import this
• In Place Value Swapping
• List stepping
• __missing__ items
• Multi-line Regex
• Named string formatting
• Nested list/generator comprehensions
• New types at runtime
• .pth files
• ROT13 Encoding
• Regex Debugging
• Sending to Generators
• Tab Completion in Interactive Interpreter
• Ternary Expression
• try/except/else
• Unpacking+print() function
• with statement

Answer 1

Get the python regex parse tree to debug your regex.

Regular expressions are a great feature of python, but debugging them can be a pain, and it's all too easy to get a regex wrong.

Fortunately, python can print the regex parse tree, by passing the undocumented, experimental, hidden flag re.DEBUG (actually, 128) to re.compile.

>>> re.compile("^\[font(?:=(?P<size>[-+][0-9]{1,2}))?\](.*?)[/font]",
    re.DEBUG)
at at_beginning
literal 91
literal 102
literal 111
literal 110
literal 116
max_repeat 0 1
  subpattern None
    literal 61
    subpattern 1
      in
        literal 45
        literal 43
      max_repeat 1 2
        in
          range (48, 57)
literal 93
subpattern 2
  min_repeat 0 65535
    any None
in
  literal 47
  literal 102
  literal 111
  literal 110
  literal 116

Once you understand the syntax, you can spot your errors. There we can see that I forgot to escape the [] in [/font].

Of course you can combine it with whatever flags you want, like commented regexes:

>>> re.compile("""
 ^              # start of a line
 \[font         # the font tag
 (?:=(?P<size>  # optional [font=+size]
 [-+][0-9]{1,2} # size specification
 ))?
 \]             # end of tag
 (.*?)          # text between the tags
 \[/font\]      # end of the tag
 """, re.DEBUG|re.VERBOSE|re.DOTALL)

Answer 2

enumerate

Wrap an iterable with enumerate and it will yield the item along with its index.

For example:

>>> a = ['a', 'b', 'c', 'd', 'e']
>>> for index, item in enumerate(a): print index, item
...
0 a
1 b
2 c
3 d
4 e
>>>

References:

• Python tutorial—looping techniques
• Python docs—built-in functions—enumerate
• PEP 279

Answer 3

Conditional Assignment

x = 3 if (y == 1) else 2

It does exactly what it sounds like: "assign 3 to x if y is 1, otherwise assign 2 to x". Note that the parens are not necessary, but I like them for readability. You can also chain it if you have something more complicated:

x = 3 if (y == 1) else 2 if (y == -1) else 1

Though at a certain point, it goes a little too far.

Note that you can use if ... else in any expression. For example:

(func1 if y == 1 else func2)(arg1, arg2) 

Here func1 will be called if y is 1 and func2, otherwise. In both cases the corresponding function will be called with arguments arg1 and arg2.

Analogously, the following is also valid:

x = (class1 if y == 1 else class2)(arg1, arg2)

where class1 and class2 are two classes.

Answer 4

Dictionaries have a get() method

Dictionaries have a 'get()' method. If you do d['key'] and key isn't there, you get an exception. If you do d.get('key'), you get back None if 'key' isn't there. You can add a second argument to get that item back instead of None, eg: d.get('key', 0).

It's great for things like adding up numbers:

sum[value] = sum.get(value, 0) + 1

Answer 5

Sending values into generator functions. For example having this function:

def mygen():
    """Yield 5 until something else is passed back via send()"""
    a = 5
    while True:
        f = (yield a) #yield a and possibly get f in return
        if f is not None: 
            a = f  #store the new value

You can:

>>> g = mygen()
>>> g.next()
5
>>> g.next()
5
>>> g.send(7)  #we send this back to the generator
7
>>> g.next() #now it will yield 7 until we send something else
7

Answer 6

Named formatting

% -formatting takes a dictionary (also applies %i/%s etc. validation).

>>> print "The %(foo)s is %(bar)i." % {'foo': 'answer', 'bar':42}
The answer is 42.

>>> foo, bar = 'question', 123

>>> print "The %(foo)s is %(bar)i." % locals()
The question is 123.

And since locals() is also a dictionary, you can simply pass that as a dict and have % -substitions from your local variables. I think this is frowned upon, but simplifies things..

New Style Formatting

>>> print("The {foo} is {bar}".format(foo='answer', bar=42))