What does underscoring methods connote?

Question

I am relatively new to the Python language and encountered this in doing the following:

help(list)

Here is what I encountered:

__add__(...)
|      x.__add__(y) <==> x+y
|  
|  __contains__(...)
|      x.__contains__(y) <==> y in x
|  
|  __delitem__(...)
|      x.__delitem__(y) <==> del x[y]

Regarding these, what are the underscores for? Because they aren't used (to my knowledge) when you use a method normally, I'm struggling to understand why they'd take the time to write them out with underscores in the documentation.

Answer 1

Here is Guido van Rossum, the creator of Python, explaining the use of double underscores:

... rather than devising a new syntax for special kinds of class methods (such as initializers and destructors), I decided that these features could be handled by simply requiring the user to implement methods with special names such as init, del, and so forth. This naming convention was taken from C where identifiers starting with underscores are reserved by the compiler and often have special meaning (e.g., macros such as FILE in the C preprocessor).

...

I also used this technique to allow user classes to redefine the behavior of Python's operators. As previously noted, Python is implemented in C and uses tables of function pointers to implement various capabilities of built-in objects (e.g., “get attribute”, “add” and “call”). To allow these capabilities to be defined in user-defined classes, I mapped the various function pointers to special method names such as getattr, add, and call. There is a direct correspondence between these names and the tables of function pointers one has to define when implementing new Python objects in C.

Also see the Python documentation on special method names which reads in part:

A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names. This is Python’s approach to operator overloading, allowing classes to define their own behavior with respect to language operators.

Answer 2

They are special methods (not the "dunder" (double underscore) makes them special, but most often they have a special meaning). They are often called when using operators. You can overwrite the behaviour in your classes.

E.g if you have a class C, by defining

class C:
    def __add__(self, other):
        #....
        return something

you can define what should happen if you add two instances from the class:

val = instance1 + instance2

Answer 3

In this case, double underscores are used as a convention to denote special methods - methods that are implemented to support syntactic sugar purposes and other special interfaces.

http://docs.python.org/reference/datamodel.html#specialnames

Answer 4

Wrapping a method name in underscores is just a way to separate namespaces. If a method begins and ends with underscores, it's simply a convention to indicate that the method isn't meant to be used by external code.

The blurb from help(list) you posted simply means that when you use Python's syntax to say e in lst, you're actually invoking lst.__contains__(e).

Answer 5

See the Python style guide for a comprehensive explanation.

In practice:

the following special forms using leading or trailing
underscores are recognized (these can generally be combined with any case
convention):

- _single_leading_underscore: weak "internal use" indicator.  E.g. "from M
  import *" does not import objects whose name starts with an underscore.

- single_trailing_underscore_: used by convention to avoid conflicts with
  Python keyword, e.g.

  Tkinter.Toplevel(master, class_='ClassName')

- __double_leading_underscore: when naming a class attribute, invokes name
  mangling (inside class FooBar, __boo becomes _FooBar__boo; see below).

- __double_leading_and_trailing_underscore__: "magic" objects or
  attributes that live in user-controlled namespaces.  E.g. __init__,
  __import__ or __file__.  Never invent such names; only use them
  as documented.

Answer 6

To some extent, these underscores are nothing special; they're just part of the method name.

However, they are used to indicate "magical" methods, such as constructors and overloaded operators. You should not use them in your own method names. From PEP 8:

• __double_leading_and_trailing_underscore__: "magic" objects or attributes that live in user-controlled namespaces. E.g. __init__, __import__ or __file__. Never invent such names; only use them as documented.

You will rarely have to call any of these directly. For example:

• MyClass(...) will call MyClass.__init__(...),
• a + b will call a.__plus__(b),
• str(a) will call a.__str__().