Python 3.11 is Coming! Here’s How It Fares Against Python 3.10

A look at the new Python 3.11 features

Photo by Francesco Gallarotti on Unsplash

According to the Python Software Foundation (PSF), Python 3.11 is in it’s 7th alpha revision and is planned to be released in October 2022.

What changes are included for the upcoming version?

Prolog

To evaluate the differences between version 3.10 and 3.11, I’ve set up 2 docker containers.

First container for version 3.10:

docker run -t -d python:3.10.4-bullseye

Second container for version 3.11:

docker run -t -d python:3.11-rc-bullseye

Having the two containers running I can use vs code remote container to attach to the running containers.

Then I can execute my Python code in the two environments and see the difference.

In the following sections, I’ll first show a code example, and then I’ll show the difference between the two versions.

1: Error Location

Output in version 3.10:

1
100
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/py310/myapp/__main__.py", line 4, in <module>
    print(d["key_11"])
KeyError: 'key_11'

Output in version 3.11

1
100
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/py311/myapp/__main__.py", line 4, in <module>
    print(d["key_11"])
          ~^^^^^^^^^^
KeyError: 'key_11'

Python 3.11 discloses a better error location to the developer making an awesome developer experience.

2: The ‘self’ type

The self type has already been introduced in the typing extensions module and now it’s promoted to the standard typing library.

The code above represents the structure of a directory. Directories have subdirectories so the definition is recursive.

Output in Python 3.10:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/py310/myapp/__main__.py", line 2, in <module>
    from typing import List, Tuple, Self

Output in Python 3.11:

{'content': (['a.txt', 'b.txt'],
             [{'content': (['file1', 'file2'], None), 'name': 'dir1'}]),
 'name': 'dir2'}

Note: You can annotate the type of the subdir by the name of the class itself. But if the parent class name changes you’d have to change all the annotation references accordingly.

In order for this code to work in 3.11 and 3.10 you can perform importing as follow:

try:
    from typing import Self
except ImportError:
    from typing_extensions import Self

3: Exception note

The BaseException class now has a __note__ class attribute defaults to None.
You can override it with any string of your choice and provide further information.

Output in Python 3.10:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/py310/myapp/__main__.py", line 6, in <module>
    raise MyException("some exception")
__main__.MyException: some exception

Output in Python 3.11:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/py311/myapp/__main__.py", line 6, in <module>
    raise MyException("some exception")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
MyException: some exception
this is my note :)

We see that the note has been added to the output and it allows developers to be more communicative in their exceptions.

4: Exception groups

Version 3.11 introduces a new exception type name ExceptionGroup to throw a bundle of exceptions and handle them in an except clause.

In addition to that a new except* syntax is introduced.

Output in Python 3.10:

File "/usr/local/lib/python3.10/runpy.py", line 189, in _run_module_as_main
    mod_name, mod_spec, code = _get_main_module_details(_Error)
  File "/usr/local/lib/python3.10/runpy.py", line 223, in _get_main_module_details
    return _get_module_details(main_name)
  File "/usr/local/lib/python3.10/runpy.py", line 157, in _get_module_details
    code = loader.get_code(mod_name)
  File "<frozen importlib._bootstrap_external>", line 1017, in get_code
  File "<frozen importlib._bootstrap_external>", line 947, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/py310/myapp/__main__.py", line 18
    except * (ToYoungException, EmailIsInvalidException) as exception_group_1:

Output in Python 3.11:

validations failed
  + Exception Group Traceback (most recent call last):
  |   File "<frozen runpy>", line 198, in _run_module_as_main
  |   File "<frozen runpy>", line 88, in _run_code
  | ExceptionGroup:  (1 sub-exception)
  +-+---------------- 1 ----------------
    | Exception Group Traceback (most recent call last):
    |   File "/root/py311/myapp/__main__.py", line 14, in <module>
    |     raise ExceptionGroup(
    |     ^^^^^^^^^^^^^^^^^^^^^
    | ExceptionGroup: Data validations (2 sub-exceptions)
    +-+---------------- 1 ----------------
      | ToYoungException: Age must be over 18 - age is 11
      +---------------- 2 ----------------
      | EmailIsInvalidException: Email must be valid some_wannabe_email
      +------------------------------------
    | 
    | The above exception was the direct cause of the following exception:
    | 
    | Traceback (most recent call last):
    |   File "/root/py311/myapp/__main__.py", line 20, in <module>
    |     raise ValueError from exception_group_1
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | ValueError
    +------------------------------------

As we can see, it is highly valuable when we have multiple failure reasons which we want to disclose in one go.

5. Nested Async Comprehensions

Output in Python 3.10:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 189, in _run_module_as_main
    mod_name, mod_spec, code = _get_main_module_details(_Error)
  File "/usr/local/lib/python3.10/runpy.py", line 223, in _get_main_module_details
    return _get_module_details(main_name)
  File "/usr/local/lib/python3.10/runpy.py", line 157, in _get_module_details
    code = loader.get_code(mod_name)
  File "<frozen importlib._bootstrap_external>", line 1017, in get_code
  File "<frozen importlib._bootstrap_external>", line 947, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/py310/myapp/__main__.py", line 11
    return { n: [x async for x in elements(n)] for n in range(3)}
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: asynchronous comprehension outside of an asynchronous function

Output in Python 3.11:

{0: [1], 1: [1, 1], 2: [1, 2, 4], 3: [1, 3, 9, 27], 4: [1, 4, 16, 64, 256]}

The moment the code enters a comprehension block it's now aware of the comprehension as to its current “function context”.
If the comprehension is not asynchronous then the inner code block can not have asynchronous statements.

In Python 3.11 though, comprehensions become implicitly asynchronous if they contain inner async statements that allows for inner async comprehensions.

6: TOML parser

TOML parsing is now part of the standard library just like JSON and CSV.

Output in Python 3.10:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/py310/myapp/__main__.py", line 2, in <module>
    import tomllib
ModuleNotFoundError: No module named 'tomllib'

Output in Python 3.11:

{'clients': {'data': [['gamma', 'delta'], [1, 2]], 'hosts': ['alpha', 'omega']},
 'database': {'connection_max': 5000,
              'enabled': True,
              'ports': [8000, 8001, 8002],
              'server': '192.168.1.1'},
 'owner': {'dob': datetime.datetime(1979, 5, 27, 7, 32, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=57600))),
           'name': 'Tom Preston-Werner'},
 'servers': {'alpha': {'dc': 'eqdc10', 'ip': '10.0.0.1'},
             'beta': {'dc': 'eqdc10', 'ip': '10.0.0.2'}},
 'title': 'TOML Example'}

7. Performance Optimization

It’s argued that Python 3.11 is 10–60% faster in run time compared to python 3.10. You can read more about it in the benchmarking section here.

Python 3.11 is coming with a suite of improvements, both in performance and developer experience.

Learn these updates and up your programming game!