+++++++++++
Python News
+++++++++++
What's New in Python 3.11.1 final?
==================================
*Release date: 2022-12-06*
Security
--------
- gh-issue-100001: ``python -m http.server`` no longer allows terminal
control characters sent within a garbage request to be printed to the
stderr server log.
This is done by changing the :mod:`http.server`
:class:`BaseHTTPRequestHandler` ``.log_message`` method to replace control
characters with a ``\xHH`` hex escape before printing.
- gh-issue-87604: Avoid publishing list of active per-interpreter audit
hooks via the :mod:`gc` module
- gh-issue-98433: The IDNA codec decoder used on DNS hostnames by
:mod:`socket` or :mod:`asyncio` related name resolution functions no
longer involves a quadratic algorithm. This prevents a potential CPU
denial of service if an out-of-spec excessive length hostname involving
bidirectional characters were decoded. Some protocols such as
:mod:`urllib` http ``3xx`` redirects potentially allow for an attacker to
supply such a name.
- gh-issue-98739: Update bundled libexpat to 2.5.0
- gh-issue-97612: Fix a shell code injection vulnerability in the
``get-remote-certificate.py`` example script. The script no longer uses a
shell to run ``openssl`` commands. Issue reported and initial fix by Caleb
Shortt. Patch by Victor Stinner.
Core and Builtins
-----------------
- gh-issue-99886: Fix a crash when an object which does not have a
dictionary frees its instance values.
- gh-issue-99891: Fix a bug in the tokenizer that could cause infinite
recursion when showing syntax warnings that happen in the first line of
the source. Patch by Pablo Galindo
- gh-issue-99729: Fix an issue that could cause frames to be visible to
Python code as they are being torn down, possibly leading to memory
corruption or hard crashes of the interpreter.
- gh-issue-99578: Fix a reference bug in :func:`_imp.create_builtin()` after
the creation of the first sub-interpreter for modules ``builtins`` and
``sys``. Patch by Victor Stinner.
- gh-issue-99581: Fixed a bug that was causing a buffer overflow if the
tokenizer copies a line missing the newline caracter from a file that is
as long as the available tokenizer buffer. Patch by Pablo galindo
- gh-issue-99553: Fix bug where an :exc:`ExceptionGroup` subclass can wrap a
:exc:`BaseException`.
- gh-issue-99370: Fix zip path for venv created from a non-installed python
on POSIX platforms.
- gh-issue-99298: Fix an issue that could potentially cause incorrect error
handling for some bytecode instructions.
- gh-issue-99205: Fix an issue that prevented :c:type:`PyThreadState` and
:c:type:`PyInterpreterState` memory from being freed properly.
- gh-issue-99181: Fix failure in :keyword:`except* <except_star>` with
unhashable exceptions.
- gh-issue-99204: Fix calculation of :data:`sys._base_executable` when
inside a POSIX virtual environment using copies of the python binary when
the base installation does not provide the executable name used by the
venv. Calculation will fall back to alternative names ("python<MAJOR>",
"python<MAJOR>.<MINOR>").
- gh-issue-96055: Update :mod:`faulthandler` to emit an error message with
the proper unexpected signal number. Patch by Dong-hee Na.
- gh-issue-99153: Fix location of :exc:`SyntaxError` for a :keyword:`try`
block with both :keyword:`except` and :keyword:`except* <except_star>`.
- gh-issue-99103: Fix the error reporting positions of specialized traceback
anchors when the source line contains Unicode characters.
- gh-issue-98852: Fix subscription of type aliases containing bare generic
types or types like :class:`~typing.TypeVar`: for example ``tuple[A,
T][int]`` and ``tuple[TypeVar, T][int]``, where ``A`` is a generic type,
and ``T`` is a type variable.
- gh-issue-98925: Lower the recursion depth for marshal on WASI to support
wasmtime 2.0/main.
- gh-issue-98783: Fix multiple crashes in debug mode when ``str`` subclasses
are used instead of ``str`` itself.
- gh-issue-99257: Fix an issue where member descriptors (such as those for
:attr:`~object.__slots__`) could behave incorrectly or crash instead of
raising a :exc:`TypeError` when accessed via an instance of an invalid
type.
- gh-issue-98374: Suppress ImportError for invalid query for help() command.
Patch by Dong-hee Na.
- gh-issue-98415: Fix detection of MAC addresses for :mod:`uuid` on certain
OSs. Patch by Chaim Sanders
- gh-issue-92119: Print exception class name instead of its string
representation when raising errors from :mod:`ctypes` calls.
- gh-issue-96078: :func:`os.sched_yield` now release the GIL while calling
sched_yield(2). Patch by Dong-hee Na.
- gh-issue-93354: Fix an issue that could delay the specialization of
:opcode:`PRECALL` instructions.
- gh-issue-97943: Bugfix: :func:`PyFunction_GetAnnotations` should return a
borrowed reference. It was returning a new reference.
- gh-issue-97779: Ensure that all Python frame objects are backed by
"complete" frames.
- gh-issue-97591: Fixed a missing incref/decref pair in
``Exception.__setstate__()``. Patch by Ofey Chan.
- gh-issue-94526: Fix the Python path configuration used to initialized
:data:`sys.path` at Python startup. Paths are no longer encoded to
UTF-8/strict to avoid encoding errors if it contains surrogate characters
(bytes paths are decoded with the surrogateescape error handler). Patch by
Victor Stinner.
- gh-issue-95921: Fix overly-broad source position information for chained
comparisons used as branching conditions.
- gh-issue-96387: At Python exit, sometimes a thread holding the GIL can
wait forever for a thread (usually a daemon thread) which requested to
drop the GIL, whereas the thread already exited. To fix the race
condition, the thread which requested the GIL drop now resets its request
before exiting. Issue discovered and analyzed by Mingliang ZHAO. Patch by
Victor Stinner.
- gh-issue-96864: Fix a possible assertion failure, fatal error, or
:exc:`SystemError` if a line tracing event raises an exception while
opcode tracing is enabled.
- gh-issue-96678: Fix undefined behaviour in C code of null pointer
arithmetic.
- gh-issue-96754: Make sure that all frame objects created are created from
valid interpreter frames. Prevents the possibility of invalid frames in
backtraces and signal handlers.
- gh-issue-95196: Disable incorrect pickling of the C implemented
classmethod descriptors.
- gh-issue-96005: On WASI :data:`~errno.ENOTCAPABLE` is now mapped to
:exc:`PermissionError`. The :mod:`errno` modules exposes the new error
number. ``getpath.py`` now ignores :exc:`PermissionError` when it cannot
open landmark files ``pybuilddir.txt`` and ``pyenv.cfg``.
- gh-issue-93696: Allow :mod:`pdb` to locate source for frozen modules in
the standard library.
- bpo-31718: Raise :exc:`ValueError` instead of :exc:`SystemError` when
methods of uninitialized :class:`io.IncrementalNewlineDecoder` objects are
called. Patch by Oren Milman.
- bpo-38031: Fix a possible assertion failure in :class:`io.FileIO` when the
opener returns an invalid file descriptor.
Library
-------
- gh-issue-100001: Also \ escape \s in the http.server
BaseHTTPRequestHandler.log_message so that it is technically possible to
parse the line and reconstruct what the original data was. Without this a
\xHH is ambiguious as to if it is a hex replacement we put in or the
characters r"\x" came through in the original request line.
- gh-issue-93453: :func:`asyncio.get_event_loop` now only emits a
deprecation warning when a new event loop was created implicitly. It no
longer emits a deprecation warning if the current event loop was set.
- gh-issue-51524: Fix bug when calling trace.CoverageResults with valid
infile.
- gh-issue-99645: Fix a bug in handling class cleanups in
:class:`unittest.TestCase`. Now ``addClassCleanup()`` uses separate lists
for different ``TestCase`` subclasses, and ``doClassCleanups()`` only
cleans up the particular class.
- gh-issue-97001: Release the GIL when calling termios APIs to avoid
blocking threads.
- gh-issue-99341: Fix :func:`ast.increment_lineno` to also cover
:class:`ast.TypeIgnore` when changing line numbers.
- gh-issue-99418: Fix bug in :func:`urllib.parse.urlparse` that causes URL
schemes that begin with a digit, a plus sign, or a minus sign to be parsed
incorrectly.
- gh-issue-99382: Check the number of arguments in substitution in user
generics containing a :class:`~typing.TypeVarTuple` and one or more
:class:`~typing.TypeVar`.
- gh-issue-99379: Fix substitution of :class:`~typing.ParamSpec` followed by
:class:`~typing.TypeVarTuple` in generic aliases.
- gh-issue-99344: Fix substitution of :class:`~typing.TypeVarTuple` and
:class:`~typing.ParamSpec` together in user generics.
- gh-issue-74044: Fixed bug where :func:`inspect.signature` reported
incorrect arguments for decorated methods.
- gh-issue-99275: Fix ``SystemError`` in :mod:`ctypes` when exception was
not set during ``__initsubclass__``.
- gh-issue-99277: Remove older version of
``_SSLProtocolTransport.get_write_buffer_limits`` in
:mod:`!asyncio.sslproto`
- gh-issue-99248: fix negative numbers failing in verify()
- gh-issue-99155: Fix :class:`statistics.NormalDist` pickle with ``0`` and
``1`` protocols.
- gh-issue-93464: ``enum.auto()`` is now correctly activated when combined
with other assignment values. E.g. ``ONE = auto(), 'some text'`` will now
evaluate as ``(1, 'some text')``.
- gh-issue-99134: Update the bundled copy of pip to version 22.3.1.
- gh-issue-83004: Clean up refleak on failed module initialisation in
:mod:`_zoneinfo`
- gh-issue-83004: Clean up refleaks on failed module initialisation in in
:mod:`_pickle`
- gh-issue-83004: Clean up refleak on failed module initialisation in
:mod:`_io`.
- gh-issue-98897: Fix memory leak in :func:`math.dist` when both points
don't have the same dimension. Patch by Kumar Aditya.
- gh-issue-98706: [3.11] Applied changes from importlib_metadata `4.11.4
through 4.13
<https://importlib-metadata.readthedocs.io/en/latest/history.html#v4-13-0>`_,
including compatibility and robustness fixes for ``Distribution`` objects
without ``_normalized_name``, disallowing invalid inputs to
``Distribution.from_name``, and refined behaviors in
``PathDistribution._name_from_stem`` and
``PathDistribution._normalized_name``.
- gh-issue-98793: Fix argument typechecks in :func:`!_overlapped.WSAConnect`
and :func:`!_overlapped.Overlapped.WSASendTo` functions.
- gh-issue-98744: Prevent crashing in :mod:`traceback` when retrieving the
byte-offset for some source files that contain certain unicode characters.
- gh-issue-98740: Fix internal error in the :mod:`re` module which in very
rare circumstances prevented compilation of a regular expression
containing a :ref:`conditional expression <re-conditional-expression>`
without the "else" branch.
- gh-issue-98703: Fix :meth:`asyncio.StreamWriter.drain` to call
``protocol.connection_lost`` callback only once on Windows.
- gh-issue-98624: Add a mutex to unittest.mock.NonCallableMock to protect
concurrent access to mock attributes.
- gh-issue-89237: Fix hang on Windows in ``subprocess.wait_closed()`` in
:mod:`asyncio` with :class:`~asyncio.ProactorEventLoop`. Patch by Kumar
Aditya.
- gh-issue-98458: Fix infinite loop in unittest when a self-referencing
chained exception is raised
- gh-issue-97928: :meth:`tkinter.Text.count` raises now an exception for
options starting with "-" instead of silently ignoring them.
- gh-issue-97966: On ``uname_result``, restored expectation that ``_fields``
and ``_asdict`` would include all six properties including ``processor``.
- gh-issue-98307: A :meth:`~logging.handlers.SysLogHandler.createSocket`
method was added to :class:`~logging.handlers.SysLogHandler`.
- gh-issue-96035: Fix bug in :func:`urllib.parse.urlparse` that causes
certain port numbers containing whitespace, underscores, plus and minus
signs, or non-ASCII digits to be incorrectly accepted.
- gh-issue-98251: Allow :mod:`venv` to pass along :envvar:`PYTHON*`
variables to ``ensurepip`` and ``pip`` when they do not impact path
resolution
- gh-issue-98178: On macOS, fix a crash in :func:`syslog.syslog` in
multi-threaded applications. On macOS, the libc ``syslog()`` function is
not thread-safe, so :func:`syslog.syslog` no longer releases the GIL to
call it. Patch by Victor Stinner.
- gh-issue-96151: Allow ``BUILTINS`` to be a valid field name for frozen
dataclasses.
- gh-issue-87730: Wrap network errors consistently in urllib FTP support, so
the test suite doesn't fail when a network is available but the public
internet is not reachable.
- gh-issue-98086: Make sure ``patch.dict()`` can be applied on async
functions.
- gh-issue-90985: Earlier in 3.11 we deprecated
``asyncio.Task.cancel("message")``. We realized we were too harsh, and
have undeprecated it.
- gh-issue-97837: Change deprecate warning message in :mod:`unittest` from
``It is deprecated to return a value!=None``
to
``It is deprecated to return a value that is not None from a test case``
- gh-issue-97825: Fixes :exc:`AttributeError` when
:meth:`subprocess.check_output` is used with argument ``input=None`` and
either of the arguments *encoding* or *errors* are used.
- gh-issue-82836: Fix :attr:`~ipaddress.IPv4Address.is_private` properties
in the :mod:`ipaddress` module. Previously non-private networks
(0.0.0.0/0) would return True from this method; now they correctly return
False.
- gh-issue-96827: Avoid spurious tracebacks from :mod:`asyncio` when default
executor cleanup is delayed until after the event loop is closed (e.g. as
the result of a keyboard interrupt).
- gh-issue-97592: Avoid a crash in the C version of
:meth:`asyncio.Future.remove_done_callback` when an evil argument is
passed.
- gh-issue-97639: Remove ``tokenize.NL`` check from :mod:`tabnanny`.
- gh-issue-73588: Fix generation of the default name of
:class:`tkinter.Checkbutton`. Previously, checkbuttons in different parent
widgets could have the same short name and share the same state if
arguments "name" and "variable" are not specified. Now they are globally
unique.
- gh-issue-97005: Update bundled libexpat to 2.4.9
- gh-issue-85760: Fix race condition in :mod:`asyncio` where
:meth:`~asyncio.SubprocessProtocol.process_exited` called before the
:meth:`~asyncio.SubprocessProtocol.pipe_data_received` leading to
inconsistent output. Patch by Kumar Aditya.
- gh-issue-96819: Fixed check in :mod:`multiprocessing.resource_tracker`
that guarantees that the length of a write to a pipe is not greater than
``PIPE_BUF``.
- gh-issue-96741: Corrected type annotation for dataclass attribute
``pstats.FunctionProfile.ncalls`` to be ``str``.
- gh-issue-95987: Fix ``repr`` of ``Any`` subclasses.
- gh-issue-96388: Work around missing socket functions in
:class:`~socket.socket`'s ``__repr__``.
- gh-issue-96073: In :mod:`inspect`, fix overeager replacement of
"``typing.``" in formatting annotations.
- gh-issue-96192: Fix handling of ``bytes`` :term:`path-like objects
<path-like object>` in :func:`os.ismount()`.
- gh-issue-96052: Fix handling compiler warnings (SyntaxWarning and
DeprecationWarning) in :func:`codeop.compile_command` when checking for
incomplete input. Previously it emitted warnings and raised a SyntaxError.
Now it always returns ``None`` for incomplete input without emitting any
warnings.
- gh-issue-88863: To avoid apparent memory leaks when
:func:`asyncio.open_connection` raises, break reference cycles generated
by local exception and future instances (which has exception instance as
its member var). Patch by Dong Uk, Kang.
- gh-issue-91212: Fixed flickering of the turtle window when the trac