Merge remote-tracking branch 'origin/main' into pythongh-94906

Author: matthiasgoergens

.github/CONTRIBUTING.rst

@@ -6,19 +6,19 @@ Build Status
 
 - main
 
-  + `Stable buildbots <http://buildbot.python.org/3.x.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.x.stable/>`_
 
 - 3.9
 
-  + `Stable buildbots <http://buildbot.python.org/3.9.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.9.stable/>`_
 
 - 3.8
 
-  + `Stable buildbots <http://buildbot.python.org/3.8.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.8.stable/>`_
 
 - 3.7
 
-  + `Stable buildbots <http://buildbot.python.org/3.7.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.7.stable/>`_
 
 
 Thank You

Doc/about.rst

@@ -6,7 +6,7 @@ About these documents
 These documents are generated from `reStructuredText`_ sources by `Sphinx`_, a
 document processor specifically written for the Python documentation.
 
-.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _reStructuredText: https://docutils.sourceforge.io/rst.html
 .. _Sphinx: http://sphinx-doc.org/
 
 .. In the online version of these documents, you can submit comments and suggest
@@ -21,7 +21,7 @@ Many thanks go to:
 
 * Fred L. Drake, Jr., the creator of the original Python documentation toolset
   and writer of much of the content;
-* the `Docutils <http://docutils.sourceforge.net/>`_ project for creating
+* the `Docutils <https://docutils.sourceforge.io/>`_ project for creating
   reStructuredText and the Docutils suite;
 * Fredrik Lundh for his Alternative Python Reference project from which Sphinx
   got many good ideas.

Doc/c-api/call.rst

@@ -57,6 +57,15 @@ This bears repeating:
    A class supporting vectorcall **must** also implement
    :c:member:`~PyTypeObject.tp_call` with the same semantics.
 
+.. versionchanged:: 3.12
+
+   The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class
+   when the class's :py:meth:`~object.__call__` method is reassigned.
+   (This internally sets :c:member:`~PyTypeObject.tp_call` only, and thus
+   may make it behave differently than the vectorcall function.)
+   In earlier Python versions, vectorcall should only be used with
+   :const:`immutable <Py_TPFLAGS_IMMUTABLETYPE>` or static types.
+
 A class should not implement vectorcall if that would be slower
 than *tp_call*. For example, if the callee needs to convert
 the arguments to an args tuple and kwargs dict anyway, then there is no point

Doc/c-api/code.rst

@@ -90,3 +90,28 @@ bound into a function.
 
    .. versionadded:: 3.11
 
+.. c:function:: PyObject* PyCode_GetVarnames(PyCodeObject *co)
+
+   Equivalent to the Python code ``getattr(co, 'co_varnames')``.
+   Returns a new reference to a :c:type:`PyTupleObject` containing the names of
+   the local variables. On error, ``NULL`` is returned and an exception
+   is raised.
+
+   .. versionadded:: 3.11
+
+.. c:function:: PyObject* PyCode_GetCellvars(PyCodeObject *co)
+
+   Equivalent to the Python code ``getattr(co, 'co_cellvars')``.
+   Returns a new reference to a :c:type:`PyTupleObject` containing the names of
+   the local variables that are referenced by nested functions. On error, ``NULL``
+   is returned and an exception is raised.
+
+   .. versionadded:: 3.11
+
+.. c:function:: PyObject* PyCode_GetFreevars(PyCodeObject *co)
+
+   Equivalent to the Python code ``getattr(co, 'co_freevars')``.
+   Returns a new reference to a :c:type:`PyTupleObject` containing the names of
+   the free variables. On error, ``NULL`` is returned and an exception is raised.
+
+   .. versionadded:: 3.11

Doc/c-api/object.rst

@@ -126,6 +126,14 @@ Object Protocol
    A generic implementation for the getter of a ``__dict__`` descriptor. It
    creates the dictionary if necessary.
 
+   This function may also be called to get the :py:attr:`~object.__dict__`
+   of the object *o*. Pass ``NULL`` for *context* when calling it.
+   Since this function may need to allocate memory for the
+   dictionary, it may be more efficient to call :c:func:`PyObject_GetAttr`
+   when accessing an attribute on the object.
+
+   On failure, returns ``NULL`` with an exception set.
+
    .. versionadded:: 3.3
 
 
@@ -137,6 +145,16 @@ Object Protocol
    .. versionadded:: 3.3
 
 
+.. c:function:: PyObject** _PyObject_GetDictPtr(PyObject *obj)
+
+   Return a pointer to :py:attr:`~object.__dict__` of the object *obj*.
+   If there is no ``__dict__``, return ``NULL`` without setting an exception.
+
+   This function may need to allocate memory for the
+   dictionary, so it may be more efficient to call :c:func:`PyObject_GetAttr`
+   when accessing an attribute on the object.
+
+
 .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid)
 
    Compare the values of *o1* and *o2* using the operation specified by *opid*,

Doc/c-api/typeobj.rst

@@ -135,7 +135,7 @@ Quick Reference
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
    | [:c:member:`~PyTypeObject.tp_cache`]           | :c:type:`PyObject` *              |                   |   |   |       |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
-   | [:c:member:`~PyTypeObject.tp_subclasses`]      | :c:type:`PyObject` *              | __subclasses__    |   |   |       |
+   | [:c:member:`~PyTypeObject.tp_subclasses`]      | void *                            | __subclasses__    |   |   |       |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
    | [:c:member:`~PyTypeObject.tp_weaklist`]        | :c:type:`PyObject` *              |                   |   |   |       |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
@@ -720,29 +720,29 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    with the *vectorcallfunc* function.
    This can be done by setting *tp_call* to :c:func:`PyVectorcall_Call`.
 
-   .. warning::
-
-      It is not recommended for :ref:`mutable heap types <heap-types>` to implement
-      the vectorcall protocol.
-      When a user sets :attr:`__call__` in Python code, only *tp_call* is updated,
-      likely making it inconsistent with the vectorcall function.
-
    .. versionchanged:: 3.8
 
       Before version 3.8, this slot was named ``tp_print``.
       In Python 2.x, it was used for printing to a file.
       In Python 3.0 to 3.7, it was unused.
 
+   .. versionchanged:: 3.12
+
+      Before version 3.12, it was not recommended for
+      :ref:`mutable heap types <heap-types>` to implement the vectorcall
+      protocol.
+      When a user sets :attr:`~type.__call__` in Python code, only *tp_call* is
+      updated, likely making it inconsistent with the vectorcall function.
+      Since 3.12, setting ``__call__`` will disable vectorcall optimization
+      by clearing the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag.
+
    **Inheritance:**
 
    This field is always inherited.
    However, the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not
-   always inherited. If it's not, then the subclass won't use
+   always inherited. If it's not set, then the subclass won't use
    :ref:`vectorcall <vectorcall>`, except when
    :c:func:`PyVectorcall_Call` is explicitly called.
-   This is in particular the case for types without the
-   :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set (including subclasses defined in
-   Python).
 
 
 .. c:member:: getattrfunc PyTypeObject.tp_getattr
@@ -1178,12 +1178,18 @@ and :c:type:`PyType_Type` effectively act as defaults.)
 
       **Inheritance:**
 
-      This bit is inherited for types with the
-      :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set, if
-      :c:member:`~PyTypeObject.tp_call` is also inherited.
+      This bit is inherited if :c:member:`~PyTypeObject.tp_call` is also
+      inherited.
 
       .. versionadded:: 3.9
 
+      .. versionchanged:: 3.12
+
+         This flag is now removed from a class when the class's
+         :py:meth:`~object.__call__` method is reassigned.
+
+         This flag can now be inherited by mutable classes.
+
    .. data:: Py_TPFLAGS_IMMUTABLETYPE
 
       This bit is set for type objects that are immutable: type attributes cannot be set nor deleted.
@@ -1709,18 +1715,11 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    :c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is
    at the very end of the structure.
 
-   The real dictionary offset in an instance can be computed from a negative
-   :c:member:`~PyTypeObject.tp_dictoffset` as follows::
-
-      dictoffset = tp_basicsize + abs(ob_size)*tp_itemsize + tp_dictoffset
-      if dictoffset is not aligned on sizeof(void*):
-          round up to sizeof(void*)
-
-   where :c:member:`~PyTypeObject.tp_basicsize`, :c:member:`~PyTypeObject.tp_itemsize` and :c:member:`~PyTypeObject.tp_dictoffset` are
-   taken from the type object, and :attr:`ob_size` is taken from the instance.  The
-   absolute value is taken because ints use the sign of :attr:`ob_size` to
-   store the sign of the number.  (There's never a need to do this calculation
-   yourself; it is done for you by :c:func:`_PyObject_GetDictPtr`.)
+   The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only.
+   To get the pointer to the dictionary call :c:func:`PyObject_GenericGetDict`.
+   Calling :c:func:`PyObject_GenericGetDict` may need to allocate memory for the
+   dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr`
+   when accessing an attribute on the object.
 
    **Inheritance:**
 
@@ -1928,9 +1927,17 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    This field is not inherited.
 
 
-.. c:member:: PyObject* PyTypeObject.tp_subclasses
+.. c:member:: void* PyTypeObject.tp_subclasses
+
+   A collection of subclasses.  Internal use only.  May be an invalid pointer.
+
+   To get a list of subclasses, call the Python method
+   :py:meth:`~class.__subclasses__`.
+
+   .. versionchanged:: 3.12
 
-   List of weak references to subclasses.  Internal use only.
+      For some types, this field does not hold a valid :c:expr:`PyObject*`.
+      The type was changed to :c:expr:`void*` to indicate this.
 
    **Inheritance:**
 

Doc/c-api/unicode.rst

@@ -477,9 +477,6 @@ APIs:
    |                   |                     | :c:func:`PyObject_Repr`.         |
    +-------------------+---------------------+----------------------------------+
 
-   An unrecognized format character causes all the rest of the format string to be
-   copied as-is to the result string, and any extra arguments discarded.
-
    .. note::
       The width formatter unit is number of characters rather than bytes.
       The precision formatter unit is number of bytes for ``"%s"`` and
@@ -500,6 +497,11 @@ APIs:
       Support width and precision formatter for ``"%s"``, ``"%A"``, ``"%U"``,
       ``"%V"``, ``"%S"``, ``"%R"`` added.
 
+   .. versionchanged:: 3.12
+      An unrecognized format character now sets a :exc:`SystemError`.
+      In previous versions it caused all the rest of the format string to be
+      copied as-is to the result string, and any extra arguments discarded.
+
 
 .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs)
 

Doc/conf.py

@@ -79,6 +79,13 @@
     'root_include_title': False   # We use the version switcher instead.
 }
 
+# Override stylesheet fingerprinting for Windows CHM htmlhelp to fix GH-91207
+# https://github.com/python/cpython/issues/91207
+if any('htmlhelp' in arg for arg in sys.argv):
+    html_style = 'pydoctheme.css'
+    print("\nWARNING: Windows CHM Help is no longer supported.")
+    print("It may be removed in the future\n")
+
 # Short title used e.g. for <title> HTML tags.
 html_short_title = '%s Documentation' % release
 

Doc/data/stable_abi.dat

@@ -27,8 +27,8 @@ Recommended third party tools
 
 This guide only covers the basic tools for creating extensions provided
 as part of this version of CPython. Third party tools like
-`Cython <http://cython.org/>`_, `cffi <https://cffi.readthedocs.io>`_,
-`SWIG <http://www.swig.org>`_ and `Numba <https://numba.pydata.org/>`_
+`Cython <https://cython.org/>`_, `cffi <https://cffi.readthedocs.io>`_,
+`SWIG <https://www.swig.org>`_ and `Numba <https://numba.pydata.org/>`_
 offer both simpler and more sophisticated approaches to creating C and C++
 extensions for Python.
 

Doc/extending/index.rst

@@ -321,8 +321,8 @@ is exactly the same type of object that a lambda expression yields) is assigned!
 Can Python be compiled to machine code, C or some other language?
 -----------------------------------------------------------------
 
-`Cython <http://cython.org/>`_ compiles a modified version of Python with
-optional annotations into C extensions.  `Nuitka <http://www.nuitka.net/>`_ is
+`Cython <https://cython.org/>`_ compiles a modified version of Python with
+optional annotations into C extensions.  `Nuitka <https://www.nuitka.net/>`_ is
 an up-and-coming compiler of Python into C++ code, aiming to support the full
 Python language.
 
@@ -338,8 +338,8 @@ cycles and deletes the objects involved. The :mod:`gc` module provides functions
 to perform a garbage collection, obtain debugging statistics, and tune the
 collector's parameters.
 
-Other implementations (such as `Jython <http://www.jython.org>`_ or
-`PyPy <http://www.pypy.org>`_), however, can rely on a different mechanism
+Other implementations (such as `Jython <https://www.jython.org>`_ or
+`PyPy <https://www.pypy.org>`_), however, can rely on a different mechanism
 such as a full-blown garbage collector.  This difference can cause some
 subtle porting problems if your Python code depends on the behavior of the
 reference counting implementation.

Doc/faq/design.rst

@@ -41,18 +41,18 @@ on what you're trying to do.
 
 .. XXX make sure these all work
 
-`Cython <http://cython.org>`_ and its relative `Pyrex
+`Cython <https://cython.org>`_ and its relative `Pyrex
 <https://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/>`_ are compilers
 that accept a slightly modified form of Python and generate the corresponding
 C code.  Cython and Pyrex make it possible to write an extension without having
 to learn Python's C API.
 
 If you need to interface to some C or C++ library for which no Python extension
 currently exists, you can try wrapping the library's data types and functions
-with a tool such as `SWIG <http://www.swig.org>`_.  `SIP
+with a tool such as `SWIG <https://www.swig.org>`_.  `SIP
 <https://riverbankcomputing.com/software/sip/intro>`__, `CXX
 <http://cxx.sourceforge.net/>`_ `Boost
-<http://www.boost.org/libs/python/doc/index.html>`_, or `Weave
+<https://www.boost.org/libs/python/doc/index.html>`_, or `Weave
 <https://github.com/scipy/weave>`_ are also
 alternatives for wrapping C++ libraries.
 
@@ -286,6 +286,6 @@ Can I create an object class with some methods implemented in C and others in Py
 Yes, you can inherit from built-in classes such as :class:`int`, :class:`list`,
 :class:`dict`, etc.
 
-The Boost Python Library (BPL, http://www.boost.org/libs/python/doc/index.html)
+The Boost Python Library (BPL, https://www.boost.org/libs/python/doc/index.html)
 provides a way of doing this from C++ (i.e. you can inherit from an extension
 class written in C++ using the BPL).

Doc/faq/extending.rst

@@ -330,7 +330,7 @@ different companies and organizations.
 
 High-profile Python projects include `the Mailman mailing list manager
 <http://www.list.org>`_ and `the Zope application server
-<http://www.zope.org>`_.  Several Linux distributions, most notably `Red Hat
+<https://www.zope.dev>`_.  Several Linux distributions, most notably `Red Hat
 <https://www.redhat.com>`_, have written part or all of their installer and
 system administration software in Python.  Companies that use Python internally
 include Google, Yahoo, and Lucasfilm Ltd.

Doc/faq/general.rst

@@ -95,7 +95,7 @@ The following packages can help with the creation of console and GUI
 executables:
 
 * `Nuitka <https://nuitka.net/>`_ (Cross-platform)
-* `PyInstaller <http://www.pyinstaller.org/>`_ (Cross-platform)
+* `PyInstaller <https://pyinstaller.org/>`_ (Cross-platform)
 * `PyOxidizer <https://pyoxidizer.readthedocs.io/en/stable/>`_ (Cross-platform)
 * `cx_Freeze <https://marcelotduarte.github.io/cx_Freeze/>`_ (Cross-platform)
 * `py2app <https://github.com/ronaldoussoren/py2app>`_ (macOS only)
@@ -1066,7 +1066,7 @@ performance levels:
   detrimental to readability).
 
 If you have reached the limit of what pure Python can allow, there are tools
-to take you further away.  For example, `Cython <http://cython.org>`_ can
+to take you further away.  For example, `Cython <https://cython.org>`_ can
 compile a slightly modified version of Python code into a C extension, and
 can be used on many different platforms.  Cython can take advantage of
 compilation (and optional type annotations) to make your code significantly

Doc/faq/programming.rst

@@ -566,9 +566,9 @@ Glossary
       from their :func:`id`.
 
    IDLE
-      An Integrated Development Environment for Python.  IDLE is a basic editor
-      and interpreter environment which ships with the standard distribution of
-      Python.
+      An Integrated Development and Learning Environment for Python.
+      :ref:`idle` is a basic editor and interpreter environment
+      which ships with the standard distribution of Python.
 
    immutable
       An object with a fixed value.  Immutable objects include numbers, strings and