Merge branch 'main' into typedocs

* main: (103 commits)
  pythongh-100248: Add missing `ssl_shutdown_timeout` parameter in `asyncio` docs (python#100249)
  Assorted minor fixes for specialization stats. (pythonGH-100219)
  pythongh-100176: venv: Remove redundant compat code for Python <= 3.2 (python#100177)
  pythonGH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (pythonGH-100223)
  pythongh-99955: undef ERROR and SUCCESS before redefining (fixes sanitizer warning) (python#100215)
  pythonGH-100206: use versionadded for the addition of sysconfig.get_default_scheme (python#100207)
  pythongh-81057: Move _Py_RefTotal to the "Ignored Globals" List (pythongh-100203)
  pythongh-81057: Move Signal-Related Globals to _PyRuntimeState (pythongh-100085)
  pythongh-81057: Move faulthandler Globals to _PyRuntimeState (pythongh-100152)
  pythongh-81057: Move tracemalloc Globals to _PyRuntimeState (pythongh-100151)
  pythonGH-100143: Improve collecting pystats for parts of runs (pythonGH-100144)
  pythongh-99955: standardize return values of functions in compiler's code-gen (python#100010)
  pythongh-79218: Define `MS_WIN64` macro for Mingw-w64 64bit on Windows (pythonGH-100137)
  Fix: typo (Indention) (pythonGH-99904)
  pythongh-96715 Remove redundant NULL check in `profile_trampoline` function (python#96716)
  pythongh-100176: remove incorrect version compatibility check from argument clinic (python#100190)
  clarify the 4300-digit limit on int-str conversion (python#100175)
  pythongh-70393: Clarify mention of "middle" scope (python#98839)
  pythongh-99688: Fix outdated tests in test_unary (python#99712)
  pythongh-100174: [Enum] Correct PowersOfThree example. (pythonGH-100178)
  ...

Author: carljm

Doc/_static/og-image.png

@@ -115,3 +115,51 @@ bound into a function.
    the free variables. On error, ``NULL`` is returned and an exception is raised.
 
    .. versionadded:: 3.11
+
+.. c:function:: int PyCode_AddWatcher(PyCode_WatchCallback callback)
+
+   Register *callback* as a code object watcher for the current interpreter.
+   Return an ID which may be passed to :c:func:`PyCode_ClearWatcher`.
+   In case of error (e.g. no more watcher IDs available),
+   return ``-1`` and set an exception.
+
+   .. versionadded:: 3.12
+
+.. c:function:: int PyCode_ClearWatcher(int watcher_id)
+
+   Clear watcher identified by *watcher_id* previously returned from
+   :c:func:`PyCode_AddWatcher` for the current interpreter.
+   Return ``0`` on success, or ``-1`` and set an exception on error
+   (e.g. if the given *watcher_id* was never registered.)
+
+   .. versionadded:: 3.12
+
+.. c:type:: PyCodeEvent
+
+   Enumeration of possible code object watcher events:
+   - ``PY_CODE_EVENT_CREATE``
+   - ``PY_CODE_EVENT_DESTROY``
+
+   .. versionadded:: 3.12
+
+.. c:type:: int (*PyCode_WatchCallback)(PyCodeEvent event, PyCodeObject* co)
+
+   Type of a code object watcher callback function.
+
+   If *event* is ``PY_CODE_EVENT_CREATE``, then the callback is invoked
+   after `co` has been fully initialized. Otherwise, the callback is invoked
+   before the destruction of *co* takes place, so the prior state of *co*
+   can be inspected.
+
+   Users of this API should not rely on internal runtime implementation
+   details. Such details may include, but are not limited to, the exact
+   order and timing of creation and destruction of code objects. While
+   changes in these details may result in differences observable by watchers
+   (including whether a callback is invoked or not), it does not change
+   the semantics of the Python code being executed.
+
+   If the callback returns with an exception set, it must return ``-1``; this
+   exception will be printed as an unraisable exception using
+   :c:func:`PyErr_WriteUnraisable`. Otherwise it should return ``0``.
+
+   .. versionadded:: 3.12

Doc/c-api/code.rst

@@ -7,8 +7,8 @@
 Reference Counting
 ******************
 
-The macros in this section are used for managing reference counts of Python
-objects.
+The functions and macros in this section are used for managing reference counts
+of Python objects.
 
 
 .. c:function:: Py_ssize_t Py_REFCNT(PyObject *o)
@@ -129,6 +129,11 @@ objects.
    It is a good idea to use this macro whenever decrementing the reference
    count of an object that might be traversed during garbage collection.
 
+   .. versionchanged:: 3.12
+      The macro argument is now only evaluated once. If the argument has side
+      effects, these are no longer duplicated.
+
+
 .. c:function:: void Py_IncRef(PyObject *o)
 
    Increment the reference count for object *o*. A function version of :c:func:`Py_XINCREF`.
@@ -139,3 +144,40 @@ objects.
 
    Decrement the reference count for object *o*. A function version of :c:func:`Py_XDECREF`.
    It can be used for runtime dynamic embedding of Python.
+
+
+.. c:macro:: Py_SETREF(dst, src)
+
+   Macro safely decrementing the `dst` reference count and setting `dst` to
+   `src`.
+
+   As in case of :c:func:`Py_CLEAR`, "the obvious" code can be deadly::
+
+       Py_DECREF(dst);
+       dst = src;
+
+   The safe way is::
+
+        Py_SETREF(dst, src);
+
+   That arranges to set `dst` to `src` _before_ decrementing reference count of
+   *dst* old value, so that any code triggered as a side-effect of `dst`
+   getting torn down no longer believes `dst` points to a valid object.
+
+   .. versionadded:: 3.6
+
+   .. versionchanged:: 3.12
+      The macro arguments are now only evaluated once. If an argument has side
+      effects, these are no longer duplicated.
+
+
+.. c:macro:: Py_XSETREF(dst, src)
+
+   Variant of :c:macro:`Py_SETREF` macro that uses :c:func:`Py_XDECREF` instead
+   of :c:func:`Py_DECREF`.
+
+   .. versionadded:: 3.6
+
+   .. versionchanged:: 3.12
+      The macro arguments are now only evaluated once. If an argument has side
+      effects, these are no longer duplicated.

Doc/c-api/refcounting.rst

@@ -13,9 +13,25 @@
 # General configuration
 # ---------------------
 
-extensions = ['sphinx.ext.coverage', 'sphinx.ext.doctest',
-              'pyspecific', 'c_annotations', 'escape4chm',
-              'asdl_highlight', 'peg_highlight', 'glossary_search']
+extensions = [
+    'asdl_highlight',
+    'c_annotations',
+    'escape4chm',
+    'glossary_search',
+    'peg_highlight',
+    'pyspecific',
+    'sphinx.ext.coverage',
+    'sphinx.ext.doctest',
+]
+
+# Skip if downstream redistributors haven't installed it
+try:
+    import sphinxext.opengraph
+except ImportError:
+    pass
+else:
+    extensions.append('sphinxext.opengraph')
+
 
 doctest_global_setup = '''
 try:
@@ -89,6 +105,14 @@
 # Short title used e.g. for <title> HTML tags.
 html_short_title = '%s Documentation' % release
 
+# Deployment preview information, from Netlify
+# (See netlify.toml and https://docs.netlify.com/configure-builds/environment-variables/#git-metadata)
+html_context = {
+    "is_deployment_preview": os.getenv("IS_DEPLOYMENT_PREVIEW"),
+    "repository_url": os.getenv("REPOSITORY_URL"),
+    "pr_id": os.getenv("REVIEW_ID")
+}
+
 # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 # using the given strftime format.
 html_last_updated_fmt = '%b %d, %Y'
@@ -114,7 +138,7 @@
 html_use_opensearch = 'https://docs.python.org/' + version
 
 # Additional static files.
-html_static_path = ['tools/static']
+html_static_path = ['_static', 'tools/static']
 
 # Output file base name for HTML help builder.
 htmlhelp_basename = 'python' + release.replace('.', '')
@@ -238,3 +262,13 @@
 # Relative filename of the data files
 refcount_file = 'data/refcounts.dat'
 stable_abi_file = 'data/stable_abi.dat'
+
+# sphinxext-opengraph config
+ogp_site_url = 'https://docs.python.org/3/'
+ogp_site_name = 'Python documentation'
+ogp_image = '_static/og-image.png'
+ogp_custom_meta_tags = [
+    '<meta property="og:image:width" content="200">',
+    '<meta property="og:image:height" content="200">',
+    '<meta name="theme-color" content="#3776ab">',
+]

Doc/conf.py

@@ -158,6 +158,7 @@ And a function to display the chores for a given day::
     ...     for chore, days in chores.items():
     ...         if day in days:
     ...             print(chore)
+    ...
     >>> show_chores(chores_for_ethan, Weekday.SATURDAY)
     answer SO questions
 
@@ -459,6 +460,31 @@ sense to allow sharing some common behavior between a group of enumerations.
 (See `OrderedEnum`_ for an example.)
 
 
+.. _enum-dataclass-support:
+
+Dataclass support
+-----------------
+
+When inheriting from a :class:`~dataclasses.dataclass`,
+the :meth:`~Enum.__repr__` omits the inherited class' name.  For example::
+
+    >>> @dataclass
+    ... class CreatureDataMixin:
+    ...     size: str
+    ...     legs: int
+    ...     tail: bool = field(repr=False, default=True)
+    ...
+    >>> class Creature(CreatureDataMixin, Enum):
+    ...     BEETLE = 'small', 6
+    ...     DOG = 'medium', 4
+    ...
+    >>> Creature.DOG
+    <Creature.DOG: size='medium', legs=4>
+
+Use the :func:`!dataclass` argument ``repr=False``
+to use the standard :func:`repr`.
+
+
 Pickling
 --------
 
@@ -687,6 +713,7 @@ It is also possible to name the combinations::
     ...     W = 2
     ...     X = 1
     ...     RWX = 7
+    ...
     >>> Perm.RWX
     <Perm.RWX: 7>
     >>> ~Perm.RWX

Doc/howto/enum.rst

@@ -565,6 +565,7 @@ arguments they contain.  For example::
 
    >>> with open('args.txt', 'w', encoding=sys.getfilesystemencoding()) as fp:
    ...     fp.write('-f\nbar')
+   ...
    >>> parser = argparse.ArgumentParser(fromfile_prefix_chars='@')
    >>> parser.add_argument('-f')
    >>> parser.parse_args(['-f', 'foo', '@args.txt'])

Doc/library/argparse.rst

@@ -33,7 +33,8 @@ an event loop:
 
    Return the running event loop in the current OS thread.
 
-   If there is no running event loop a :exc:`RuntimeError` is raised.
+   Raise a :exc:`RuntimeError` if there is no running event loop.
+
    This function can only be called from a coroutine or a callback.
 
    .. versionadded:: 3.7
@@ -42,27 +43,31 @@ an event loop:
 
    Get the current event loop.
 
-   If there is no current event loop set in the current OS thread,
-   the OS thread is main, and :func:`set_event_loop` has not yet
-   been called, asyncio will create a new event loop and set it as the
-   current one.
+   When called from a coroutine or a callback (e.g. scheduled with
+   call_soon or similar API), this function will always return the
+   running event loop.
+
+   If there is no running event loop set, the function will return
+   the result of calling ``get_event_loop_policy().get_event_loop()``.
 
    Because this function has rather complex behavior (especially
    when custom event loop policies are in use), using the
    :func:`get_running_loop` function is preferred to :func:`get_event_loop`
    in coroutines and callbacks.
 
-   Consider also using the :func:`asyncio.run` function instead of using
-   lower level functions to manually create and close an event loop.
+   As noted above, consider using the higher-level :func:`asyncio.run` function,
+   instead of using these lower level functions to manually create and close an
+   event loop.
 
-   .. deprecated:: 3.10
-      Deprecation warning is emitted if there is no running event loop.
-      In future Python releases, this function will be an alias of
-      :func:`get_running_loop`.
+   .. note::
+      In Python versions 3.10.0--3.10.8 and 3.11.0 this function
+      (and other functions which used it implicitly) emitted a
+      :exc:`DeprecationWarning` if there was no running event loop, even if
+      the current loop was set.
 
 .. function:: set_event_loop(loop)
 
-   Set *loop* as a current event loop for the current OS thread.
+   Set *loop* as the current event loop for the current OS thread.
 
 .. function:: new_event_loop()
 

Doc/library/asyncio-eventloop.rst

@@ -19,7 +19,7 @@ Obtaining the Event Loop
       - The **preferred** function to get the running event loop.
 
     * - :func:`asyncio.get_event_loop`
-      - Get an event loop instance (current or via the policy).
+      - Get an event loop instance (running or current via the current policy).
 
     * - :func:`asyncio.set_event_loop`
       - Set the event loop as current via the current policy.

Doc/library/asyncio-llapi-index.rst

@@ -116,6 +116,10 @@ asyncio ships with the following built-in policies:
 
       On Windows, :class:`ProactorEventLoop` is now used by default.
 
+   .. versionchanged:: 3.12
+      :meth:`get_event_loop` now raises a :exc:`RuntimeError` if there is no
+      current event loop set.
+
 
 .. class:: WindowsSelectorEventLoopPolicy
 

Doc/library/asyncio-policy.rst

@@ -52,6 +52,7 @@ and work with streams:
                           limit=None, ssl=None, family=0, proto=0, \
                           flags=0, sock=None, local_addr=None, \
                           server_hostname=None, ssl_handshake_timeout=None, \
+                          ssl_shutdown_timeout=None, \
                           happy_eyeballs_delay=None, interleave=None)
 
    Establish a network connection and return a pair of
@@ -82,14 +83,17 @@ and work with streams:
    .. versionchanged:: 3.10
       Removed the *loop* parameter.
 
+   .. versionchanged:: 3.11
+      Added the *ssl_shutdown_timeout* parameter.
+
 
 .. coroutinefunction:: start_server(client_connected_cb, host=None, \
                           port=None, *, limit=None, \
                           family=socket.AF_UNSPEC, \
                           flags=socket.AI_PASSIVE, sock=None, \
                           backlog=100, ssl=None, reuse_address=None, \
                           reuse_port=None, ssl_handshake_timeout=None, \
-                          start_serving=True)
+                          ssl_shutdown_timeout=None, start_serving=True)
 
    Start a socket server.
 
@@ -121,12 +125,15 @@ and work with streams:
    .. versionchanged:: 3.10
       Removed the *loop* parameter.
 
+   .. versionchanged:: 3.11
+      Added the *ssl_shutdown_timeout* parameter.
+
 
 .. rubric:: Unix Sockets
 
 .. coroutinefunction:: open_unix_connection(path=None, *, limit=None, \
                         ssl=None, sock=None, server_hostname=None, \
-                        ssl_handshake_timeout=None)
+                        ssl_handshake_timeout=None, ssl_shutdown_timeout=None)
 
    Establish a Unix socket connection and return a pair of
    ``(reader, writer)``.
@@ -150,10 +157,14 @@ and work with streams:
    .. versionchanged:: 3.10
       Removed the *loop* parameter.
 
+  .. versionchanged:: 3.11
+     Added the *ssl_shutdown_timeout* parameter.
+
 
 .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \
                           *, limit=None, sock=None, backlog=100, ssl=None, \
-                          ssl_handshake_timeout=None, start_serving=True)
+                          ssl_handshake_timeout=None, \
+                          ssl_shutdown_timeout=None, start_serving=True)
 
    Start a Unix socket server.
 
@@ -176,6 +187,9 @@ and work with streams:
    .. versionchanged:: 3.10
       Removed the *loop* parameter.
 
+   .. versionchanged:: 3.11
+      Added the *ssl_shutdown_timeout* parameter.
+
 
 StreamReader
 ============

Doc/library/asyncio-stream.rst

@@ -320,9 +320,11 @@ Writing and reading a bzip2-compressed file in binary mode:
     >>> with bz2.open("myfile.bz2", "wb") as f:
     ...     # Write compressed data to file
     ...     unused = f.write(data)
+    ...
     >>> with bz2.open("myfile.bz2", "rb") as f:
     ...     # Decompress data from file
     ...     content = f.read()
+    ...
     >>> content == data  # Check equality to original object after round-trip
     True