Merge branch 'main' into bg/restore-rel-to

Author: barneygale

Doc/c-api/bytes.rst

@@ -219,3 +219,162 @@ called with a non-bytes parameter.
    reallocation fails, the original bytes object at *\*bytes* is deallocated,
    *\*bytes* is set to ``NULL``, :exc:`MemoryError` is set, and ``-1`` is
    returned.
+
+PyBytesWriter
+-------------
+
+The :c:type:`PyBytesWriter` API can be used to create a Python :class:`bytes`
+object.
+
+.. versionadded:: next
+
+.. c:type:: PyBytesWriter
+
+   A bytes writer instance.
+
+   The API is **not thread safe**: a writer should only be used by a single
+   thread at the same time.
+
+   The instance must be destroyed by :c:func:`PyBytesWriter_Finish` on
+   success, or :c:func:`PyBytesWriter_Discard` on error.
+
+
+Create, Finish, Discard
+^^^^^^^^^^^^^^^^^^^^^^^
+
+.. c:function:: PyBytesWriter* PyBytesWriter_Create(Py_ssize_t size)
+
+   Create a :c:type:`PyBytesWriter` to write *size* bytes.
+
+   If *size* is greater than zero, allocate *size* bytes, and set the
+   writer size to *size*. The caller is responsible to write *size*
+   bytes using :c:func:`PyBytesWriter_GetData`.
+
+   On error, set an exception and return NULL.
+
+   *size* must be positive or zero.
+
+.. c:function:: PyObject* PyBytesWriter_Finish(PyBytesWriter *writer)
+
+   Finish a :c:type:`PyBytesWriter` created by
+   :c:func:`PyBytesWriter_Create`.
+
+   On success, return a Python :class:`bytes` object.
+   On error, set an exception and return ``NULL``.
+
+   The writer instance is invalid after the call in any case.
+   No API can be called on the writer after :c:func:`PyBytesWriter_Finish`.
+
+.. c:function:: PyObject* PyBytesWriter_FinishWithSize(PyBytesWriter *writer, Py_ssize_t size)
+
+   Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer
+   to *size* bytes before creating the :class:`bytes` object.
+
+.. c:function:: PyObject* PyBytesWriter_FinishWithPointer(PyBytesWriter *writer, void *buf)
+
+   Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer
+   using *buf* pointer before creating the :class:`bytes` object.
+
+   Set an exception and return ``NULL`` if *buf* pointer is outside the
+   internal buffer bounds.
+
+   Function pseudo-code::
+
+       Py_ssize_t size = (char*)buf - (char*)PyBytesWriter_GetData(writer);
+       return PyBytesWriter_FinishWithSize(writer, size);
+
+.. c:function:: void PyBytesWriter_Discard(PyBytesWriter *writer)
+
+   Discard a :c:type:`PyBytesWriter` created by :c:func:`PyBytesWriter_Create`.
+
+   Do nothing if *writer* is ``NULL``.
+
+   The writer instance is invalid after the call.
+   No API can be called on the writer after :c:func:`PyBytesWriter_Discard`.
+
+High-level API
+^^^^^^^^^^^^^^
+
+.. c:function:: int PyBytesWriter_WriteBytes(PyBytesWriter *writer, const void *bytes, Py_ssize_t size)
+
+   Grow the *writer* internal buffer by *size* bytes,
+   write *size* bytes of *bytes* at the *writer* end,
+   and add *size* to the *writer* size.
+
+   If *size* is equal to ``-1``, call ``strlen(bytes)`` to get the
+   string length.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+.. c:function:: int PyBytesWriter_Format(PyBytesWriter *writer, const char *format, ...)
+
+   Similar to :c:func:`PyBytes_FromFormat`, but write the output directly at
+   the writer end. Grow the writer internal buffer on demand. Then add the
+   written size to the writer size.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+
+Getters
+^^^^^^^
+
+.. c:function:: Py_ssize_t PyBytesWriter_GetSize(PyBytesWriter *writer)
+
+   Get the writer size.
+
+.. c:function:: void* PyBytesWriter_GetData(PyBytesWriter *writer)
+
+   Get the writer data: start of the internal buffer.
+
+   The pointer is valid until :c:func:`PyBytesWriter_Finish` or
+   :c:func:`PyBytesWriter_Discard` is called on *writer*.
+
+
+Low-level API
+^^^^^^^^^^^^^
+
+.. c:function:: int PyBytesWriter_Resize(PyBytesWriter *writer, Py_ssize_t size)
+
+   Resize the writer to *size* bytes. It can be used to enlarge or to
+   shrink the writer.
+
+   Newly allocated bytes are left uninitialized.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+   *size* must be positive or zero.
+
+.. c:function:: int PyBytesWriter_Grow(PyBytesWriter *writer, Py_ssize_t grow)
+
+   Resize the writer by adding *grow* bytes to the current writer size.
+
+   Newly allocated bytes are left uninitialized.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+   *size* can be negative to shrink the writer.
+
+.. c:function:: void* PyBytesWriter_GrowAndUpdatePointer(PyBytesWriter *writer, Py_ssize_t size, void *buf)
+
+   Similar to :c:func:`PyBytesWriter_Grow`, but update also the *buf*
+   pointer.
+
+   The *buf* pointer is moved if the internal buffer is moved in memory.
+   The *buf* relative position within the internal buffer is left
+   unchanged.
+
+   On error, set an exception and return ``NULL``.
+
+   *buf* must not be ``NULL``.
+
+   Function pseudo-code::
+
+       Py_ssize_t pos = (char*)buf - (char*)PyBytesWriter_GetData(writer);
+       if (PyBytesWriter_Grow(writer, size) < 0) {
+           return NULL;
+       }
+       return (char*)PyBytesWriter_GetData(writer) + pos;

Doc/faq/python-video-icon.png

@@ -3,6 +3,78 @@
 Remote debugging attachment protocol
 ====================================
 
+This protocol enables external tools to attach to a running CPython process and
+execute Python code remotely.
+
+Most platforms require elevated privileges to attach to another Python process.
+
+.. _permission-requirements:
+
+Permission requirements
+=======================
+
+Attaching to a running Python process for remote debugging requires elevated
+privileges on most platforms. The specific requirements and troubleshooting
+steps depend on your operating system:
+
+.. rubric:: Linux
+
+The tracer process must have the ``CAP_SYS_PTRACE`` capability or equivalent
+privileges. You can only trace processes you own and can signal. Tracing may
+fail if the process is already being traced, or if it is running with
+set-user-ID or set-group-ID. Security modules like Yama may further restrict
+tracing.
+
+To temporarily relax ptrace restrictions (until reboot), run:
+
+  ``echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope``
+
+.. note::
+
+   Disabling ``ptrace_scope`` reduces system hardening and should only be done
+   in trusted environments.
+
+If running inside a container, use ``--cap-add=SYS_PTRACE`` or
+``--privileged``, and run as root if needed.
+
+Try re-running the command with elevated privileges:
+
+  ``sudo -E !!``
+
+
+.. rubric:: macOS
+
+To attach to another process, you typically need to run your debugging tool
+with elevated privileges. This can be done by using ``sudo`` or running as
+root.
+
+Even when attaching to processes you own, macOS may block debugging unless
+the debugger is run with root privileges due to system security restrictions.
+
+
+.. rubric:: Windows
+
+To attach to another process, you usually need to run your debugging tool
+with administrative privileges. Start the command prompt or terminal as
+Administrator.
+
+Some processes may still be inaccessible even with Administrator rights,
+unless you have the ``SeDebugPrivilege`` privilege enabled.
+
+To resolve file or folder access issues, adjust the security permissions:
+
+  1. Right-click the file or folder and select **Properties**.
+  2. Go to the **Security** tab to view users and groups with access.
+  3. Click **Edit** to modify permissions.
+  4. Select your user account.
+  5. In **Permissions**, check **Read** or **Full control** as needed.
+  6. Click **Apply**, then **OK** to confirm.
+
+
+.. note::
+
+   Ensure you've satisfied all :ref:`permission-requirements` before proceeding.
+
 This section describes the low-level protocol that enables external tools to
 inject and execute a Python script within a running CPython process.
 

Doc/howto/remote_debugging.rst

@@ -468,7 +468,8 @@ Dialects support the following attributes:
 .. attribute:: Dialect.skipinitialspace
 
    When :const:`True`, spaces immediately following the *delimiter* are ignored.
-   The default is :const:`False`.
+   The default is :const:`False`.  When combining ``delimiter=' '`` with
+   ``skipinitialspace=True``, unquoted empty fields are not allowed.
 
 
 .. attribute:: Dialect.strict
@@ -637,7 +638,7 @@ done::
 .. rubric:: Footnotes
 
 .. [1] If ``newline=''`` is not specified, newlines embedded inside quoted fields
-   will not be interpreted correctly, and on platforms that use ``\r\n`` linendings
+   will not be interpreted correctly, and on platforms that use ``\r\n`` line endings
    on write an extra ``\r`` will be added.  It should always be safe to specify
    ``newline=''``, since the csv module does its own
    (:term:`universal <universal newlines>`) newline handling.

Doc/library/csv.rst

@@ -1843,9 +1843,9 @@ expression support in the :mod:`re` module).
    lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold`
    converts it to ``"ss"``.
 
-   The casefolding algorithm is
-   `described in section 3.13 'Default Case Folding' of the Unicode Standard
-   <https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G33992>`__.
+   The casefolding algorithm is `described in section 3.13.3 'Default Case
+   Folding' of the Unicode Standard
+   <https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G53253>`__.
 
    .. versionadded:: 3.3
 
@@ -2056,7 +2056,7 @@ expression support in the :mod:`re` module).
    property being one of "Lm", "Lt", "Lu", "Ll", or "Lo".  Note that this is different
    from the `Alphabetic property defined in the section 4.10 'Letters, Alphabetic, and
    Ideographic' of the Unicode Standard
-   <https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-4/#G91002>`_.
+   <https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-4/#G91002>`__.
 
 
 .. method:: str.isascii()
@@ -2196,9 +2196,9 @@ expression support in the :mod:`re` module).
    Return a copy of the string with all the cased characters [4]_ converted to
    lowercase.
 
-   The lowercasing algorithm used is
-   `described in section 3.13 'Default Case Folding' of the Unicode Standard
-   <https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G33992>`__.
+   The lowercasing algorithm used is `described in section 3.13.2 'Default Case
+   Conversion' of the Unicode Standard
+   <https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G34078>`__.
 
 
 .. method:: str.lstrip(chars=None, /)
@@ -2561,9 +2561,9 @@ expression support in the :mod:`re` module).
    character(s) is not "Lu" (Letter, uppercase), but e.g. "Lt" (Letter,
    titlecase).
 
-   The uppercasing algorithm used is
-   `described in section 3.13 'Default Case Folding' of the Unicode Standard
-   <https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G33992>`__.
+   The uppercasing algorithm used is `described in section 3.13.2 'Default Case
+   Conversion' of the Unicode Standard
+   <https://www.unicode.org/versions/Unicode17.0.0/core-spec/chapter-3/#G34078>`__.
 
 
 .. method:: str.zfill(width, /)

Doc/library/stdtypes.rst

@@ -17,8 +17,8 @@
 
 This module provides access to the Unicode Character Database (UCD) which
 defines character properties for all Unicode characters. The data contained in
-this database is compiled from the `UCD version 16.0.0
-<https://www.unicode.org/Public/16.0.0/ucd>`_.
+this database is compiled from the `UCD version 17.0.0
+<https://www.unicode.org/Public/17.0.0/ucd>`_.
 
 The module uses the same names and symbols as defined by Unicode
 Standard Annex #44, `"Unicode Character Database"
@@ -211,6 +211,6 @@ In addition, the module exposes the following constant:
 
 .. rubric:: Footnotes
 
-.. [#] https://www.unicode.org/Public/16.0.0/ucd/NameAliases.txt
+.. [#] https://www.unicode.org/Public/17.0.0/ucd/NameAliases.txt
 
-.. [#] https://www.unicode.org/Public/16.0.0/ucd/NamedSequences.txt
+.. [#] https://www.unicode.org/Public/17.0.0/ucd/NamedSequences.txt

Doc/library/unicodedata.rst

@@ -384,8 +384,8 @@ Character Database.
 
 
 .. _UAX-31: https://www.unicode.org/reports/tr31/
-.. _PropList.txt: https://www.unicode.org/Public/16.0.0/ucd/PropList.txt
-.. _DerivedCoreProperties.txt: https://www.unicode.org/Public/16.0.0/ucd/DerivedCoreProperties.txt
+.. _PropList.txt: https://www.unicode.org/Public/17.0.0/ucd/PropList.txt
+.. _DerivedCoreProperties.txt: https://www.unicode.org/Public/17.0.0/ucd/DerivedCoreProperties.txt
 .. _normalization form: https://www.unicode.org/reports/tr15/#Norm_Forms
 
 
@@ -793,7 +793,7 @@ with the given *name*::
 This sequence cannot appear in :ref:`bytes literals <bytes-literal>`.
 
 .. versionchanged:: 3.3
-   Support for `name aliases <https://www.unicode.org/Public/16.0.0/ucd/NameAliases.txt>`__
+   Support for `name aliases <https://www.unicode.org/Public/17.0.0/ucd/NameAliases.txt>`__
    has been added.
 
 .. _string-escape-long-hex: