Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Doc/c-api/structures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,22 @@ under :ref:`reference counting <countingrefs>`.
Do not use this field directly; use :c:macro:`Py_TYPE` and
:c:func:`Py_SET_TYPE` instead.

.. c:member:: PyMutex ob_mutex

A per-object lock, present only in the :term:`free-threaded <free threading>`
Comment thread
kumaraditya303 marked this conversation as resolved.
Outdated
Comment thread
vstinner marked this conversation as resolved.
Outdated
build (when :c:macro:`Py_GIL_DISABLED` is defined).

This field is **reserved for use by the critical section API**
(:c:macro:`Py_BEGIN_CRITICAL_SECTION` / :c:macro:`Py_END_CRITICAL_SECTION`).
Do **not** lock it directly with ``PyMutex_Lock``; doing so can cause
deadlocks. If you need your own lock, add a separate :c:type:`PyMutex`
field to your object struct.

See :ref:`per-object-locks` in the free-threading extension guide for
details.

.. versionadded:: 3.13
Comment thread
vstinner marked this conversation as resolved.
Outdated


.. c:type:: PyVarObject

Expand Down
49 changes: 49 additions & 0 deletions Doc/howto/free-threading-extensions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,55 @@ Important Considerations
internal extension state, standard mutexes or other synchronization
primitives might be more appropriate.

.. _per-object-locks:

Per-Object Locks (``ob_mutex``)
...............................

In the free-threaded build, each Python object contains a :c:member:`~PyObject.ob_mutex`
field of type :c:type:`PyMutex`. This mutex is **reserved for use by the
critical section API** (:c:macro:`Py_BEGIN_CRITICAL_SECTION` /
:c:macro:`Py_END_CRITICAL_SECTION`).

.. warning::

Do **not** lock ``ob_mutex`` directly with ``PyMutex_Lock(&obj->ob_mutex)``.
Mixing direct ``PyMutex_Lock`` calls with the critical section API on the
same mutex can cause deadlocks, because the critical section implementation
may suspend and release its locks when contention is detected.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because the critical section implementation may suspend and release its locks when contention is detected

I don't think this is related to why it may deadlock. I'd remove everything from ", because the ..." onwards.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed the issue.


Even if your own code never uses critical sections on a particular object type,
**CPython internals may use the critical section API on any Python object**.
For example, the garbage collector or other interpreter internals may enter a
critical section on your object. If your code holds ``ob_mutex`` directly at
that point, a deadlock can occur.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the garbage collector ever acquires critical sections on objects. Maybe just:

Even if your own code never uses critical sections on a particular object type,
CPython internals may use the critical section API on any Python object.
If your code holds ob_mutex directly at that point, a deadlock can occur.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your code holds ob_mutex directly at that point, a deadlock can occur.

It sounds redundant with the above warning. Is it worth it to say it again?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, seems fine to get rid of that as well to me.


If your extension type needs its own lock, add a separate :c:type:`PyMutex`
field (or another synchronization primitive) to your object struct.
:c:type:`PyMutex` is very lightweight — it is only one byte — so there is
negligible cost to having an additional one::

/* WRONG — do not lock ob_mutex directly */
Comment thread
kumaraditya303 marked this conversation as resolved.
Outdated
PyMutex_Lock(&obj->ob_mutex);
...
PyMutex_Unlock(&obj->ob_mutex);

/* RIGHT — use critical sections for ob_mutex */
Py_BEGIN_CRITICAL_SECTION(obj);
...
Py_END_CRITICAL_SECTION();

/* RIGHT — use your own mutex for your own state */
typedef struct {
PyObject_HEAD
PyMutex my_mutex; /* separate lock for extension state */
int my_data;
} MyObject;

PyMutex_Lock(&self->my_mutex);
self->my_data++;
PyMutex_Unlock(&self->my_mutex);


Building Extensions for the Free-Threaded Build
===============================================
Expand Down
Loading