Skip to content

free threading stop_the_world can be slow to complete #162052

Open

Description

@johng

When a stop-the-world is triggered, most commonly in a GC cycle, it can be
slow to take effect because it falls back to the timeout on its notification
event.

Stop-the-world has the following high-level steps before confirming all
threads have stopped:

  1. Set the requested flag in the interpreter state and count the threads
    that need to stop.
  2. Scan the threads. DETACHED threads are suspended and taken off the count;
    ATTACHED threads have a stop bit set in their eval_breaker.
  3. Wait on PyEvent_WaitTimed(&stw->stop_event, 1 ms), then rescan on timeout.
    The event is set when all threads have suspended, via a countdown mechanism
    in decrement_stoptheworld_countdown.

A worker thread that is ATTACHED at step 2 and then detaches does not call
decrement_stoptheworld_countdown, so the stop_event wait times out.

queue.Queue is particularly at risk of this, since a thread blocking in
Queue.get() spends a fairly long time attached, with no eval-breaker check,
before it detaches. This makes it likely to still be ATTACHED at step 2.

import queue
import statistics
import sys
import threading
import time

done = False


def ping_pong(inbox, outbox):
    while not done:
        try:
            outbox.put(inbox.get(timeout=0.1))
        except queue.Empty:
            pass


a, b = queue.Queue(), queue.Queue()
threads = [threading.Thread(target=ping_pong, args=(a, b)),
           threading.Thread(target=ping_pong, args=(b, a))]
for t in threads:
    t.start()
a.put("token")

limit = sys.getrecursionlimit()
times_us = []
for _ in range(1000):
    t0 = time.perf_counter_ns()
    sys.setrecursionlimit(limit)          # stops the world
    times_us.append((time.perf_counter_ns() - t0) / 1000)
    time.sleep(0.0002)

done = True
for t in threads:
    t.join()

slow = sum(t >= 900 for t in times_us)
print(f"GIL enabled: {sys._is_gil_enabled()}")
print(f"median stop: {statistics.median(times_us):7.1f} us")
print(f"stops that took the 1 ms poll: {slow} of {len(times_us)}")

GIL enabled: False
median stop: 1263.6 us
stops that took the 1 ms poll: 880 of 1000

One proposed solution is for a thread that detaches while a stop-the-world is
requested to suspend itself and call decrement_stoptheworld_countdown().
Detach is a critical code path, so I am unsure whether this was intentionally
left out. The cost when no stop is requested would be an atomic read of the
requested flag. In a prototype, the example above gives:

GIL enabled: False
median stop:    15.4 us
stops that took the 1 ms poll: 0 of 1000

Another option is to accept the current wait but tune the timeout, as the code
comment alludes to ("1ms (arbitrary, may need tuning)"). The trade-off is that
a shorter timeout means more wake-ups and rescans of the thread list while
threads are still running.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions