我想在我的多线程Python应用程序中定期执行操作。我已经看到了两种不同的方法
exit = False
def thread_func():
while not exit:
action()
time.sleep(DELAY)
要么
exit_flag = threading.Event()
def thread_func():
while not exit_flag.wait(timeout=DELAY):
action()
一种方法比另一种方法有优势吗?是使用更少的资源还是与其他线程和GIL更好地协作?哪一个使我的应用程序中的其余线程响应更快?
(假设有一些外部事件集exit
或exit_flag
,我愿意在关闭时等待完整的延迟)
使用exit_flag.wait(timeout=DELAY)
将具有更高的响应速度,因为在exit_flag
设置时,您将立即退出while循环。使用time.sleep
,即使在设置了事件之后,您也将在time.sleep
通话中等待直到睡了DELAY
几秒钟。
在实现方面,Python 2.x和Python 3.x具有非常不同的行为。在Python
2.xEvent.wait
中,使用大量的小time.sleep
调用在纯Python中实现:
from time import time as _time, sleep as _sleep
....
# This is inside the Condition class (Event.wait calls Condition.wait).
def wait(self, timeout=None):
if not self._is_owned():
raise RuntimeError("cannot wait on un-acquired lock")
waiter = _allocate_lock()
waiter.acquire()
self.__waiters.append(waiter)
saved_state = self._release_save()
try: # restore state no matter what (e.g., KeyboardInterrupt)
if timeout is None:
waiter.acquire()
if __debug__:
self._note("%s.wait(): got it", self)
else:
# Balancing act: We can't afford a pure busy loop, so we
# have to sleep; but if we sleep the whole timeout time,
# we'll be unresponsive. The scheme here sleeps very
# little at first, longer as time goes on, but never longer
# than 20 times per second (or the timeout time remaining).
endtime = _time() + timeout
delay = 0.0005 # 500 us -> initial delay of 1 ms
while True:
gotit = waiter.acquire(0)
if gotit:
break
remaining = endtime - _time()
if remaining <= 0:
break
delay = min(delay * 2, remaining, .05)
_sleep(delay)
if not gotit:
if __debug__:
self._note("%s.wait(%s): timed out", self, timeout)
try:
self.__waiters.remove(waiter)
except ValueError:
pass
else:
if __debug__:
self._note("%s.wait(%s): got it", self, timeout)
finally:
self._acquire_restore(saved_state)
实际上,这意味着使用wait
它可能比仅DELAY
无条件地休眠整个计算机要消耗更多的CPU
,但这样做的好处是(DELAY
响应时间长很多,取决于时间长短)。这也意味着需要经常重新获取GIL,以便可以安排下一次睡眠,同时time.sleep
可以完全释放GIL
DELAY
。现在,更频繁地获取GIL是否会对您应用程序中的其他线程产生明显影响?也许不是。这取决于正在运行的其他线程数以及它们承担的工作量。我的猜测是,除非您有大量线程,或者另一个线程正在执行大量CPU限制工作,否则它不会特别引人注目,但是它很容易同时尝试和观察。
在Python 3.x中,大部分实现都移至纯C代码:
import _thread # C-module
_allocate_lock = _thread.allocate_lock
class Condition:
...
def wait(self, timeout=None):
if not self._is_owned():
raise RuntimeError("cannot wait on un-acquired lock")
waiter = _allocate_lock()
waiter.acquire()
self._waiters.append(waiter)
saved_state = self._release_save()
gotit = False
try: # restore state no matter what (e.g., KeyboardInterrupt)
if timeout is None:
waiter.acquire()
gotit = True
else:
if timeout > 0:
gotit = waiter.acquire(True, timeout) # This calls C code
else:
gotit = waiter.acquire(False)
return gotit
finally:
self._acquire_restore(saved_state)
if not gotit:
try:
self._waiters.remove(waiter)
except ValueError:
pass
class Event:
def __init__(self):
self._cond = Condition(Lock())
self._flag = False
def wait(self, timeout=None):
self._cond.acquire()
try:
signaled = self._flag
if not signaled:
signaled = self._cond.wait(timeout)
return signaled
finally:
self._cond.release()
和获得锁的C代码:
/* Helper to acquire an interruptible lock with a timeout. If the lock acquire
* is interrupted, signal handlers are run, and if they raise an exception,
* PY_LOCK_INTR is returned. Otherwise, PY_LOCK_ACQUIRED or PY_LOCK_FAILURE
* are returned, depending on whether the lock can be acquired withing the
* timeout.
*/
static PyLockStatus
acquire_timed(PyThread_type_lock lock, PY_TIMEOUT_T microseconds)
{
PyLockStatus r;
_PyTime_timeval curtime;
_PyTime_timeval endtime;
if (microseconds > 0) {
_PyTime_gettimeofday(&endtime);
endtime.tv_sec += microseconds / (1000 * 1000);
endtime.tv_usec += microseconds % (1000 * 1000);
}
do {
/* first a simple non-blocking try without releasing the GIL */
r = PyThread_acquire_lock_timed(lock, 0, 0);
if (r == PY_LOCK_FAILURE && microseconds != 0) {
Py_BEGIN_ALLOW_THREADS // GIL is released here
r = PyThread_acquire_lock_timed(lock, microseconds, 1);
Py_END_ALLOW_THREADS
}
if (r == PY_LOCK_INTR) {
/* Run signal handlers if we were interrupted. Propagate
* exceptions from signal handlers, such as KeyboardInterrupt, by
* passing up PY_LOCK_INTR. */
if (Py_MakePendingCalls() < 0) {
return PY_LOCK_INTR;
}
/* If we're using a timeout, recompute the timeout after processing
* signals, since those can take time. */
if (microseconds > 0) {
_PyTime_gettimeofday(&curtime);
microseconds = ((endtime.tv_sec - curtime.tv_sec) * 1000000 +
(endtime.tv_usec - curtime.tv_usec));
/* Check for negative values, since those mean block forever.
*/
if (microseconds <= 0) {
r = PY_LOCK_FAILURE;
}
}
}
} while (r == PY_LOCK_INTR); /* Retry if we were interrupted. */
return r;
}
该实现是响应式的,不需要频繁唤醒就可以重新获取GIL,因此您可以两全其美。
在C语言中,假设每个算法被赋予完全相同的一组进程,那么先到先得、最短作业优先和循环之间的周转时间是否相等?还是调度算法不同?
问题内容: 为了为 HTML5 Doctype 定义字符集,我应该使用哪种表示法? 短: 长: 问题答案: 在HTML5中,它们是等效的。使用较短的一个,更容易记住和键入。浏览器支持很好,因为它是为向后兼容而设计的。
连接的多个输入都相当于Yes的时候才会输出Yes。 用法 Your browser does not support the video tag. 案例:小闹钟 功能:今天15:10:00,响起猫叫声小闹钟 工作原理 当所有的输入都是Yes的时候,与节点才输出Yes。
问题内容: 似乎有三种 相同的 方法可以独立于平台获取依赖于平台的“文件分隔符”: 我们如何决定何时使用哪个? 它们之间甚至有什么区别吗? 问题答案: 可以通过调用命令行参数或使用命令行参数覆盖 获取默认文件系统的分隔符。 获取默认文件系统。 获取文件系统的分隔符。请注意,作为一种实例方法,在需要代码在一个JVM中对多个文件系统进行操作的情况下,可以使用该方法将不同的文件系统传递给代码(而不是默认
lxml requests
问题内容: 我今天刚刚与一些同事讨论了python的db-api fetchone vs fetchmany vs fetchall。 我确定每个应用程序的用例都取决于我正在使用的db-api的实现,但是总的来说,fetchone,fetchmany,fetchall的用例是什么? 换句话说,以下等效项是什么?还是其中之一比其他人更受青睐?如果是这样,在哪些情况下? 问题答案: 我认为这确实取决于