Beanstalkc超时问题

Question

我使用Python中的beanstalkc作为程序的队列进程，该程序必须解析url列表。因此，我在beanstalk中使用timeout以避免任何URL占用大量时间。但即使在使用它之后，我的进程也不会在限制内超时，而且要花很多时间解析几个url。我使用以下代码：

for seed in seedlist:
    print 'Put data: %s' % seed
    bean.put(seed,ttr =5)
while True: 
    job = bean.reserve() 
    spider.spider(job.body)
    print 'Got data: %s' % job.body

Answer 1

我想你误解了beanstalkd的TTR超时的目的。引用beanstalkd FAQ：

How does TTR work

TTR only applies to a job at the moment it becomes reserved. At that event, a timer (called “time-left” in the job stats) starts counting down from the job’s TTR.

• If the timer reaches zero, the job gets put back in the ready queue.
• If the job is buried, deleted, or released before the timer runs out, the timer ceases to exist.
• If the job is "touch"ed before the timer reaches zero, the timer starts over counting down from TTR.

(The job stats of a job that isn’t reserved still contain a “time-left” entry, but its value is meaningless.)

因此，TTR可以帮助您“避免任何URL占用大量时间”。它不会神奇地杀死您的工作进程。它所做的只是，如果工作线程在给定的时间跨度（TTR）之后没有将作业标记为已完成，则beanstalkd会将作业放回队列中。在