• Python中巧用retry
  • 发布于 2个月前
  • 240 热度
    0 评论

郑秀文说,漫漫人生路总会错几步。而我们跑自动化也是如此,特别是UI自动化,因为各种客观因素,就fail了,但是不一定是真的fail了,如果一切可以重来,结局也许是不一样的。那么,你就得retry了。


如果自动化中失败了一次,我们给机会了,成功了就既往不咎。只能说明不够稳定。那么retry该如何写呢?

小明刚开始学爬虫的时候,发现请求有很多时候不能得到正确的返回。他这样写的。
def crawl_page(url):
    pass

def log_error(url):
    pass

url = ""
try:
   crawl_page(url)
except:
    log_error(url)

发现很多时候请求一次,得到的是exception,但是多请求两次,就得到了正确的结果。
于是,改进一下:
attempts = 0
success = False
while attempts < 3 and not success:
    try:
        crawl_page(url)
        success = True
    except:
        attempts += 1
        if attempts == 3:
            break

用一个循环来判断函数是否要多次执行。虽然问题解决了,但是不通用,于是乎,他自己写了个装饰器来实现retry的功能:
def retry(attempt):
    def decorator(func):
        def wrapper(*args, **kw):
            att = 0
            while att < attempt:
                try:
                    return func(*args, **kw)
                except Exception as e:
                    att += 1
        return wrapper
    return decorator


@rety(attempt=3)
def crawl_page(url):
    pass

他发现,这样用起来很顺手,后面又改进了一下,变成了这样。
wait_time = 3
max_retry_times =5
import time

from functools import wraps
def retry_for_errors(errors=Exception, retry_times=max_retry_times,
                     poll_time=wait_time):
    """
    Decorator to retry for multiple errors.

    Example::

        @retry_for_errors(errors=(RuntimeError,NameError))
        def func():
            pass
    """

    assert retry_times > 0, 'retry_times must larger than 0!'

    def wrapper_(func):
        @wraps(wrapped=func)
        def wrapper(*args, **kwargs):
            retry = 1
            while retry <= retry_times:
                try:
                    return func(*args, **kwargs)
                except errors as exc:
                    msg = "Retry for {} for {} time...".format(type(exc).__name__, retry)
                    print(msg)
                    retry += 1

                    if retry > retry_times:
                        raise exc
                    else:
                        time.sleep(poll_time)
        return wrapper
    return wrapper_

可是后来发现,python 中有retry这个库,只要安装就可以了。
pip install retry

其中的解释如下:
def retry(exceptions=Exception, tries=-1, delay=0, max_delay=None, backoff=1, jitter=0, logger=logging_logger):
    """Return a retry decorator.
 
    :param exceptions: an exception or a tuple of exceptions to catch. default: Exception.
    :param tries: the maximum number of attempts. default: -1 (infinite).
    :param delay: initial delay between attempts. default: 0.
    :param max_delay: the maximum value of delay. default: None (no limit).
    :param backoff: multiplier applied to delay between attempts. default: 1 (no backoff).
    :param jitter: extra seconds added to delay between attempts. default: 0.
                   fixed if a number, random if a range tuple (min, max)
    :param logger: logger.warning(fmt, error, delay) will be called on failed attempts.
                   default: retry.logging_logger. if None, logging is disabled.
    """

直接调用就可以了,不需要重复造轮子。

真是磨刀不误砍柴工!
用户评论