有没有办法提高for循环的速度

Question

我需要发布一系列XML元素（每次调用的数量可能不同）。我正在Python3.9.1上使用Requests v2.25.1。虽然我的解决方案有效，但执行大约需要27秒，尽管requests模块显示的完成时间不到1秒。我已经证实的事情：

• r.text编码问题不存在
• 标题信息是正确的
• 创建requests.Session()不会提高响应时间

当运行邮递员，我也看到结果在不到1秒。我已将此问题与我正在运行的for loop隔离。数据从SQL中提取，存储为变量，然后for循环处理该数据并在我的XML请求中运行

我的问题是，我下面的代码是否是最佳实践，或者是否有一种更“pythonian”的方式来完成我希望完成的任务？感谢您的指导

    skill_id=[]
    agent_state=[]
    agent_name=[]

    for db_users in db_results:
       skill_id.append(db_users[0])
       agent_state.append(db_users[1])
       agent_name.append(db_users[2])
    if db_users[1] == 'NOT_READY':
       try:
            cursor.execute(sqlskillgroup)
            skillgroups = []
            for sg_query_result in cursor.fetchall():
                sg = sg_query_result[0]
                skillgroups.append(sg)

    except pyodbc.Error as e:
            print("Error retreiving skill group information from database.")
            quit()

    finally:
        connect.close()
    icm_url = "https://url_of_post"
    xmlfile = open('skill_remove.xml', 'r')
    body = xmlfile.read()
    icm_header = {
           'Content-Type': 'application/xml',
           'Authorization': 'Basic Q2hhZF9NZXllckBhamcuY29tOmNpc2Nv',
           'Cookie': 'JSESSIONID=0C8456E3901DF7A3A0862E17FD50547D'
           }

    for sgid in skillgroups:
        r = requests.request("POST",icm_url, headers = icm_header,
            data = body.format(agent = str(skill_id[0]), skill_urls = '<refURL>/unifiedconfig/config/skillgroup/' + str(sgid) + '</refURL>',),
            verify = r"CAchain.pem",
            cert = (r"cert.cer", r"cert.key"),
            )

    print(r.text)
    print(r.elapsed)

  

Answer 1

看起来您正在对循环中最后一个请求的运行时间进行计时。您可以通过将r.elapsed移动到循环中并对每个请求求和，或者通过将循环中的运行总计相加并在最后打印，来计算总运行时间

total_elapsed = 0
   for sgid in skillgroups:
    r = requests.request("POST",icm_url, headers = icm_header,
            data = body.format(agent = str(skill_id[0]), skill_urls = '<refURL>/unifiedconfig/config/skillgroup/' + str(sgid) + '</refURL>',),
            verify = r"CAchain.pem",
            cert = (r"cert.cer", r"cert.key"),
            )
    total_elapsed += r.elapsed

基本异步IO和aiohttp实现

import asyncio
import aiohttp

async def post_request(session, url):
    async with session.post() as request: #add your request headers, certificate etc
        await request.status

async def main():
    async with aiohttp.ClientSession() as session: # use client session to auto close at the end
        tasks = []
        for url in urls:
            t = asyncio.create_task(post_request(session, url)) #create a number of tasks to run concurrently
            tasks.append(t)

        await asyncio.gather(*tasks) # wait for all tasks to finish before close the session

asyncio.run(main())