python - Asyncio Making HTTP Requests Slower? -

python - Asyncio Making HTTP Requests Slower? -

i'm using asyncio , requests benchmark series of http requests.

for reason, it's slower utilize asyncio straight requests. thought why? using asyncio incorrectly?

import asyncio import functools import requests import time ts = time.time() in range(10): @asyncio.coroutine def do_checks(): loop = asyncio.get_event_loop() req = loop.run_in_executor(none, functools.partial(requests.get, "http://google.com", timeout=3)) resp = yield req print(resp.status_code) loop = asyncio.get_event_loop() loop.run_until_complete(do_checks()) te = time.time() print("version a: " + str(te - ts)) ts = time.time() in range(10): r = requests.get("http://google.com", timeout=3) print(r.status_code) te = time.time() print("version b: " + str(te - ts))

output:

version = asyncio; version b = requests

200 200 200 200 200 200 200 200 200 200 version a: 5.7215821743011475 200 200 200 200 200 200 200 200 200 200 version b: 5.320340156555176

you waiting each request finish before start next one. have overhead of event loop no benefits.

try this:

import asyncio import functools import requests import time ts = time.time() loop = asyncio.get_event_loop() @asyncio.coroutine def do_checks(): futures = [] in range(10): futures.append(loop.run_in_executor(none, functools.partial(requests.get, "http://google.com", timeout=3))) req in asyncio.as_completed(futures): resp = yield req print(resp.status_code) loop.run_until_complete(do_checks()) te = time.time() print("version a: " + str(te - ts)) ts = time.time() in range(10): r = requests.get("http://google.com", timeout=3) print(r.status_code) te = time.time() print("version b: " + str(te - ts))

this when run it:

$ python test.py 200 ... version a: 0.43438172340393066 200 ... version b: 1.6541109085083008

much faster, spawning threads , waiting http library finish, don't need asyncio that.

you might want checkout aiohttp built utilize asyncio. requests fabulous library, not made asyncio.

python performance python-3.x benchmarking python-asyncio