I made 1000 HTTP requests using Goroutines and Python’s Coroutines

Used Go 1.6.2 and Python 3.6

Implemented in Go using net/http package

Implemented in Python using aiohttp, requests and urllib3 libraries

Ran it over $10 DigitalOcean droplet

Scroll to bottom of this post to see results.

Go

Go implementation using goroutines:

package main import ( "fmt" "net/http" ) func main () { url := "http://example.com" resc , errc := make ( chan bool ), make ( chan error ) for i := 0 ; i < 1000 ; i ++ { go func ( url string ) { _ , err := http . Head ( url ) if err != nil { errc <- err return } resc <- true }( url ) } for i := 0 ; i < 1000 ; i ++ { select { case <- resc : case err := <- errc : fmt . Println ( err ) } } }

Python

Python implementation using asyncio with aiohttp, requests and urllib3

1. aiohttp

import asyncio import aiohttp async def main (): async with aiohttp . ClientSession () as session : for i in range ( 1000 ): await session . head ( 'http://example.com' ) if __name__ == '__main__' : loop = asyncio . get_event_loop () loop . run_until_complete ( main ())

2. requests

import asyncio import requests async def main (): futures = [ loop . run_in_executor ( None , requests . get , 'http://example.com' ) for i in range ( 1000 ) ] return await asyncio . gather ( * futures ) if __name__ == '__main__' : loop = asyncio . get_event_loop () _ = loop . run_until_complete ( main ())

3. urllib3

import asyncio import urllib3 async def main ( http ): futures = [ loop . run_in_executor ( None , http . request , 'HEAD' , 'http://example.com' ) for i in range ( 1000 ) ] return await asyncio . gather ( * futures ) if __name__ == '__main__' : loop = asyncio . get_event_loop () http = urllib3 . PoolManager () _ = loop . run_until_complete ( main ( http ))

Analysis (Time)

1. Sync/Async HEAD requests (time in seconds)

2. Sync/Async GET requests (time in seconds)

Analysis (Memory)

1. Sync/Async HEAD requests (memory in MB)

2. Async/Sync GET requests (memory in MB)

Code for both synchronous and asynchronous implementations are at: https://github.com/vipul-sharma20/async-go-py