It may be wrong to use async/await or coroutine to execute low-level I/O.

I have spent several years on Node.js and libuv. I felt desperately that callback functions , Promise and async/await in Node.js work pretty well all the time. The callback functions is responsible for the low-level I/O, the data are read and written chunk by chunk. The Promise is responsible to wrap multiple callback functions to make a atomic I/O operation. And the async/await is responsible only to form a logical workflow.

And I have two to three years experience in asyncdispatch , asyncnet , asynchttpserver in Nim and I have written several asynchronous modules. Now, I’m writting an asynchronous Mysql client/connector in pure Nim. However, I now feel that it is not appropriate to use async/await entirely.

Now, let us suppose that we use a mysql client to execute some queries. This is a pseudo code:

var conn = newAsyncSocket()



proc query(sql: string ): Stream =

await conn.send(mysql_format_function(sql))

let data = await conn.recv_mysql_function()

mysql_parse_function(data)

return newStream(conn)



proc main() =

let stream1 = await query( "select 100; select 200;" )

while true :

let data = await stream1.read()

echo data

if data == "" :

break



let stream2 = await query( "select 300; select 400;" )

while true :

let data = await stream2.read()

echo data

if data == "" :

break



waitFor main()



The code work fine if stream2 is always executing after stream1 . However, if not:

proc main() =

let stream1 = await query( "select 100; select 200;" )

let stream2 = await query( "select 300; select 400;" )



while true :

let data = await stream1.read()

echo data

if data == "" :

break



while true :

let data = await stream2.read()

echo data

if data == "" :

break



or

proc do1() =

let stream1 = await query( "select 100; select 200;" )

while true :

let data = await stream1.read()

echo data

if data == "" :

break



proc do2() =

let stream2 = await query( "select 300; select 400;" )

while true :

let data = await stream2.read()

echo data

if data == "" :

break



proc main() =

asyncCheck do1()

asyncCheck do2()



What happened then? The main function will blocking at (await) stream2 because stream1 is never finished. What does finished mean? The stream1 finished is mean that stream1 recv all the data that belong to the query of select 100; select 200; .

stream1 data stream2 data

|......................|....................|

| | |

v V V

stream1 begin stream1 finished stream2 finished

stream2 begin



A solution is to use lock or queue cache. This can solve the second problems, but it also introduce data race:

proc do1() =

await conn.lock()

let stream1 = await query( "select 100; select 200;" )



conn.unlock()



proc do2() =

await conn.lock()

let stream1 = await query( "select 100; select 200;" )



conn.unlock()



And it cannot prevent programmers to write the first kind of code:

proc main() =

let stream1 = await query( "select 100; select 200;" )

let stream2 = await query( "select 300; select 400;" )



Each of your asynchronous functions must be atomic. Otherwise, there are traps in them. The atomic function is mean that you can not construct streaming interfaces:

proc main() =

let all_data1 = await query( "select 100; select 200;" )

let all_data2 = await query( "select 300; select 400;" )



It is very bad to transfer large data, and it is a huge issue whenever you use async/await to write any program .

Ideally, using some callback functions to execute low-level I/O, using some Future to wrap these callback functions into atomic operations, and using async/await to construct the final logical workflow, then you can get the perfect solution: