- Dec 2023
-
www.bitecode.dev www.bitecode.dev
-
The code isn't that different from your typical asyncio script:
``` import re import time
import httpx import trio
urls = [ "https://www.bitecode.dev/p/relieving-your-python-packaging-pain", "https://www.bitecode.dev/p/hype-cycles", "https://www.bitecode.dev/p/why-not-tell-people-to-simply-use", "https://www.bitecode.dev/p/nobody-ever-paid-me-for-code", "https://www.bitecode.dev/p/python-cocktail-mix-a-context-manager", "https://www.bitecode.dev/p/the-costly-mistake-so-many-makes", "https://www.bitecode.dev/p/the-weirdest-python-keyword", ]
title_pattern = re.compile(r"<title[^>]>(.?)</title>", re.IGNORECASE)
user_agent = ( "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0" )
async def fetch_url(url): start_time = time.time()
async with httpx.AsyncClient() as client: headers = {"User-Agent": user_agent} response = await client.get(url, headers=headers) match = title_pattern.search(response.text) title = match.group(1) if match else "Unknown" print(f"URL: {url}\nTitle: {title}") end_time = time.time() elapsed_time = end_time - start_time print(f"Time taken for {url}: {elapsed_time:.4f} seconds\n")
async def main(): global_start_time = time.time()
# That's the biggest API difference async with trio.open_nursery() as nursery: for url in urls: nursery.start_soon(fetch_url, url) global_end_time = time.time() global_elapsed_time = global_end_time - global_start_time print(f"Total time taken for all URLs: {global_elapsed_time:.4f} seconds")
if name == "main": trio.run(main) ```
Because it doesn't create nor schedule coroutines immediately (notice the
nursery.start_soon(fetch_url, url)
is notnursery.start_soon(fetch_url(url)))
, it will also consume less memory. But the most important part is the nursery:# That's the biggest API difference async with trio.open_nursery() as nursery: for url in urls: nursery.start_soon(fetch_url, url)
The
with
block scopes all the tasks, meaning everything that is started inside that context manager is guaranteed to be finished (or terminated) when it exits. First, the API is better than expecting the user to wait manually like withasyncio.gather
: you cannot start concurrent coroutines without a clear scope in trio, it doesn't rely on the coder's discipline. But under the hood, the design is also different. The whole bunch of coroutines you group and start can be canceled easily, because trio always knows where things begin and end.As soon as things get complicated, code with curio-like design become radically simpler than ones with asyncio-like design.
-
trio
For many years, the very talented dev and speaker David Beazley has been showing unease with asyncio's design, and made more and more experiments and public talks about what could an alternative look like. It culminated with the excellent Die Threads presentation, live coding the sum of the experience of all those ideas, that eventually would become the curio library. Watch it. It’s so good.
Trio is not compatible with asyncio, nor gevent or twisted by default. This means it's also its little own async island.
But in exchange for that, it provides a very different internal take on how to deal with this kind of concurrency, where every coroutine is tied to an explicit scope, everything can be awaited easily, or canceled.
-
So what's the deal with asyncio, twisted, gevent, trio and all that stuff?
asyncio
asyncio is the modern module for asynchronous network programming provided with the python stdlib since 3.4. In other words, it's the default stuff at your disposal if you want to code something without waiting on the network.
asyncio replaces the old deprecated asyncore module. It is quite low level, so while you can manually code most network-related things with it, you are still at the level of TCP or UDP. If you want higher-level protocols, like FTP, HTTP or SSH, you have to either code it yourself, or install a third party library or module.
Because asyncio is the default solution, it has a the biggest ecosystem of 3rd party libs, and pretty much everything async strives to be compatible with it directly, or through compatibility layers like anyio.
Twisted
20 years ago, there was no asyncio, there was no async/await, nodejs didn't exist and Python 3 was half a decade away. Yet, it was the .com bubble, everything needed to be connected now. And so was born twisted, the grandfather of all the asynchronous frameworks we have today. Twisted ecosystem grew to include everything, from mail to ssh.
To this day, twisted is still a robust and versatile tool. But you do pay the price of its age. It doesn't follow PEP8 very well, and the design lean on the heavy size.
Tornado
Tornado was developed after Twisted, by FriendFeed, at this weird 2005-2015 web dev period where everything needed to be social web scale. It was like Twisted, but tooted to be faster, and was higher level. Out of the box, the HTTP story is way nicer.
Today, you are unlikely to use Tornado unless you work at Facebook or contribute to jupyter. After all, if you want to make async web things, the default tool is FastAPI in 2023.
gevent
Gevent came about in 2009, the same year as Tornado, but with a fundamentally different design. Instead of attempting to provide an asychronous API, it decided to do black magic. When you use gevent, you call
from gevent import monkey; monkey.patch_all()
and it changes the underlying mechanism of Python networking, making everything non-blocking.
-
- Jun 2023
-
-
Die Agenda für die COP28, die erst am Ende der Bonner Vorbereitungskonferenz beschlossen wurde, die in den vergangenen beiden Wochen stattfand, sieht weder Verhandlungen über emissionsminderungen noch solche über die finanzielle Unterstützung ärmererer Länder vor. Damit haben sich die Gegner von Verhandlungen über wirksame Schritte zur Dekarbonisierung durchgesetzt https://taz.de/Naechste-Weltklimakonferenz/!5938710/
-
- Mar 2022
-
musicbrainz.org musicbrainz.org
-
Witzig: 2 Drumsets gleichzeitig im Einsatz – bei klassischem Piano-Trio-Jazz :-)
-
- Dec 2018
-
www.nature.com www.nature.com
-
Insights into imprinting from parent-of-origin phased methylomes and transcriptomes
Tags
Annotators
URL
-
- Jun 2017
-
www.soultrainonline.de www.soultrainonline.de
-
-
www.soultrainonline.de www.soultrainonline.de
-
Horst Hansen Trio: Auf'm Teppich [oooo]
VÖ: 26.5.2017
AppleMusic / iTunes
-