The Future of Asynchronous Programming in R - Charlie Gao
Asynchronous programming can be a powerful paradigm, whereby computations are allowed to run concurrently without blocking the main session. It is an opportune time to survey the current landscape, as R infrastructure in this respect has matured significantly over recent years.
Instead of running a script sequentially from top to bottom, logic that takes a long or unpredictable amount of time to complete may be offloaded to different R processes, possibly on other computers or in the cloud. In the meantime, the main session may be running constantly and non-interactively, performing operations in real time, synchronizing with these tasks only when necessary.
This style of programming requires a very specific set of tooling. At the very base, there is an infrastructure layer involving key enabling packages such as later and mirai. It will be explained at a high level why these two packages together currently offer the most complete and efficient implementation of async for the R language.
There are further tools which expand async functionality to cover specific needs, such as the watcher package for filesystem monitoring. There are then a range of tools built on top of these, bringing async capabilities to the end-user, such as the httr2 package for querying APIs and the ellmer package for interacting with LLMs.
In addition to these existing tools, exciting developments in asynchronous programming are just around the corner. These will be previewed, together with speculation on what might be possible at some point in the future
ellmer
httr2
mirai