How it works

The library · two worlds

The page and the worker

In a browser kernel, your Python runs in a Web Worker — a background thread with no DOM, no getUserMedia, and no Web Audio API. Those live only on the page's main thread. So Python can't reach the microphone or the speakers on its own.

The library · the bridge

The bridge is an anywidget

browseraudio is built on anywidget. Each widget is two halves that talk over the Jupyter Comm channel:

recorder.js
player.js

An ES module on the main thread. It calls getUserMedia / builds an AudioContext, and reads or writes the samples.

Recorder
Player

A Python anywidget.AnyWidget subclass in the worker. Its traitlets mirror state to and from the frontend.

_pcm_b64

The audio itself: float32 PCM, base64-encoded, carried as a synced trait. Recording sends it worker-ward; playback sends it page-ward.

So record() captures on the page and pushes bytes up to Python; play() takes an array and pushes bytes down to an AudioContext. The kernel never touches an audio API — it only sets and reads traits.

The library · timing

Why recording needs two cells

The frontend's reply travels over the same Comm channel the kernel uses — but the kernel processes comm messages between cell executions, not during one. While the cell that calls record() is still running, the "here are your samples" message sits unread. Display the widget in one cell, then read rec.samples in the next, once the kernel is idle. Playback has no such constraint — nothing comes back — so play() is single-cell.

The library · drop-in

The sounddevice facade

browseraudio.sounddevice has the shape of sounddevice (play, wait, OutputStream, query_devices, default), so libraries can fall back to it:

try:
    import sounddevice as sd
except (OSError, ModuleNotFoundError):
    import browseraudio.sounddevice as sd

OutputStream works by making the browser the clock: an AudioWorklet plays a short queue and asks the kernel for each next block, which the kernel answers whenever no cell is running (demo). rec raises instead: a browser can't fill a buffer synchronously (two cells). On a bare main-thread Pyodide page, await rec.result() works in one cell.

Running it · the notebook stack

How you'd run browseraudio for real

In day-to-day use you don't wire any of this up — you pip install browseraudio in a browser notebook and call record() / play(). That kernel is itself a stack, the Python sitting on the interpreter sitting on the browser:

JupyterLite and thebe are the usual hosts; Pyodide is the interpreter underneath both.

Running it · this demo

How this demo runs — on the real stack

The interactive pages run the real stack above — thebe + thebe-lite + Pyodide, vendored under vendor/thebe-dist/ — rather than faking it. On the first Run, assets/live.js boots the kernel and micropip-installs the actual browseraudio package, so the cells run the genuine API: record() mounts the real Recorder anywidget, play() the real Player, keyboard() the real Keyboard, and stream() the real Stream — exactly what you'd get in a notebook. Genie mode runs Piano Genie (Donahue, Simon & Dieleman, IUI 2019) — Magenta's trained model, executed in plain JavaScript in the page.

// assets/live.js — on first Run
await loadScript("vendor/thebe-dist/lite/thebe-lite.min.js");   // + Pyodide
await kernel.requestExecute({ code: 'await micropip.install("browseraudio")' });

The runtime is a one-time ~25 MB download (cached after); reload the page to reset to a clean kernel. The cells are editable — change the code and re-run. (The CodeMirror editors are styled to match the static blocks via assets/live.css.)

Run it yourself

Serve the folder over localhost

Open it over localhost (not as a file:// URL — microphone access needs a secure context):

cd demo
python3 -m http.server 8000 --bind 127.0.0.1
# then visit http://127.0.0.1:8000/

Any static host works for deployment (GitHub Pages, Netlify, …) as long as it's served over https://.