Embeddable place: listening port, file voice source, heartbeat stop

alloverse/allonet2#55 · Koja onboarding demo office
origin/main...HEAD · 13 files, +455 −116

KojaApp's onboarding demo office runs a PlaceServer inside the app and six scripted avatars that speak pre-recorded tracks. Three things an embedder needed were missing: the port an OS-assigned listener bound, a library-grade file voice source with one clock for several streams, and a place that really stops when told to. Additive API only; nothing on the wire changes.

Prologue

What an embedder runs into

Three problems, in the order the demo office hit them. PlaceServer(httpPort: 0) binds a free port, but PlaceServerHTTP.http is private, so nobody can learn which one; the only workaround was to reserve a port, release it and hope. voicedemo already read a file into 960-sample frames and looped it without a gap, but as an internal type of an executable, and with its own timer per stream, so six streams would drift apart. And PlaceServer.stop() never stopped its HeartbeatTimer, whose closure retains the server: every stopped place lived on at 1 Hz for the rest of the process.1

  1. Found while building the demo office: closing the onboarding window stopped the place, and a beat kept firing on the main actor afterwards. ↩︎
Chapter 1

The port the listener bound

FlyingFox's HTTPServer.listeningAddress is public but Socket.Address is a FlyingSocks type that FlyingFox does not re-export, hence the new product dependency 1. PlaceServerHTTP unwraps the two IP cases 2; a Unix socket, or no listener yet, reads as nil. PlaceServer.listeningPort is the public face 3, and the startup notice no longer prints http://localhost:0/ when the port is OS-assigned 4.

TestPlace is the first consumer: it used to bind a socket to port 0, read the number, close it and hope nothing took the port before the place did. Now it passes httpPort: 0 and waits for listeningPort 5; freePort() and isListening() are gone. EphemeralPortTests pins that a client can connect to the reported port.

Package.swiftPackage.swift
@@ -104,6 +104,8 @@ let package = Package(
104104 dependencies: [
105105 "PotentCodables",
106106 "FlyingFox",
107+ // FlyingFox does not re-export it, and HTTPServer.listeningAddress is one of its types.
108+ .product(name: "1FlyingSocks", package: "FlyingFox"),
107109 "AlloDataChannel",
108110 "Version",
109111 .product(name: "kvSIMD", package: "kvSIMD.swift"),
PlaceServerHTTP.swiftSources/allonet2/PlaceServer/PlaceServerHTTP.swift
@@ -7,6 +7,7 @@
77
88import Foundation
99import FlyingFox
10+import FlyingSocks
1011import Version
1112
1213public struct AppDescription
PlaceServerHTTP.swiftSources/allonet2/PlaceServer/PlaceServerHTTP.swift
@@ -53,6 +54,18 @@ class PlaceServerHTTP
5354 self.appDescription = appDescription
5455 self.assets = PlaceServerAssets(directory: assetsDirectory)
5556 }
57+
58+ /// The port the listening socket is bound to, or nil before `start()` has bound it.
59+ var listeningPort: UInt16? {
60+ get async {
61+ switch await http?.listeningAddress
62+ {
63+ 2case .ip4(_, port: let port), .ip6(_, port: let port): return port
64+ case .unix, .none: return nil
65+ }
66+ }
67+ }
68+
5669 func start() async throws
5770 {
5871 self.http = HTTPServer(port: port, timeout: Self.requestTimeout)
PlaceServer.swiftSources/allonet2/PlaceServer/PlaceServer.swift
@@ -97,10 +97,18 @@ public class PlaceServer : AlloSessionDelegate
9797 FileManager.default.temporaryDirectory.appendingPathComponent("alloplace-assets", isDirectory: true)
9898 }
9999
100+ /// The TCP port the HTTP listener (signalling, assets, dashboard) is bound to, or nil until
101+ /// `start()` has bound it. Pass `httpPort: 0` to let the OS pick a free port, and read it
102+ /// here; the port an embedder hands its clients cannot be known any other way.
103+ 3public var listeningPort: UInt16? {
104+ get async { await web.listeningPort }
105+ }
106+
100107 public func start() async throws
101108 {
102109 let myIp = options.ipOverride?.to ?? "localhost"
103− logger.notice("Serving '\(name)' at http://\(myIp):\(httpPort)/ and UDP ports \(options.portRange)")
110+ let address = httpPort == 0 ? "4a port the OS picks (see listeningPort)" : "http://\(myIp):\(httpPort)/"
111+ logger.notice("Serving '\(name)' at \(address) and UDP ports \(options.portRange)")
104112
105113 try await self.web.start()
106114 }
TestPlace.swiftTests/E2ESupport/TestPlace.swift
@@ -20,17 +20,20 @@ public final class TestPlace
2020
2121 public init() async throws
2222 {
23− // A fixed range per process would collide between test cases; ask the OS instead.
24− port = try Self.freePort()
25− server = PlaceServer(
23+ // A fixed range per process would collide between test cases; let the OS pick one, and
24+ // read back what it picked once the listener is bound.
25+ let server = PlaceServer(
2626 name: "Voice E2E",
27− httpPort: port,
27+ httpPort: 0,
2828 // Loopback only: this host gathers no candidates otherwise.
2929 options: TransportConnectionOptions(routing: .direct, bindAddress: "127.0.0.1"),
3030 alloAppAuthToken: ""
3131 )
32+ self.server = server
3233 Task { try await server.start() }
33− try await waitUntil(timeout: 10) { Self.isListening(on: self.port) }
34+ try await waitUntil(timeout: 10) { 5await server.listeningPort != nil }
35+ guard let port = await server.listeningPort else { throw TestPlaceError.noFreePort }
36+ self.port = port
3437 }
3538
3639 public func connectClient(named name: String) async throws -> TestClient
TestPlace.swiftTests/E2ESupport/TestPlace.swift
@@ -48,42 +51,6 @@ public final class TestPlace
4851 await server.stop()
4952 try? await Task.sleep(nanoseconds: 300_000_000)
5053 }
51−
52− private static func freePort() throws -> UInt16
53− {
54− let handle = socket(AF_INET, SOCK_STREAM, 0)
55− defer { close(handle) }
56− var address = sockaddr_in()
57− address.sin_family = sa_family_t(AF_INET)
58− address.sin_addr.s_addr = INADDR_ANY
59− address.sin_port = 0
60− var length = socklen_t(MemoryLayout<sockaddr_in>.size)
61− let bound = withUnsafePointer(to: &address) {
62− $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(handle, $0, length) }
63− }
64− guard bound == 0 else { throw TestPlaceError.noFreePort }
65− let named = withUnsafeMutablePointer(to: &address) {
66− $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(handle, $0, &length) }
67− }
68− guard named == 0 else { throw TestPlaceError.noFreePort }
69− return UInt16(bigEndian: address.sin_port)
70− }
71−
72− private static func isListening(on port: UInt16) -> Bool
73− {
74− let handle = socket(AF_INET, SOCK_STREAM, 0)
75− defer { close(handle) }
76− var address = sockaddr_in()
77− address.sin_family = sa_family_t(AF_INET)
78− address.sin_addr.s_addr = inet_addr("127.0.0.1")
79− address.sin_port = port.bigEndian
80− let connected = withUnsafePointer(to: &address) {
81− $0.withMemoryRebound(to: sockaddr.self, capacity: 1) {
82− connect(handle, $0, socklen_t(MemoryLayout<sockaddr_in>.size))
83− }
84− }
85− return connected == 0
86− }
8754}
8855
8956public enum TestPlaceError: Error, CustomStringConvertible
EphemeralPortTests.swiftTests/VoiceE2ETests/EphemeralPortTests.swiftadded
@@ -0,0 +1,26 @@
1+//
2+// EphemeralPortTests.swift
3+// allonet2
4+//
5+// An embedded place lets the OS pick its port, so `listeningPort` is the only way anything
6+// can be pointed at it. `TestPlace` is built that way, which is what this asserts.
7+//
8+
9+import Testing
10+import Foundation
11+import E2ESupport
12+@testable import allonet2
13+
14+@MainActor
15+@Suite struct EphemeralPortTests
16+{
17+ @Test func aPlaceOnAnOSChosenPortReportsItAndServesAClient() async throws
18+ {
19+ try await withPlace { place in
20+ let port = await place.server.listeningPort
21+ #expect(port != nil && port != 0)
22+ #expect(port == place.port)
23+ _ = try await place.connectClient(named: "ephemeral")
24+ }
25+ }
26+}
README.mdREADME.md
@@ -72,6 +72,8 @@ swift run AlloPlace -n "Local Place"
7272
7373* `-n <name>` — human-facing name of the place.
7474* `-p <port>` — TCP port for the HTTP listener (signalling, assets, dashboard; default 9080).
75+ Embedding `PlaceServer`: pass `httpPort: 0` to pick a free port, and read it back with
76+ `PlaceServer.listeningPort` after `start()`.
7577* `-u <min-max>` — UDP port range for WebRTC (default 10000-11000).
7678* `-t <token>` — token an AlloApp must present to be granted app privileges; omitted, any app
7779 that asks is authenticated. It is a role credential, not a connection gate: clients
Chapter 2

A file as a voice source

voicedemo's Recording moves into AlloAudio as public VoiceRecording, with its contract written down: nextFrame() returns exactly chunkFrameCount sample frames 1, looping inside the converter's input block so the seam is sample-continuous and no frame is short. The one behavioural change is that Failure carries a URL rather than a path string 2, so a caller that already holds a URL does not round-trip through a string. VoiceRecordingTests covers the loop seam with a 1000-sample file (one frame and a bit) and the failure naming the file.

VoiceRecording.swiftSources/AlloAudio/VoiceRecording.swiftrenamed from Sources/voicedemo/Recording.swift
@@ -1,8 +1,6 @@
11//
2−// Recording.swift
3−// allonet2
4−//
5−// A sound file as an endless source of voice frames; see VOICEDEMO_WAV in VoiceDemo.swift.
2+// VoiceRecording.swift
3+// AlloAudio
64//
75
86import AVFoundation
VoiceRecording.swiftSources/AlloAudio/VoiceRecording.swiftrenamed from Sources/voicedemo/Recording.swift
@@ -11,53 +9,68 @@ import allonet2
119
1210/// A sound file decoded on demand into voice frames: 20 ms of 48 kHz mono Float32, the shape
1311/// `DataChannelMediaStream.send(samples:frameCount:)` takes. The end of the file wraps to its
14−/// start inside the same conversion pass, so the loop is sample-continuous at the seam.
12+/// start inside the same conversion pass, so the loop is sample-continuous at the seam and no
13+/// frame is ever short.
14+///
15+/// `VoiceRecordingPlayer` drives one or more of these on a shared clock.
1516///
1617/// Not thread-safe: hand it to one queue and pull frames only from there.
17−final class Recording
18+public final class VoiceRecording
1819{
19− enum Failure: Error, CustomStringConvertible
20+ /// Sample frames in one chunk `nextFrame()` returns: the 20 ms the wire carries.
21+ 1public static let chunkFrameCount = DataChannelMediaStream.frameDuration
22+
23+ /// Why a recording could not be opened, or could not go on being decoded. Every case names
24+ /// the file, because CoreAudio's own errors say only what went wrong.
25+ public enum Failure: Error, CustomStringConvertible
2026 {
21− case cannotOpen(String, Error)
22− case empty(String)
23− case cannotConvert(String, AVAudioFormat)
24− case decodeFailed(String, Error?)
27+ /// `AVAudioFile` refused the URL: missing, unreadable, or not audio it decodes.
28+ 2case cannotOpen(URL, Error)
29+ /// The file opened but holds no audio, or ran dry mid-loop.
30+ case empty(URL)
31+ /// No conversion exists from the file's format to 48 kHz mono Float32.
32+ case cannotConvert(URL, AVAudioFormat)
33+ /// Decoding failed after the file had already opened; nothing recovers it.
34+ case decodeFailed(URL, Error?)
2535
26− var description: String
36+ public var description: String
2737 {
2838 switch self
2939 {
30− case .cannotOpen(let path, let underlying):
31− return "Cannot read \(path) as audio: \(underlying)"
32− case .empty(let path):
33− return "\(path) decodes to no audio"
34− case .cannotConvert(let path, let format):
35− return "Cannot convert \(path) (\(format)) to 48 kHz mono"
36− case .decodeFailed(let path, let underlying):
37− return "Decoding \(path) failed: \(underlying.map { "\($0)" } ?? "converter gave no reason")"
40+ case .cannotOpen(let url, let underlying):
41+ return "Cannot read \(url.path) as audio: \(underlying)"
42+ case .empty(let url):
43+ return "\(url.path) decodes to no audio"
44+ case .cannotConvert(let url, let format):
45+ return "Cannot convert \(url.path) (\(format)) to 48 kHz mono"
46+ case .decodeFailed(let url, let underlying):
47+ return "Decoding \(url.path) failed: \(underlying.map { "\($0)" } ?? "converter gave no reason")"
3848 }
3949 }
4050 }
4151
42− /// The file's own length, before looping.
43− let duration: TimeInterval
52+ /// The file's own length in seconds, before looping.
53+ public let duration: TimeInterval
54+
55+ /// The file this reads, as given.
56+ public let url: URL
4457
45− private let path: String
4658 private let file: AVAudioFile
4759 private let converter: AVAudioConverter
4860 private let input: AVAudioPCMBuffer
4961 private let output: AVAudioPCMBuffer
5062
51− /// - Parameter path: anything AVAudioFile reads - wav, m4a, aiff, caf.
52− /// - Throws: `Failure` naming `path`; CoreAudio's own errors say only what went wrong,
53− /// never with which file.
54− init(path: String) throws
63+ /// Open a file and prepare it for conversion; nothing is decoded until `nextFrame()`.
64+ /// - Parameter url: a file URL to anything `AVAudioFile` reads - wav, m4a, aiff, caf - at any
65+ /// sample rate. A multi-channel file is not downmixed; its first channel is what plays.
66+ /// - Throws: `Failure` naming `url`.
67+ public init(url: URL) throws
5568 {
56− self.path = path
57− do { file = try AVAudioFile(forReading: URL(fileURLWithPath: path)) }
58− catch { throw Failure.cannotOpen(path, error) }
69+ self.url = url
70+ do { file = try AVAudioFile(forReading: url) }
71+ catch { throw Failure.cannotOpen(url, error) }
5972 let source = file.processingFormat
60− guard file.length > 0 else { throw Failure.empty(path) }
73+ guard file.length > 0 else { throw Failure.empty(url) }
6174 duration = Double(file.length) / source.sampleRate
6275
6376 let target = AVAudioFormat(commonFormat: .pcmFormatFloat32,
VoiceRecording.swiftSources/AlloAudio/VoiceRecording.swiftrenamed from Sources/voicedemo/Recording.swift
@@ -67,8 +80,8 @@ final class Recording
6780 guard let converter = AVAudioConverter(from: source, to: target),
6881 let input = AVAudioPCMBuffer(pcmFormat: source, frameCapacity: 4096),
6982 let output = AVAudioPCMBuffer(pcmFormat: target,
70− frameCapacity: AVAudioFrameCount(DataChannelMediaStream.frameDuration))
71− else { throw Failure.cannotConvert(path, source) }
83+ frameCapacity: AVAudioFrameCount(Self.chunkFrameCount))
84+ else { throw Failure.cannotConvert(url, source) }
7285 // Discrete channel layout has no downmix rule; without a map the converter emits
7386 // silence. See docs/voice-implementation.md, One engine.
7487 if source.channelCount != target.channelCount { converter.channelMap = [0] }
VoiceRecording.swiftSources/AlloAudio/VoiceRecording.swiftrenamed from Sources/voicedemo/Recording.swift
@@ -78,11 +91,11 @@ final class Recording
7891 }
7992
8093 /// Decode the next 20 ms, wrapping to the start of the file when it runs out.
81− /// - Returns: `DataChannelMediaStream.frameDuration` mono samples in the recording's own
82− /// buffer, valid until the next call.
94+ /// - Returns: exactly `chunkFrameCount` mono samples in the recording's own buffer, valid until
95+ /// the next call. Pass it straight to `DataChannelMediaStream.send(samples:frameCount:)`.
8396 /// - Throws: `Failure` naming the file. Nothing recovers a decode failure mid-file; the
8497 /// caller should stop pulling.
85− func nextFrame() throws -> UnsafeBufferPointer<Float>
98+ public func nextFrame() throws -> UnsafeBufferPointer<Float>
8699 {
87100 var readFailure: Error?
88101 var conversionError: NSError?
VoiceRecording.swiftSources/AlloAudio/VoiceRecording.swiftrenamed from Sources/voicedemo/Recording.swift
@@ -92,10 +105,9 @@ final class Recording
92105 outStatus.pointee = .haveData
93106 return input
94107 }
95− if let readFailure { throw Failure.decodeFailed(path, readFailure) }
96− guard status == .haveData,
97− output.frameLength == AVAudioFrameCount(DataChannelMediaStream.frameDuration)
98− else { throw Failure.decodeFailed(path, conversionError) }
108+ if let readFailure { throw Failure.decodeFailed(url, readFailure) }
109+ guard status == .haveData, output.frameLength == AVAudioFrameCount(Self.chunkFrameCount)
110+ else { throw Failure.decodeFailed(url, conversionError) }
99111 return UnsafeBufferPointer(start: output.floatChannelData![0], count: Int(output.frameLength))
100112 }
101113
VoiceRecording.swiftSources/AlloAudio/VoiceRecording.swiftrenamed from Sources/voicedemo/Recording.swift
@@ -106,7 +118,7 @@ final class Recording
106118 if file.framePosition >= file.length { file.framePosition = 0 }
107119 if try readSome() { return }
108120 file.framePosition = 0
109− guard try readSome() else { throw Failure.empty(path) }
121+ guard try readSome() else { throw Failure.empty(url) }
110122 }
111123
112124 /// - Returns: false when the file gave nothing because it has ended - which a compressed
VoiceRecordingTests.swiftTests/AlloAudioTests/VoiceRecordingTests.swiftadded
@@ -0,0 +1,123 @@
1+import Testing
2+import Foundation
3+import AVFoundation
4+@testable import AlloAudio
5+import allonet2
6+
7+/// Writes `samples` of a sine to a temp file and opens it as a recording. The length is
8+/// deliberately not a multiple of a frame, so looping has to stitch frames across the seam.
9+private func makeRecording(samples: Int, hz: Double = 440) throws -> VoiceRecording
10+{
11+ let url = FileManager.default.temporaryDirectory.appendingPathComponent("\(UUID().uuidString).caf")
12+ let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 48000, channels: 1, interleaved: false)!
13+ do
14+ {
15+ let file = try AVAudioFile(forWriting: url, settings: format.settings)
16+ let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: AVAudioFrameCount(samples))!
17+ buffer.frameLength = AVAudioFrameCount(samples)
18+ for i in 0..<samples
19+ {
20+ buffer.floatChannelData![0][i] = Float(sin(2 * .pi * hz * Double(i) / 48000)) * 0.5
21+ }
22+ try file.write(from: buffer)
23+ } // the writer has to be gone before the reader opens the file
24+ return try VoiceRecording(url: url)
25+}
26+
27+@Suite struct VoiceRecordingTests
28+{
29+ @Test func loopsPastTheEndWithoutEverGivingAShortFrame() throws
30+ {
31+ let recording = try makeRecording(samples: 1000) // one frame and a bit
32+ #expect(abs(recording.duration - 1000.0 / 48000.0) < 0.001)
33+ for _ in 0..<5
34+ {
35+ #expect(try recording.nextFrame().count == VoiceRecording.chunkFrameCount)
36+ }
37+ }
38+
39+ @Test func aFileThatIsNotThereFailsByName() throws
40+ {
41+ let url = URL(fileURLWithPath: "/tmp/allonet2-no-such-recording.wav")
42+ do
43+ {
44+ _ = try VoiceRecording(url: url)
45+ Issue.record("opening a file that is not there should fail")
46+ }
47+ catch let failure as VoiceRecording.Failure
48+ {
49+ #expect("\(failure)".contains(url.path))
50+ }
51+ }
52+}
53+
⋯
Chapter 3

One clock for several recordings

VoiceRecordingPlayer is the piece that makes six puppets sound like one room. There is no per-recording timer: one 20 ms DispatchSourceTimer on the player's serial queue, and on every tick the player derives how many frames are due from elapsed monotonic time 1 and sends every recording the same number of frames. A late tick therefore catches up, and because every recording owes the same debt they cannot drift apart.

The catch-up is capped 2. Past the cap the debt is forgiven, not skipped: framesSent = due while only sending frames are pulled, so every recording pauses for the same moment and carries on from where it was. That is the right thing for looping speech; a jitter buffer would discard a burst anyway, and skipping ahead in six files by different amounts is how they would fall out of step.

start() sends the first frame before returning 3, so playback starts on the caller's word and the catch-up test is deterministic. The two contracts a caller has to respect are in the doc comment: a codec must be installed before start() (the player cannot install one itself, because AlloOpus depends on allonet2, not the other way round), and onFrameSent runs on the player's queue, so it must not call start() or stop().

The tests drive the player with an injected clock through the internal tick() rather than waiting out real time 4: two recordings stay in step over nine ticks, and a 200 ms stall sends five frames and then forgets the backlog.

VoiceRecordingPlayer.swiftSources/AlloAudio/VoiceRecordingPlayer.swiftadded
@@ -0,0 +1,134 @@
1+//
2+// VoiceRecordingPlayer.swift
3+// AlloAudio
4+//
5+
6+import Foundation
7+import Logging
8+import allonet2
9+
10+/// Sends recordings into voice streams from one 20 ms clock, so that several recordings stay in
11+/// step with each other for as long as the player runs.
12+///
13+/// Each tick works out how many frames are due since `start()` from elapsed monotonic time, and
14+/// sends every recording the frames it still owes, in that same tick. A late tick therefore
15+/// catches up instead of slowing the room down. A tick later than `maximumCatchUp` frames sends
16+/// that many and forgives the rest of the debt: every recording pauses for the same moment and
17+/// carries on from where it was, so no speech is skipped and the recordings stay in step.
18+///
19+/// Thread-safe. The player owns the recordings it is given and pulls frames on its own queue,
20+/// which is the one queue a `VoiceRecording` may be used from.
21+///
22+/// A voice codec must be installed before `start()` (`Opus.install()` from AlloOpus, which a
23+/// `VoiceEngine` also does); without one every frame is refused and the stream logs why.
24+public final class VoiceRecordingPlayer
25+{
26+ /// Frames one tick may send per recording. A longer backlog is forgiven, not sent: a burst
27+ /// is discarded by the receiver's jitter buffer regardless.
28+ 2public static let maximumCatchUp = 5
29+
30+ /// Called on the player's own queue for every frame that reached the wire, with the sequence
31+ /// `DataChannelMediaStream.send(samples:frameCount:)` gave it. For measuring latency; leave
32+ /// it nil otherwise. Set it before `start()`, and do not call `start()` or `stop()` from it:
33+ /// both wait for the queue the callback runs on.
34+ public var onFrameSent: ((DataChannelMediaStream, UInt32, Date) -> Void)?
35+
36+ private static let frameInterval = Double(VoiceRecording.chunkFrameCount) / DataChannelMediaStream.sampleRate
37+
38+ private let queue = DispatchQueue(label: "allonet2.voicerecordingplayer")
39+ private let monotonicNow: () -> Double
40+ private var voices: [(recording: VoiceRecording, stream: DataChannelMediaStream)] = []
41+ private var timer: DispatchSourceTimer?
42+ private var startedAt: Double?
43+ private var framesSent = 0
44+ private let logger = Logger(labelSuffix: "voice.recordings")
45+
46+ /// - Parameter monotonicNow: seconds on a clock that only moves forward. Override it to drive
47+ /// the player from a test's own clock.
48+ public init(monotonicNow: @escaping () -> Double = { Double(DispatchTime.now().uptimeNanoseconds) / 1e9 })
49+ {
50+ self.monotonicNow = monotonicNow
51+ }
52+
53+ deinit { timer?.cancel() }
54+
55+ /// Play `recording` into `stream` until the player stops or the recording fails to decode.
56+ /// The player takes the recording over: do not pull frames from it anywhere else.
57+ ///
58+ /// Added before `start()`, a recording begins at its first frame together with every other.
59+ /// Added while the player runs, it joins on the next tick and is in step from there.
60+ public func add(_ recording: VoiceRecording, to stream: DataChannelMediaStream)
61+ {
62+ queue.async { self.voices.append((recording, stream)) }
63+ }
64+
65+ /// Start the clock and send frames until `stop()`. The first frame of every recording goes
66+ /// out before this returns, so playback begins on the caller's word rather than up to 20 ms
67+ /// later. Starting a running player does nothing; starting a stopped one restarts the clock,
68+ /// and each recording carries on from the frame it had reached.
69+ public func start()
70+ {
71+ queue.sync {
72+ guard timer == nil else { return }
73+ 3startedAt = monotonicNow()
74+ framesSent = 0
75+ pump()
76+ let timer = DispatchSource.makeTimerSource(queue: queue)
77+ timer.schedule(deadline: .now() + .milliseconds(20), repeating: .milliseconds(20), leeway: .milliseconds(2))
78+ timer.setEventHandler { [weak self] in self?.pump() }
79+ self.timer = timer
80+ timer.resume()
81+ }
82+ }
83+
84+ /// Stop sending. Idempotent, and safe from any thread; a frame already on its way out of the
85+ /// player's queue finishes first.
86+ public func stop()
87+ {
88+ queue.sync {
89+ timer?.cancel()
90+ timer = nil
91+ startedAt = nil
92+ }
93+ }
94+
95+ /// One pass of the clock, from the caller's thread. Internal so a test can pump the player
96+ /// on its own clock rather than waiting out real time.
97+ 4func tick() { queue.sync { pump() } }
98+
99+ private func pump()
100+ {
101+ guard let startedAt else { return }
102+ // +1 so the first tick, at zero elapsed, sends the frame that plays now.
103+ 1let due = Int((monotonicNow() - startedAt) / Self.frameInterval) + 1
104+ guard due > framesSent else { return }
105+ let sending = min(due - framesSent, Self.maximumCatchUp)
106+ framesSent = due
107+ for _ in 0..<sending
108+ {
109+ // A recording that stops decoding is dropped, once and loudly, rather than failing
110+ // again on every later tick.
111+ voices.removeAll { !send(from: $0.recording, to: $0.stream) }
112+ }
113+ }
114+
115+ /// - Returns: false when the recording failed and must be dropped.
116+ private func send(from recording: VoiceRecording, to stream: DataChannelMediaStream) -> Bool
117+ {
118+ do
119+ {
120+ let frame = try recording.nextFrame()
121+ let at = Date()
122+ if let sequence = stream.send(samples: frame.baseAddress!, frameCount: frame.count)
123+ {
124+ onFrameSent?(stream, sequence, at)
125+ }
126+ return true
127+ }
128+ catch
129+ {
130+ logger.error("Stopped playing \(recording.url.path) into \(stream.mediaId): \(error)")
131+ return false
132+ }
133+ }
134+}
VoiceRecordingTests.swiftTests/AlloAudioTests/VoiceRecordingTests.swiftadded
@@ -0,0 +1,123 @@
⋯
54+@Suite struct VoiceRecordingPlayerTests
55+{
56+ /// The player's queue writes these; the test thread reads them.
57+ private final class FrameCount: @unchecked Sendable
58+ {
59+ private let lock = NSLock()
60+ private var value = 0
61+ func increment() { lock.lock(); value += 1; lock.unlock() }
62+ var count: Int { lock.lock(); defer { lock.unlock() }; return value }
63+ }
64+
65+ /// A clock the test moves, read from the player's queue.
66+ private final class TestClock: @unchecked Sendable
67+ {
68+ private let lock = NSLock()
69+ private var seconds = 1000.0
70+ func now() -> Double { lock.lock(); defer { lock.unlock() }; return seconds }
71+ func advance(by delta: Double) { lock.lock(); seconds += delta; lock.unlock() }
72+ }
73+
74+ private func makeStream(_ counted: FrameCount) -> DataChannelMediaStream
75+ {
76+ DataChannelMediaStream(mediaId: UUID().uuidString, direction: .sendonly) { _ in
77+ counted.increment()
78+ return true
79+ }
80+ }
81+
82+ /// Frames are due by elapsed time, not by tick count, so both recordings owe the same number
83+ /// of frames on every tick however the ticks fall.
84+ @Test func oneClockKeepsTwoRecordingsInStep() throws
85+ {
86+ VoiceCodecs.makeEncoder = { RawPCMVoiceCodec() }
87+ let clock = TestClock()
88+ let player = VoiceRecordingPlayer(monotonicNow: clock.now)
89+ let first = FrameCount(), second = FrameCount()
90+ player.add(try makeRecording(samples: 1000), to: makeStream(first))
91+ player.add(try makeRecording(samples: 7000), to: makeStream(second))
92+
93+ player.start() // sends frame one before it returns
94+ defer { player.stop() }
95+ clock.advance(by: 0.01) // half a frame, so no tick lands on a boundary and rounds the wrong way
96+ for _ in 0..<9 { clock.advance(by: 0.02); player.tick() }
97+
98+ #expect(first.count == 10)
99+ #expect(second.count == 10)
100+ }
101+
102+ /// A tick that arrives very late sends what it can and drops the rest, rather than bursting
103+ /// a backlog the receiver's jitter buffer would discard anyway.
104+ @Test func aLateTickCatchesUpOnlySoFar() throws
105+ {
106+ VoiceCodecs.makeEncoder = { RawPCMVoiceCodec() }
107+ let clock = TestClock()
108+ let player = VoiceRecordingPlayer(monotonicNow: clock.now)
109+ let counted = FrameCount()
110+ player.add(try makeRecording(samples: 1000), to: makeStream(counted))
111+
112+ player.start()
113+ defer { player.stop() }
114+ clock.advance(by: 0.01) // half a frame, so no tick lands on a boundary and rounds the wrong way
115+ clock.advance(by: 0.2) // ten frames' worth of stall
116+ player.tick()
117+ #expect(counted.count == 1 + VoiceRecordingPlayer.maximumCatchUp)
118+
119+ clock.advance(by: 0.02)
120+ player.tick()
121+ #expect(counted.count == 2 + VoiceRecordingPlayer.maximumCatchUp) // the backlog is gone, not queued
122+ }
123+}
voice.mddocs/voice.md
@@ -103,6 +103,32 @@ libwebrtc never retransmitted voice either; this keeps the same two tools, both
103103 decoder is told so and extrapolates 20 ms from what it last decoded, instead of playing
104104 silence.
105105
106+## Sending a file instead of a microphone
107+
108+A sender can speak a file instead of a capture device. `AlloAudio` has the two pieces.
109+
110+`VoiceRecording(url:)` opens anything `AVAudioFile` reads and converts it on demand into the
111+960-sample 48 kHz mono Float32 frames `DataChannelMediaStream.send(samples:frameCount:)` takes.
112+It loops inside the same conversion pass, so the seam is sample-continuous and no frame is ever
113+short. It is not thread-safe: one queue pulls from it.
114+
115+`VoiceRecordingPlayer` drives one or more of them from a single 20 ms clock. Each tick works out
116+how many frames are due since `start()` from elapsed monotonic time and sends every recording
117+the frames it owes in that same tick, which is what keeps several recordings in step with each
118+other. A tick later than `maximumCatchUp` frames sends that many and forgives the rest, since a
119+burst is discarded by the receiver's jitter buffer anyway: every recording pauses for the same
120+moment and carries on from where it was, so no speech is skipped. `start()` sends the first frame before it returns; `stop()` is
121+idempotent, and starting again restarts the clock while each recording carries on where it was.
122+
123+```swift
124+Opus.install() // once per process; without a codec every frame is refused
125+let player = VoiceRecordingPlayer()
126+player.add(try VoiceRecording(url: url), to: stream)
127+player.start()
128+```
129+
130+`voicedemo` uses both for `VOICEDEMO_WAV`.
131+
106132## Counters
107133
108134`VoiceCounters` are values tests assert on, not log lines. On a receiver every `received`
Chapter 4

voicedemo uses the library

The executable keeps its two sources, file and tone, but the file path now goes through VoiceRecordingPlayer 1. VOICEDEMO_LATENCY_LOG needs the sequence number every sent frame got, which is what onFrameSent exists for 2. With the generic startGenerating gone, the tone generator keeps its own timer inline; the "return nil to stop" path it had is dead, because a recording that fails is now dropped inside the player.

VoiceDemo.swiftSources/voicedemo/VoiceDemo.swift
@@ -37,10 +37,10 @@ struct VoiceDemo
3737
3838 // Decode before connecting: a bad path should fail as itself, not as a client that joins
3939 // the place and then says nothing.
40− var recording: Recording?
40+ var recording: VoiceRecording?
4141 if let path = ProcessInfo.processInfo.environment["VOICEDEMO_WAV"]
4242 {
43− do { recording = try Recording(path: path) }
43+ do { recording = try VoiceRecording(url: URL(fileURLWithPath: path)) }
4444 catch
4545 {
4646 FileHandle.standardError.write(Data("VOICEDEMO_WAV: \(error)\n".utf8))
VoiceDemo.swiftSources/voicedemo/VoiceDemo.swift
@@ -85,9 +85,10 @@ final class VoiceDemoClient: AlloClient
8585 private let engine = VoiceEngine(voiceProcessing: ProcessInfo.processInfo.environment["VOICEDEMO_NO_VPIO"] == nil)
8686 private var outgoing: DataChannelMediaStream?
8787 private var incoming: [MediaStreamId: DataChannelMediaStream] = [:]
88− private var generator: DispatchSourceTimer?
88+ private var tone: DispatchSourceTimer?
89+ private let recordings = VoiceRecordingPlayer()
8990 var latency: LatencyLog?
90− var recording: Recording?
91+ var recording: VoiceRecording?
9192 private var outgoingMediaId: MediaStreamId?
9293 private var lastPolled: [MediaStreamId: UInt32] = [:]
9394
VoiceDemo.swiftSources/voicedemo/VoiceDemo.swift
@@ -112,14 +113,13 @@ final class VoiceDemoClient: AlloClient
112113 let source: String
113114 if let recording
114115 {
115− startGenerating(into: stream) {
116− do { return Array(try recording.nextFrame()) }
117− catch
118− {
119− FileHandle.standardError.write(Data("Stopped sending: \(error)\n".utf8))
120− return nil
121− }
116+ // LatencyLog only appends, so the player's own queue may note captures directly.
117+ if let latency, let mediaId = outgoingMediaId
118+ {
119+ 2recordings.onFrameSent = { _, sequence, at in latency.note(capture: mediaId, sequence: sequence, at: at) }
122120 }
121+ 1recordings.add(recording, to: stream)
122+ recordings.start()
123123 source = String(format: "recording, %.0f s looping", recording.duration)
124124 }
125125 else if let hz = ProcessInfo.processInfo.environment["VOICEDEMO_TONE"].flatMap(Double.init)
VoiceDemo.swiftSources/voicedemo/VoiceDemo.swift
@@ -147,33 +147,18 @@ final class VoiceDemoClient: AlloClient
147147 let frameCount = DataChannelMediaStream.frameDuration
148148 var phase = 0.0
149149 let step = 2 * Double.pi * hz / DataChannelMediaStream.sampleRate
150− startGenerating(into: stream) {
151− var samples = [Float](repeating: 0, count: frameCount)
152− for i in 0..<frameCount { samples[i] = Float(sin(phase)) * 0.25; phase += step }
153− return samples
154− }
155− }
156−
157− /// Send one frame every 20 ms from `nextFrame`, off a timer rather than an audio clock.
158− /// Returning nil stops the timer for good; the reason belongs on stderr before it does.
159− private func startGenerating(into stream: DataChannelMediaStream, _ nextFrame: @escaping () -> [Float]?)
160− {
161− let timer = DispatchSource.makeTimerSource(queue: DispatchQueue(label: "voicedemo.generator"))
150+ let timer = DispatchSource.makeTimerSource(queue: DispatchQueue(label: "voicedemo.tone"))
162151 timer.schedule(deadline: .now(), repeating: .milliseconds(20), leeway: .milliseconds(2))
163152 let latency = self.latency
164153 let mediaId = outgoingMediaId
165− timer.setEventHandler { [weak self] in
166− guard let samples = nextFrame()
167− else
168− {
169− Task { @MainActor in self?.generator?.cancel(); self?.generator = nil }
170− return
171− }
154+ timer.setEventHandler {
155+ var samples = [Float](repeating: 0, count: frameCount)
156+ for i in 0..<frameCount { samples[i] = Float(sin(phase)) * 0.25; phase += step }
172157 let at = Date()
173158 let sequence = samples.withUnsafeBufferPointer { stream.send(samples: $0.baseAddress!, frameCount: samples.count) }
174159 if let sequence, let mediaId { latency?.note(capture: mediaId, sequence: sequence, at: at) }
175160 }
176− generator = timer
161+ tone = timer
177162 timer.resume()
178163 }
179164
Chapter 5

A stopped place stops beating

PlaceServer.stop() now stops the heartbeat 1. Three things had to be true for that to be enough. The timer re-arms itself after every sync, so a sync in flight when stop() ran would schedule the next beat; setupTimer now refuses once stopped 2. A disconnecting client's cleanup awaits the next beat before it removes the client's entities, so stopping the timer would have left those tasks, and through them the clients and the server, waiting forever; stop() finishes the sync stream 3 and awaitNextSync() returns at once on a stopped timer. And the timer's action captured the server strongly while the server owned the timer; it is weak now 4.

HeartbeatTimerTests gained one test per claim: nothing fires after stop(), and a waiter in awaitNextSync() is released by it.

PlaceServer.swiftSources/allonet2/PlaceServer/PlaceServer.swift
@@ -40,8 +40,8 @@ public class PlaceServer : AlloSessionDelegate
4040 // The scenegraph state of the Place
4141 let place: PlaceState
4242 lazy var heartbeat: HeartbeatTimer = {
43− return HeartbeatTimer {
44− self.applyAndBroadcastState()
43+ return HeartbeatTimer { 4[weak self] in
44+ await self?.applyAndBroadcastState()
4545 }
4646 }()
4747 internal var outstandingPlaceChanges: [PlaceChange] = []
PlaceServer.swiftSources/allonet2/PlaceServer/PlaceServer.swift
@@ -112,6 +120,7 @@ public class PlaceServer : AlloSessionDelegate
112120 client.session.disconnect()
113121 }
114122 sfu.stop()
123+ 1await heartbeat.stop()
115124 }
116125
117126 public func session(didConnect sess: AlloSession)
HeartbeatTimer.swiftSources/allonet2/PlaceServer/HeartbeatTimer.swift
@@ -17,6 +17,7 @@ actor HeartbeatTimer
1717 private let timerQueue = DispatchQueue(label: "HeartbeatTimerQueue")
1818 private var timer: DispatchSourceTimer?
1919 private var pendingChanges = false
20+ private var stopped = false
2021
2122 // This stream must not buffer events; otherwise any awaitNextSync() will trigger immediately based on an outdated heartbeat,
2223 // not the latest one it's actually waiting for.
HeartbeatTimer.swiftSources/allonet2/PlaceServer/HeartbeatTimer.swift
@@ -47,18 +48,24 @@ actor HeartbeatTimer
4748
4849 public func awaitNextSync() async
4950 {
51+ guard !stopped else { return }
5052 for await _ in syncStream { break }
5153 }
5254
55+ /// Fires no more beats, including the one a sync in flight would have scheduled, and
56+ /// releases everyone in `awaitNextSync()`.
5357 public func stop()
5458 {
59+ stopped = true
5560 timer?.cancel()
5661 timer = nil
62+ 3syncContinuation?.finish() // nobody is left waiting for a beat that will not come
5763 }
5864
5965 2private func setupTimer(delay: Int)
6066 {
6167 timer?.cancel()
68+ guard !stopped else { return }
6269
6370 let newTimer = DispatchSource.makeTimerSource(queue: timerQueue)
6471 newTimer.setEventHandler { [weak self] in
HeartbeatTimerTests.swiftTests/allonet2Tests/HeartbeatTimerTests.swift
@@ -27,12 +27,45 @@ struct HeartbeatTimerTests
2727
2828 await timer.stop()
2929 }
30+
31+ /// The timer re-arms itself after every sync, so a stop that only cancelled the pending
32+ /// timer let a sync in flight schedule the next beat, and the owner never went away.
33+ @Test func firesNothingAfterStop() async throws
34+ {
35+ let syncs = Syncs()
36+ let timer = HeartbeatTimer(coalesceDelay: 5_000_000, keepaliveDelay: 20_000_000)
37+ {
38+ await syncs.record()
39+ }
40+ try await syncs.waitForCount(1, within: .seconds(2))
41+
42+ await timer.stop()
43+ let countAtStop = await syncs.current
44+ try await Task.sleep(for: .milliseconds(200))
45+ #expect(await syncs.current <= countAtStop + 1) // one sync may already have been in flight
46+ await timer.markChanged()
47+ try await Task.sleep(for: .milliseconds(100))
48+ #expect(await syncs.current <= countAtStop + 1)
49+ }
50+
51+ /// A disconnecting client's cleanup waits for the next beat; stopping the place must not
52+ /// leave it waiting forever.
53+ @Test func stopReleasesThoseWaitingForTheNextSync() async throws
54+ {
55+ let timer = HeartbeatTimer(coalesceDelay: 5_000_000, keepaliveDelay: 10_000_000_000) {}
56+ let waiter = Task { await timer.awaitNextSync() }
57+ try await Task.sleep(for: .milliseconds(50))
58+ await timer.stop()
59+ await waiter.value
60+ await timer.awaitNextSync() // and nobody starts waiting afterwards
61+ }
3062}
3163
3264/// Counts syncAction invocations, and can re-enter markChanged from inside one.
3365private actor Syncs
3466{
3567 private var count = 0
68+ var current: Int { count }
3669 private var markNext: HeartbeatTimer?
3770
3871 func record() async

What this change leaves alone

PlaceServerHTTP still binds the wildcard address: FlyingFox has no bind-address option, and TransportConnectionOptions.bindAddress only constrains ICE. An embedded place is reachable from the LAN while it runs; the demo office accepts that and documents it on its side. AlloUserClient.MicrophoneTrack keeps its single fixed voice-mic stream; the player is for app clients.

Open questions

Open question

The full parallel swift test failed on the development machine in CondemnedClientTests.fatalErrorResponseSaysSoAndTheHangupWaitsForIt and two NoOpWriteTests cases. They pass in isolation, one of them still fails with this PR's suites skipped, and neither suite calls PlaceServer.start(). They look like wall-clock assertions that a shared test process cannot guarantee, but that was never measured against an untouched main on an idle machine. CI on this PR only runs Linux builds, so the macOS suite has no clean data point.