Node.js FFmpeg PCM Audio Stream to WebRTC Audio Stream

1. Introduction and Use Cases

Real-time audio streaming—whether for live broadcasting, telephony, remote collaboration, or smart devices—frequently requires a robust pipeline capable of capturing, processing, encoding, and delivering audio data with minimal latency and maximum reliability. PCM (Pulse-Code Modulation) audio is often the intermediate format for raw audio, but browsers and WebRTC require audio in the Opus format.

Key Stack:

• Source: Microphone, file, or virtual audio.
• FFmpeg: Converts/captures audio, typically outputs PCM or Opus.
• Node.js: Handles streams, encoding, signaling, and pipeline management.
• WebRTC: Transports live audio to browsers and other peers.

Common use cases involve streaming from FFmpeg through Node.js as PCM, encoding to Opus, and sending through WebRTC as a secure, low-latency stream. This article will provide actual implementation steps for that workflow.

References: FFmpeg Docs, WebRTC.org, Live Audio Node Example

---

2. Fundamentals: PCM Audio, FFmpeg, and Node.js Streams

PCM Audio

PCM is the purest form of digital audio—uncompressed and direct—making it suitable for further transcoding and manipulation. For streaming, a typical PCM configuration is 16-bit, 48kHz, mono or stereo.

FFmpeg for PCM Output

Test sine wave into PCM:

Sh
1ffmpeg -f lavfi -i sine=frequency=440 -f s16le -ar 48000 -ac 1 pipe:1
2

• -f s16le: Signed 16-bit, little endian PCM
• -ar 48000: 48kHz sample rate
• -ac 1: Mono
• pipe:1: Output to stdout

Node.js Streams

Node.js leverages streams for efficient, event-driven data handling:

Js
1const { spawn } = require('child_process');
2const ffmpeg = spawn('ffmpeg', [ /* FFmpeg args above */ ]);
3ffmpeg.stdout.on('data', (chunk) => {
4  // Process or forward PCM chunks
5});
6

Custom readable stream:

Js
1const { Readable } = require('stream');
2const audioStream = new Readable({ read() {} });
3ffmpeg.stdout.on('data', chunk => audioStream.push(chunk));
4

References: Node Streams API

---

3. Designing the Pipeline: From FFmpeg to Node.js Streams

Pipeline Flow:

1. Audio Ingestion: Any source/file → converted to PCM by FFmpeg
2. PCM Processing: Buffered and chunked in Node.js in precise frame sizes
3. Opus Encoding: PCM encoded to Opus with @discordjs/opus (or node-opus)
4. Real-Time Delivery: Forwarded to WebRTC media track

Buffering and Chunking Example:

Js
1const OpusEncoder = require('@discordjs/opus').OpusEncoder;
2const opus = new OpusEncoder(48000, 1);
3let buffer = Buffer.alloc(0);
4
5ffmpeg.stdout.on('data', (chunk) => {
6  buffer = Buffer.concat([buffer, chunk]);
7  while (buffer.length >= 1920) { // 960 samples, 2 bytes/sample
8    const pcmFrame = buffer.slice(0, 1920);
9    buffer = buffer.slice(1920);
10    const opusPacket = opus.encode(pcmFrame, 960);
11    // Forward opusPacket to WebRTC
12  }
13});
14

References: @discordjs/opus docs

---

4. WebRTC Audio Integration in Node.js

Setting up wrtc PeerConnection

Js
1const { RTCPeerConnection, RTCAudioSource } = require('wrtc');
2const pc = new RTCPeerConnection({ iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] });
3const audioSource = new RTCAudioSource();
4const audioTrack = audioSource.createTrack();
5pc.addTrack(audioTrack);
6

SDP and ICE Signaling

Js
1pc.onicecandidate = ({ candidate }) => {
2  if(candidate) signaling.send({ type: 'ice', candidate });
3};
4// After creating an offer
5const offer = await pc.createOffer();
6await pc.setLocalDescription(offer);
7signaling.send({ type: 'offer', sdp: offer.sdp });
8

Feeding Frames to WebRTC

Js
1audioSource.onData({ samples: pcmInt16, sampleRate: 48000 });
2

Signaling (WebSocket/Socket.io)

• Send/receive offers, answers, ICE as JSON payloads
• Handle reconnection, timeouts, and signaling errors

References: WebRTC.org, wrtc npm

---

5. Complete Application Example: Real-Time Streaming from Node.js to Browser

1. Start a WebSocket-based Signaling Server:

Js
1// Short snippet, see full version above
2const WebSocket = require('ws');
3const wss = new WebSocket.Server({ port: 8080 });
4// Basic routing/broadcast logic
5

2. Node.js WebRTC/Audio Server:

• Spawns FFmpeg for PCM
• Chunks/processes PCM, encodes to Opus
• Sets up wrtc PeerConnection, signaling, and feeds audio

3. Browser WebRTC Listener:

Html
1<audio id="remoteAudio" autoplay></audio>
2<script>
3  // Establish websocket, handle peer connection, and assign remote stream to audio.srcObject
4</script>
5

4. Troubleshooting:

• Log ICE/SDP exchange and connection states
• Validate PCM/Opus frame sizes and orders
• Check browser and server logs for negotiation or media errors

References: Simple WebRTC Sample

---

6. Best Practices and Troubleshooting

Quality & Latency:

• Always buffer PCM for exact Opus frame-size packets
• Use 48kHz, mono for Opus unless stereo is justified
• Monitor both ends for underruns and overruns

Security:

• Use secure signaling (wss:// or HTTPS)
• Enforce authentication and limit peer session rates
• Never expose system hardware or raw FFmpeg endpoints without isolation

Troubleshooting Patterns:

• Log every step: FFmpeg stderr, Node process, SDP/ICE signals
• Use browser developer tools and pc.getStats() for connection metrics
• Automate cleanup (close sockets, PeerConnections, kill processes on disconnect)

Typical Fixes:

Problem	Diagnosis	Fix
No Audio	Inspect FFmpeg & stream	Fix input, buffer, codec
Choppy Output	Buffer or network issue	Tune frame/packet size; debug logs
Conn. Drops	ICE signaling	Use public/redundant STUN/TURN

References: WebRTC Troubleshooting, FFmpeg Security

---

7. References and Further Reading

• WebRTC.org
• FFmpeg Official Docs
• wrtc package
• @discordjs/opus
• WebRTC for the Curious
• Google WebRTC Codelab
• Simple WebRTC Sample