Duplex Timeline

Installation

pnpm dlx shadcn@latest add @pipecat/duplex-timeline

Usage

import { DuplexTimeline } from "@/components/pipecat/duplex-timeline";

// Connected: both participants' audio and RTVI turn events
<DuplexTimeline />;

The agent lane sits over the user lane, both scrolling on one clock with now at the right edge. Bars take the lane's color for speech, backchannelColor for backchannels ("mm-hmm" from either side), and interruptionColor where the user talked over the bot. A dashed rule marks the end of the user's turn and the start of the bot's reply, with the latency between them; replies slower than slowReplyMs draw it in slowReplyColor. Ticks under the lanes count elapsed time from when the timeline started, labelled every five seconds, and scroll with the waveforms. A color key sits above the lanes.

DuplexTimeline scrolls while the transport is ready and holds its last frame after the session ends. It builds its markers from RTVI turn events, and backchannels from bot-output (text_type: "backchannel") and user-input (input_type: "backchannel"), which bots send from RTVI 2.2.0.

Your own data

DuplexTimelineView takes two tracks and a marker list. Inside a PipecatClientProvider, useDuplexTimelineMarkers builds the markers from RTVI events:

import {
  DuplexTimelineView,
  useDuplexTimelineMarkers,
} from "@/components/pipecat/duplex-timeline";

const { markers, agentSpeaking, userSpeaking } = useDuplexTimelineMarkers();

<DuplexTimelineView
  agentTrack={botTrack}
  userTrack={micTrack}
  markers={markers}
  running={live}
  agentSpeaking={agentSpeaking}
  userSpeaking={userSpeaking}
/>;

Without Pipecat, fold your own speech events through reduceDuplexTimeline, stamped with performance.now():

import { useReducer } from "react";
import {
  DUPLEX_TIMELINE_INITIAL_STATE,
  reduceDuplexTimeline,
} from "@/components/pipecat/duplex-timeline-markers";

const [timeline, dispatch] = useReducer(
  reduceDuplexTimeline,
  DUPLEX_TIMELINE_INITIAL_STATE,
);

dispatch({ type: "user-stopped-speaking", now: performance.now() });
dispatch({ type: "bot-started-speaking", now: performance.now() });
dispatch({ type: "user-backchannel", text: "Mm-hmm.", now: performance.now() });

<DuplexTimelineView
  markers={timeline.markers}
  agentSpeaking={timeline.botSpeaking}
  userSpeaking={timeline.userSpeaking}
  {...rest}
/>;

The events are connected, disconnected, user-started-speaking, user-stopped-speaking, bot-started-speaking, bot-stopped-speaking, and bot-backchannel / user-backchannel with their text. User turn events mark turns only, so send a user backchannel on its own. A user turn announced just after the bot stops becomes a possible-interruption marker; the timeline draws it as an interruption only when the audio overlapped.

Markers and waveform history belong to the components that hold them. If the timeline sits in a tab or drawer, mount useDuplexTimelineMarkers above it and pass the markers down so they survive closing it; the waveform starts again when the view remounts.

Showing less

<DuplexTimeline noInterruptions noLatency noTimeLabels />

noBackchannels and noInterruptions draw those stretches as plain speech and drop their labels; backchannels still never count as handoffs. noHandoffs and noLatency remove the dashed rules and latency brackets. noLaneLabels, noSegmentLabels, and noTimeLabels hide the lane names, the backchannel and interruption labels, and the elapsed-time labels. The color key above the lanes follows these options; noLegend hides it. To place it elsewhere, set noLegend and spread the same options into DuplexTimelineLegend.

Styling labels

Labels are DOM spans over the canvas, so each kind takes a class name:

<DuplexTimeline
  laneLabelClassName="uppercase data-[state=speaking]:text-primary"
  backchannelLabelClassName="italic"
  latencyLabelClassName="font-semibold data-slow:text-destructive"
  timeLabelClassName="opacity-30"
/>

Lane labels carry data-lane and data-state="speaking" | "silent"; latency labels for slow replies carry data-slow. Backchannel, interruption, and slow latency labels default to backchannelColor, interruptionColor, and slowReplyColor; a text color class replaces them.

Key props

Colors accept any CSS color, currentColor, or a --css-var name, and follow theme changes on <html>. Handoff rules, latency, and time labels use the element's text color. Under prefers-reduced-motion the timeline redraws twice a second instead of every frame.