| name | sip-media-negotiation |
| description | Use when handling SDP offer/answer, codec negotiation, media capabilities, and RTP session setup in SIP applications. |
| allowed-tools | ["Bash","Read"] |
SIP Media Negotiation
Master Session Description Protocol (SDP) offer/answer model, codec negotiation, and media session establishment for building robust VoIP applications with optimal media handling.
Understanding SDP and Media Negotiation
SDP (RFC 4566) is used in SIP to describe multimedia sessions. The SDP offer/answer model (RFC 3264) enables endpoints to negotiate media capabilities, codecs, and transport parameters.
SDP Structure and Syntax
Basic SDP Message
v=0
o=alice 2890844526 2890844527 IN IP4 atlanta.example.com
s=VoIP Call
c=IN IP4 192.0.2.1
t=0 0
m=audio 49170 RTP/AVP 0 8 97
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 iLBC/8000
a=ptime:20
a=maxptime:150
a=sendrecv
SDP Line Meanings
v= Protocol version (always 0)
o= Origin (username, session-id, session-version, network-type, address-type, address)
s= Session name
c= Connection information
t= Timing (start-time stop-time, 0 0 means permanent)
m= Media description (media, port, protocol, formats)
a= Attribute (codec mappings, parameters, direction)
SDP Parser Implementation
Complete SDP Parser
interface SdpOrigin {
username: string;
sessionId: string;
sessionVersion: string;
netType: string;
addrType: string;
address: string;
}
interface SdpConnection {
netType: string;
addrType: string;
address: string;
ttl?: number;
addressCount?: number;
}
interface SdpMedia {
media: string;
port: number;
portCount?: number;
protocol: string;
formats: string[];
attributes: Map<string, string[]>;
connection?: SdpConnection;
bandwidth?: Map<string, number>;
}
interface SdpSession {
version: number;
origin: SdpOrigin;
sessionName: string;
sessionInfo?: string;
uri?: string;
email?: string;
phone?: string;
connection?: SdpConnection;
bandwidth?: Map<string, number>;
timing: { start: number; stop: number }[];
attributes: Map<string, string[]>;
media: SdpMedia[];
}
class SdpParser {
static parse(sdp: string): SdpSession {
const lines = sdp.trim().split(/\r?\n/);
const session: Partial<SdpSession> = {
attributes: new Map(),
timing: [],
media: []
};
let currentMedia: SdpMedia | null = null;
for (const line of lines) {
const type = line.charAt(0);
const value = line.substring(2);
switch (type) {
case 'v':
session.version = parseInt(value);
break;
case 'o':
session.origin = this.parseOrigin(value);
break;
case 's':
session.sessionName = value;
break;
case 'i':
if (currentMedia) {
currentMedia.attributes.set('title', [value]);
} else {
session.sessionInfo = value;
}
break;
case 'u':
session.uri = value;
break;
case 'e':
session.email = value;
break;
case 'p':
session.phone = value;
break;
case 'c':
const connection = this.parseConnection(value);
if (currentMedia) {
currentMedia.connection = connection;
} else {
session.connection = connection;
}
break;
case 'b':
const [bwType, bandwidth] = value.split(':');
const bwValue = parseInt(bandwidth);
if (currentMedia) {
if (!currentMedia.bandwidth) {
currentMedia.bandwidth = new Map();
}
currentMedia.bandwidth.set(bwType, bwValue);
} else {
if (!session.bandwidth) {
session.bandwidth = new Map();
}
session.bandwidth.set(bwType, bwValue);
}
break;
case 't':
const [start, stop] = value.split(' ').map(Number);
session.timing!.push({ start, stop });
break;
case 'm':
if (currentMedia) {
session.media!.push(currentMedia);
}
currentMedia = this.parseMedia(value);
break;
case 'a':
const [attrName, attrValue] = this.parseAttribute(value);
if (currentMedia) {
if (!currentMedia.attributes.has(attrName)) {
currentMedia.attributes.set(attrName, []);
}
currentMedia.attributes.get(attrName)!.push(attrValue || '');
} else {
if (!session.attributes!.has(attrName)) {
session.attributes!.set(attrName, []);
}
session.attributes!.get(attrName)!.push(attrValue || '');
}
break;
}
}
if (currentMedia) {
session.media!.push(currentMedia);
}
return session as SdpSession;
}
private static parseOrigin(value: string): SdpOrigin {
const parts = value.split(' ');
return {
username: parts[0],
sessionId: parts[1],
sessionVersion: parts[2],
netType: parts[3],
addrType: parts[4],
address: parts[5]
};
}
private static parseConnection(value: string): SdpConnection {
const parts = value.split(' ');
const addressParts = parts[2].split('/');
return {
netType: parts[0],
addrType: parts[1],
address: addressParts[0],
ttl: addressParts[1] ? parseInt(addressParts[1]) : undefined,
addressCount: addressParts[2] ? parseInt(addressParts[2]) : undefined
};
}
private static parseMedia(value: string): SdpMedia {
const parts = value.split(' ');
const portParts = parts[1].split('/');
return {
media: parts[0],
port: parseInt(portParts[0]),
portCount: portParts[1] ? parseInt(portParts[1]) : undefined,
protocol: parts[2],
formats: parts.slice(3),
attributes: new Map()
};
}
private static parseAttribute(value: string): [string, string] {
const colonIndex = value.indexOf(':');
if (colonIndex === -1) {
return [value, ''];
}
return [value.substring(0, colonIndex), value.substring(colonIndex + 1)];
}
static stringify(session: SdpSession): string {
let sdp = '';
sdp += `v=${session.version}\r\n`;
const o = session.origin;
sdp += `o=${o.username} ${o.sessionId} ${o.sessionVersion} ${o.netType} ${o.addrType} ${o.address}\r\n`;
sdp += `s=${session.sessionName}\r\n`;
if (session.sessionInfo) {
sdp += `i=${session.sessionInfo}\r\n`;
}
if (session.uri) {
sdp += `u=${session.uri}\r\n`;
}
if (session.email) {
sdp += `e=${session.email}\r\n`;
}
if (session.phone) {
sdp += `p=${session.phone}\r\n`;
}
if (session.connection) {
sdp += this.stringifyConnection(session.connection);
}
if (session.bandwidth) {
for (const [type, value] of session.bandwidth) {
sdp += `b=${type}:${value}\r\n`;
}
}
for (const timing of session.timing) {
sdp += `t=${timing.start} ${timing.stop}\r\n`;
}
for (const [name, values] of session.attributes) {
for (const value of values) {
sdp += value ? `a=${name}:${value}\r\n` : `a=${name}\r\n`;
}
}
for (const media of session.media) {
sdp += this.stringifyMedia(media);
}
return sdp;
}
private static stringifyConnection(conn: SdpConnection): string {
let line = `c=${conn.netType} ${conn.addrType} ${conn.address}`;
if (conn.ttl !== undefined) {
line += `/${conn.ttl}`;
if (conn.addressCount !== undefined) {
line += `/${conn.addressCount}`;
}
}
return line + '\r\n';
}
private static stringifyMedia(media: SdpMedia): string {
let sdp = `m=${media.media} ${media.port}`;
if (media.portCount) {
sdp += `/${media.portCount}`;
}
sdp += ` ${media.protocol} ${media.formats.join(' ')}\r\n`;
if (media.connection) {
sdp += this.stringifyConnection(media.connection);
}
if (media.bandwidth) {
for (const [type, value] of media.bandwidth) {
sdp += `b=${type}:${value}\r\n`;
}
}
for (const [name, values] of media.attributes) {
for (const value of values) {
sdp += value ? `a=${name}:${value}\r\n` : `a=${name}\r\n`;
}
}
return sdp;
}
}
Codec Negotiation
Codec Registry and Management
interface Codec {
payloadType: number;
name: string;
clockRate: number;
channels?: number;
parameters?: Map<string, string>;
}
class CodecRegistry {
private static staticCodecs = new Map<number, Codec>([
[0, { payloadType: 0, name: 'PCMU', clockRate: 8000 }],
[3, { payloadType: 3, name: 'GSM', clockRate: 8000 }],
[4, { payloadType: 4, name: 'G723', clockRate: 8000 }],
[5, { payloadType: 5, name: 'DVI4', clockRate: 8000 }],
[6, { payloadType: 6, name: 'DVI4', : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }],
[, { : , : , : }]
]);
dynamicCodecs = <, >();
(: , : ): {
[encodingName, clockRateAndChannels] = rtpmap.();
parts = clockRateAndChannels.();
clockRate = (parts[]);
channels = parts[] ? (parts[]) : ;
..(payloadType, {
payloadType,
: encodingName,
clockRate,
channels
});
}
(: ): | {
..(payloadType) ||
..(payloadType);
}
(: , : ): {
codec = .(payloadType);
(!codec) {
;
}
(!codec.) {
codec. = ();
}
params = fmtp.().( p.());
( param params) {
[key, value] = param.().( s.());
codec..(key, value);
}
}
(): [] {
: [] = [];
codecs.(.....());
codecs.(.....());
codecs;
}
(
: [],
: [],
:
): [] {
: [] = [];
( format localFormats) {
(remoteFormats.(format)) {
payloadType = (format);
codec = .(payloadType);
(codec) {
common.(codec);
}
}
}
common;
}
}
Offer/Answer Model Implementation
Complete Offer/Answer Handler
class SdpOfferAnswer {
private localSession?: SdpSession;
private remoteSession?: SdpSession;
private negotiatedCodecs = new Map<string, Codec[]>();
private codecRegistry = new CodecRegistry();
createOffer(options: {
audio?: boolean;
video?: boolean;
localIp: string;
audioPort?: number;
videoPort?: number;
}): SdpSession {
const sessionId = this.generateSessionId();
const sessionVersion = sessionId;
const session: SdpSession = {
version: 0,
origin: {
username: 'user',
sessionId,
sessionVersion,
netType: 'IN',
addrType: 'IP4',
address: options.localIp
},
sessionName: 'SIP Call',
: {
: ,
: ,
: options.
},
: [{ : , : }],
: (),
: []
};
(options. !== ) {
: = {
: ,
: options. || ,
: ,
: [, , , ],
: ([
[, [
,
,
,
]],
[, [
,
]],
[, []],
[, []],
[, []]
])
};
..(, );
..(, );
session..(audioMedia);
}
(options.) {
: = {
: ,
: options. || ,
: ,
: [, , ],
: ([
[, [
,
,
]],
[, [
,
]],
[, [
,
,
,
,
,
]],
[, []]
])
};
..(, );
..(, );
..(, );
session..(videoMedia);
}
. = session;
session;
}
(: , : {
: ;
?: ;
?: ;
}): {
. = remoteOffer;
sessionId = .();
sessionVersion = sessionId;
: = {
: ,
: {
: ,
sessionId,
sessionVersion,
: ,
: ,
: options.
},
: ,
: {
: ,
: ,
: options.
},
: [{ : , : }],
: (),
: []
};
( remoteMedia remoteOffer.) {
answerMedia = .(remoteMedia, options);
(answerMedia) {
answer..(answerMedia);
}
}
. = answer;
answer;
}
(
: ,
: {
?: ;
?: ;
}
): | {
remoteCodecs = .(remoteMedia);
localCodecs = .(remoteMedia.);
commonCodecs = .(localCodecs, remoteCodecs);
(commonCodecs. === ) {
{
: remoteMedia.,
: ,
: remoteMedia.,
: [],
: ()
};
}
..(remoteMedia., commonCodecs);
port = remoteMedia. ===
? (options. || )
: (options. || );
: = {
: remoteMedia.,
port,
: remoteMedia.,
: commonCodecs.( c..()),
: ()
};
: [] = [];
: [] = [];
( codec commonCodecs) {
rtpmap = ;
(codec. && codec. > ) {
rtpmap += ;
}
rtpmaps.(rtpmap);
(codec. && codec.. > ) {
params = .(codec..())
.( )
.();
fmtps.();
}
}
answerMedia..(, rtpmaps);
(fmtps. > ) {
answerMedia..(, fmtps);
}
direction = remoteMedia..() ? :
remoteMedia..() ? :
remoteMedia..() ? :
remoteMedia..() ? :
;
answerMedia..(direction, []);
ptime = remoteMedia..();
(ptime) {
answerMedia..(, ptime);
}
answerMedia;
}
(: ): {
. = remoteAnswer;
( remoteMedia remoteAnswer.) {
(remoteMedia. === ) {
.();
;
}
codecs = .(remoteMedia);
..(remoteMedia., codecs);
}
}
(: ): [] {
: [] = [];
rtpmaps = media..() || [];
( rtpmap rtpmaps) {
match = rtpmap.();
(match) {
payloadType = (match[]);
..(payloadType, match[]);
}
}
fmtps = media..() || [];
( fmtp fmtps) {
match = fmtp.();
(match) {
payloadType = (match[]);
..(payloadType, match[]);
}
}
( format media.) {
payloadType = (format);
codec = ..(payloadType);
(codec) {
codecs.(codec);
}
}
codecs;
}
(: ): [] {
(mediaType === ) {
[
{ : , : , : },
{ : , : , : },
{ : , : , : , : },
{ : , : , : }
];
} (mediaType === ) {
[
{ : , : , : },
{ : , : , : }
];
}
[];
}
(: [], : []): [] {
: [] = [];
( remoteCodec remoteCodecs) {
match = localCodecs.(
local..() === remoteCodec..() &&
local. === remoteCodec. &&
(local. || ) === (remoteCodec. || )
);
(match) {
common.({
...match,
: remoteCodec.,
: remoteCodec.
});
}
}
common;
}
(): {
.().();
}
(: ): [] {
..(mediaType) || [];
}
(: ): | {
codecs = ..(mediaType);
codecs && codecs. > ? codecs[] : ;
}
}
Advanced Media Features
ICE Candidate Handling
interface IceCandidate {
foundation: string;
component: number;
transport: string;
priority: number;
address: string;
port: number;
type: 'host' | 'srflx' | 'prflx' | 'relay';
relAddr?: string;
relPort?: number;
}
class IceCandidateHandler {
static parseCandidate(attr: string): IceCandidate | null {
const parts = attr.split(' ');
if (parts.length < 8) {
return null;
}
const candidate: IceCandidate = {
foundation: parts[0],
component: parseInt(parts[1]),
transport: parts[2],
: (parts[]),
: parts[],
: (parts[]),
: parts[] []
};
( i = ; i < parts.; i += ) {
key = parts[i];
value = parts[i + ];
(key === ) {
candidate. = value;
} (key === ) {
candidate. = (value);
}
}
candidate;
}
(: ): {
attr = +
+
;
(candidate. && candidate.) {
attr += ;
}
attr;
}
(
: [],
: ,
: =
): {
typePreference = {
: ,
: ,
: ,
:
}[];
( ** ) * typePreference +
( ** ) * localPreference +
( - component);
}
(: , : []): {
( media sdp.) {
mediaCandidates = candidates.(
c. === (media. === ? : )
);
candidateAttrs = mediaCandidates.( .(c));
media..(, candidateAttrs);
}
}
}
RTCP Feedback Configuration
class RtcpFeedback {
static addVideoFeedback(media: SdpMedia, payloadTypes: number[]): void {
const feedbackTypes = [
'nack',
'nack pli',
'ccm fir',
'goog-remb',
'transport-cc'
];
const rtcpFb: string[] = [];
for (const pt of payloadTypes) {
for (const fb of feedbackTypes) {
rtcpFb.push(`${pt} ${fb}`);
}
}
media.attributes.set('rtcp-fb', rtcpFb);
}
static parseFeedback(attr: string): { payloadType: number; type: string; parameter?: string } | null {
match = attr.();
(!match) {
;
}
{
: match[] === ? - : (match[]),
: match[],
: match[]
};
}
}
Media Capability Negotiation
Simulcast and SVC Support
class MediaCapabilities {
static addSimulcast(media: SdpMedia, sendRids: string[]): void {
const ridAttrs = sendRids.map(rid => `${rid} send`);
media.attributes.set('rid', ridAttrs);
const simulcastAttr = `send ${sendRids.join(';')}`;
media.attributes.set('simulcast', [simulcastAttr]);
}
static parseSimulcast(attr: string): {
send?: string[];
recv?: string[];
} {
const result: { send?: string[]; recv?: string[] } = {};
const parts = attr.split(/\s+/);
for (let i = 0; i < parts.length; i++) {
(parts[i] === && parts[i + ]) {
result. = parts[i + ].();
i++;
} (parts[i] === && parts[i + ]) {
result. = parts[i + ].();
i++;
}
}
result;
}
(: , : ): {
media..(, [
,
]);
fmtps = media..() || [];
svcParams = ;
existingIndex = fmtps.( f.());
(existingIndex >= ) {
fmtps[existingIndex] += ;
} {
fmtps.();
}
media..(, fmtps);
}
}
Complete SIP Offer/Answer Example
class SipMediaSession {
private offerAnswer = new SdpOfferAnswer();
async initiateCall(callee: string, localIp: string): Promise<string> {
const offer = this.offerAnswer.createOffer({
audio: true,
video: false,
localIp,
audioPort: 49170
});
const sdpBody = SdpParser.stringify(offer);
const invite = `INVITE sip:${callee} SIP/2.0\r
Via: SIP/2.0/UDP ${localIp}:5060;branch=z9hG4bK${this.generateBranch()}\r
Max-Forwards: 70\r
To: <sip:${callee}>\r
From: <sip:caller@example.com>;tag=${this.generateTag()}\r
Call-ID: ${this.generateCallId()}\r
CSeq: 1 INVITE\r
Contact: <sip:caller@${localIp}:5060>\r
Content-Type: application/sdp\r
Content-Length: ${sdpBody.length}\r
\r
${sdpBody}`;
return invite;
}
async acceptCall(: , : ): <> {
sdpStart = inviteMessage.();
sdpBody = inviteMessage.(sdpStart);
offer = .(sdpBody);
answer = ..(offer, {
localIp,
:
});
codec = ..();
.(, codec);
answerSdp = .(answer);
response = ;
response;
}
(: ): {
sdpStart = responseMessage.();
sdpBody = responseMessage.(sdpStart);
answer = .(sdpBody);
..(answer);
audioCodecs = ..();
.(, audioCodecs);
}
(): {
.().().();
}
(): {
.().().();
}
(): {
;
}
}
When to Use This Skill
Use sip-media-negotiation when building applications that require:
- Setting up audio/video calls with codec negotiation
- Implementing SDP offer/answer model
- Parsing and generating SDP messages
- Negotiating media capabilities between endpoints
- Handling multiple codec support
- Implementing ICE for NAT traversal
- Configuring RTCP feedback for video
- Supporting advanced features like simulcast
- Building WebRTC-SIP gateways
- Creating multi-party conferencing systems
Best Practices
- Always validate SDP structure - Parse and validate before processing
- Support multiple codecs - Offer fallback options for compatibility
- Use payload type 96+ for dynamic codecs - Follow RFC 3551 guidelines
- Include rtpmap for dynamic types - Even if well-known, be explicit
- Add format parameters (fmtp) - Specify codec configuration details
- Respect media direction attributes - sendrecv, sendonly, recvonly, inactive
- Handle rejected media (port=0) - Gracefully handle unsupported media
- Update session version on modification - Increment o= version field
- Include timing information - Required t= line even if permanent (0 0)
- Set appropriate ptime - Balance latency vs packet overhead
- Support telephone-event - Enable DTMF transmission (RFC 4733)
- Add ICE candidates when using ICE - Include all candidate types
- Configure RTCP feedback for video - Enable error resilience features
- Order codecs by preference - Most preferred first in format list
- Preserve codec parameters in answer - Match offerer's fmtp settings
Common Pitfalls
- Missing rtpmap for dynamic payloads - Causes codec mismatch
- Incorrect payload type numbering - Use 96-127 for dynamic, 0-95 for static
- Not handling rejected media - Assumes all media accepted
- Ignoring format parameters - Codec may not work correctly
- Wrong clock rate in rtpmap - Audio uses 8000, video uses 90000 typically
- Missing required SDP lines - v=, o=, s=, t= are mandatory
- Not updating session version - Causes confusion on re-INVITE
- Mismatched payload types - Using different PT for same codec
- Forgetting Content-Length - SIP requires accurate body length
- Not escaping special characters - URI parameters must be encoded
- Wrong media direction logic - sendonly should be answered with recvonly
- Missing connection information - c= required at session or media level
- Incorrect component numbers - RTP=1, RTCP=2 for ICE candidates
- Not prioritizing secure codecs - Prefer encrypted over plaintext
- Hardcoded ports - Use dynamic port allocation for multiple sessions
Resources