RE: FW: IETF BMWG Work Item Proposal for SIP PerformanceBenchmarking

I have a clarification question. How do you define 'SIP Media Plane'? It
looks to me that all the benchmark metrics and test parameters included
under this label are generic for any RTP stream benchmark and not
necessarily related to SIP. Am I missing something? 

Dan

> -----Original Message-----
> From: sporetsky <at> reefpoint.com [mailto:sporetsky <at> reefpoint.com] 
> Sent: Wednesday, May 24, 2006 12:50 AM
> To: bmwg <at> ietf.org
> Subject: [bmwg] FW: IETF BMWG Work Item Proposal for SIP 
> PerformanceBenchmarking
> Importance: High
> 
> 
> Fellow BMWG-ers,
> 
> Please find this IETF Work Item Proposal for your review and 
> comment.  The authors will be submitting a first draft 
> Terminology and Methodology for the Montreal meeting.
> 
> Scott
> 
> >  -----Original Message-----
> > From: 	Poretsky, Scott  
> > Sent:	Thursday, May 11, 2006 12:44 PM
> > To:	'acmorton <at> att.com'
> > Subject:	IETF BMWG Work Item Proposal for SIP Performance
> > Benchmarking
> > Importance:	High
> > 
> > Hi Al,
> > 
> [snip] 
> 
> > Thanks,
> > Scott
> > 
> > ########################
> > BMWG Work Item Proposal
> > 
> > TITLE: Session Initiation Protocol (SIP) Performance Benchmarking
> > 
> > AUTHOR TEAM:
> > Dr. Carol Davids, Illinois Institute of Technology Dr. 
> Vijay Gurbani, 
> > Lucent Technologies
> Scott Poretsky, Reef Point Systems
> >  
> > GOALS
> > The purpose of this work item is to provide a single 
> terminology and 
> > methodology from which SIP equipment vendors and VoIP service 
> > providers can measure performance benchmarking metrics for 
> comparison and selection.
> > It is intended to develop terms, benchmarks, and methodologies that 
> > can be applied to any type of IP device including SIP 
> Servers, Session 
> > Border Controllers (SBCs), and Security Gateways (SEGs).
> > 
> > MOTIVATION
> > Service Providers are now planning VoIP and Multimedia network 
> > deployments using the IETF developed Session Initiation Protocol 
> > (SIP).  Through SIP, service providers will be able to have rich 
> > service offerings and many even plan to turn off their 
> Public Switch Telephone Network within five
> > years.   VoIP has led to development of new networking 
> devices including
> > SIP Servers, Session Border Controllers, and Security 
> Gateways.  The 
> > mix of voice and IP functions in this variety of devices 
> has produced 
> > inconsistencies in vendor reported performance metrics and 
> has caused 
> > confusion in the service provider community.
> > 
> > SCOPE:
> > This work item will provide terms, benchmarks, and 
> methodologies for 
> > performance benchmarking the SIP control and media planes.  The 
> > methodologies can be used for benchmarking SIP performance of SIP 
> > servers, SBCs, and SEGs.  The media used for the 
> benchmarking will be 
> > the IETF defined Real-Time Protocol (RTP).  Test cases will allow 
> > testing of VoIP or Multimedia over IP by varying the number 
> of media 
> > streams per SIP call from 1 to higher.  There will be at least one 
> > test case to measure the impact of a SIP DOS Attack on the 
> performance 
> > metrics.  All SIP traffic will be in the clear; no 
> encryption using TLS or IPsec will be used.
> > Similar benchmarks and methodologies using encryption is a 
> potential 
> > work item for the BMWG that could be considered in the future.
> > 
> > There will be separate test cases in the methodology document for 
> > obtaining measurements for each of the Benchmarking Metrics 
> defined in 
> > the Terminology.  The benchmarking metrics to be measured 
> for the SIP 
> > Control Plane and SIP Media Plane are as follow:
> > 
> > 	Benchmark Metrics - SIP Control Plane 
> > 	Standing Calls, maximum (calls)
> > 	Calls Per Second, maximum (CPS)
> > 	Call Attempts Per Second, maximum (CAPS)
> > 	Busy Hour Call Attempts, maximum (BHCA)
> > 	Busy Hour Call Connects, maximum (BHCC)
> > 	Call Completion Rate (%)
> > 	Call Setup Delay, average (msec)
> > 	Call Teardown Delay, average (msec)
> > 
> > 	Benchmark Metrics- SIP Media Plane
> > 	RTP Media Throughput, per Media Stream (pps)
> > 	RTP Media Throughput, Aggregate (pps)
> > 	RTP Packet Loss, average (pps)
> > 	RTP Packet Delay, average (msec)
> > 	RTP Packet Jitter, average (msec)
> > 
> > In order to obtain measurements for the benchmarking 
> metrics it will 
> > be necessary to configure the test setup with certain test 
> parameters.  
> > Some of these test parameters may also be the benchmarking metric 
> > being measured for another test case.  The test parameters to be 
> > configured for the SIP Control Plane and SIP Media Plane 
> are as follow:
> > 
> > 	Test Parameters- SIP Control Plane 
> > 	Call Duration (msec)
> > 	Call Per Second (CPS)
> > 	Call Attempts Per Second (CAPS)
> > 
> > 	Test Parameters - SIP Media Plane
> > 	RTP Media Streams per Call (streams per call)
> > 	RTP Packet Size (bytes)
> > 	RTP Media Offered Load, per Media Stream (pps)
> > 	RTP Media Offered Load, Aggregate (pps)
> > 
> > The basic test topology to be used for this benchmarking is 
> as follows:
> > 
> > Emulated Agents<--> SEG (optional) <--> SIP Proxy Server 
> (or SBC) <--> 
> > SIP Server (optional) <--> Emulated Agents
> > 
> > PROPOSED MILESTONES
> > 06/07 First Draft Terminology and Methodology
> > 02/07 First WG Last Call
> > 05/07 SIP WG and SIPPING WG Review
> > 06/07 Final BMWG Last Call for Terminology and Methodology
> > 08/07 Submittal for IESG review
> > 
> > 
> > 
> 
> _______________________________________________
> bmwg mailing list
> bmwg <at> ietf.org
> https://www1.ietf.org/mailman/listinfo/bmwg
>