Asterisk bandwidth iax2

IAX2 Bandwidth Study

John Todd (jtodd @loligo.com)

2003-06-28 <- Notice how old this is.

Purpose:

To obtain a better chart of actual bandwidth usage per codec as seen "on-the-wire" when using IAX2 trunking between two Asterisk telephony servers.

A word of warning

Q: Does anybody have any reasons for not using Trunking? Any disadvantages?
A: In good old days, jitterbuffer and IAX2 trunking made a bad match. Yourcall...wowould...sstartto....soundlike....this :-)
(2008-07-16 update: jitter buffering works much better these days - JT)

Discussion:

Past threads on the asterisk-dev and asterisk-users lists have indicated that the optimal way to save bandwidth on multiple calls to the same destination is to use IAX2 (Inter-Asterisk eXchange version 2) in "trunk" mode, which eliminates IP overhead to a large degree.
This trunking eliminates the IP overhead found in individual VoIP IP streams by pipelining RTP data from multiple calls into single (larger) packets, thus removing the redundancy of IP overhead for each RTP stream and more closely allowing bandwidth scaling as a function of codec usage instead of a function of (codec usage + IP
overhead.) Of course, this mode can only be used if all the calls are between two specific Asterisk servers, but this is frequently the case with toll-avoidance situations or between two branch offices where there is an Asterisk server at each location.

Various statements have been made as to which codec is the most efficient at moving traffic over limited bandwidth. Often, these statements would quote the codec usage itself, and not include the IP and/or IAX2 overhead which is vital in computing actual bandwidth usage between two endpoints on the Internet - merely measuring codec theoretical usage is insufficient for real-world costing and provisioning purposes.

I needed to actually quantify these numbers with 'real' on-the-wire examinations of traffic to build my own competence with quoting these figures, as they are frequently requested from me by clients. To this end I had never compared the major protocols side-by-side in any formal manner. Thus, on a beautiful Saturday morning when I should have been outside at the farmer's market or on my bike, I sat inside in a darkened computer room and said "blah" several hundred times at a movie theater recording.

These numbers may come as no surprise to those who have configured them before, but I have not as yet seen a back-to-back comparison here, so I post it for your examination and my record-keeping. I am a firm proponent of "don't believe everything you hear" which applies doubly-so for Internet rumors, so I had to test these codecs myself to prevent rude surprises in the future. In keeping with that spirit, you should not believe my numbers, and test these yourself and post updates to the list to double-check my methods and results in a public forum.

Experiment notes and methods:

  • Raw figures below are bi-directional, meaning that the numbers include both incoming/outgoing voice traffic. "Cooked" figures are included in parenthesis - raw numbers were divided by two and turned into kbps for your ease of reading. Packets per second not cooked, and that is left as an exercise to the reader.
  • Phones were Cisco 7960 devices
  • Calls were bi-directional with audio, using the same recording at the PSTN side, and my use of the word "blah" repeatedly spoken into the 7960 at a steady rate
  • Diagram:

7960(G.711) -> *(IAX2) -> Internet -> *(IAX2) -> T100P -> PSTN (PRI)

  • The number for "Estimated IP Overhead" was obtained by subtracting (additional channel usage) from (single channel usage.) This is possibly inaccurate.
  • Asterisk server talking to SIP phones: Asterisk CVS-06/27/03-07:29:29
  • Asterisk server talking to PRI: Asterisk CVS-05/26/03-15:30:05
  • Method used to obtain figures "rate -b -f 'host my.iax2.host and not port ssh' "
  • All tests were done with IAX2 trunking turned on ("trunk=yes" in iax.conf for each peer)
  • All tests were verified on the remote host for use of the proper codec during calls
  • All tests performed over one minute
  • All measurements performed twice, and averaged
  • Calls were allowed to complete, then testing started for one minute, then testing stopped, and calls were hung up (in other words, call setup and teardown was not factored into any of these figures, but I suspect that will not make a difference)
  • Protocols untested: G.723.1, adpcm, mp3, slinear
  • When measuring bandwidth, "kilobit" is 1000 bits per second (not 1024) and a megabit is 1,000,000 bits per second


Testing results:

  • G.711 (ulaw)
one call: 164333.75 bps/94.26 pps ( 82.1 kbps)
two calls: 296171.60 bps/101.46 pps (148.0 kbps)

Thus:
  • For every additional call: 131837 bps (65.9 kbps)
  • Est. IP/IAX2 overhead (1 call): 32495 bps (16.0 kbps)
  • Raw number of calls per megabit: 15

  • ILBC: see note below
one call: 56134.91 bps/67.45 pps (28.0 kbps)
two calls: 98679.11 bps/102.41 pps (49.3 kbps)

Thus:
  • For every additional call: 42544 bps (21.2 kbps)
  • Est. IP/IAX2 overhead (1 call): 13590 bps ( 6.7 kbps)
  • Raw number of calls per megabit: 47


  • G.729
one call: 60124.33 bps/101.26 pps (30.0 kbps)
two calls: 79496.23 bps/102.85 pps (39.7 kbps)

Thus:
  • For every additional call: 19372 bps ( 9.6 kbps)
  • Est. IP/IAX2 overhead (1 call): 40752 bps (20.3 kbps)
  • Raw number of calls per megabit: 103


one call: 70958.16 bps/102.13 pps (35.4 kbps)
two calls: 100455.23 bps/102.63 pps (50.2 kbps)

Thus:
  • For every additional call: 29497 bps (14.7 kbps)
  • Est. IP/IAX2 overhead (1 call): 41461 bps (20.7 kbps)
  • Raw number of calls per megabit: 68

one call: 43855.44 bps/89.94 pps (21.9 kbps)
two calls: 56059.18 bps/100.81 pps (28.0 kbps)

Thus:
For every additional call: 12203 bps ( 6.1 kbps)
Est. IP/IAX2 overhead (1 call): 31561 bps (15.8 kbps)
Raw number of calls per megabit: 164


  • (SPEEX):
one call: 74817.18 bps/101.06 pps (37.4 kbps)
two calls: 109692.68 bps/102.18 pps (54.8 kbps)

Thus:
  • For every additional call: 34875 bps (17.4 kbps)
  • Est. IP/IAX2 overhead (1 call): 39941 bps (19.9 kbps)
  • Raw number of calls per megabit: 57

Conclusions:

lpc10 is the clear winner for thrifty use of bandwidth. However, voice quality is on the robotic-sounding side, and may not be acceptable for users who are expecting something near toll-quality.
Calls are perfectly understandable, but nuances are lost with this codec. This was the only codec that had significant quality issues that were worth mentioning in this study; other codecs have different sound quality ratings, but their differences are minor enough that they are not relevant to the scope of this examination. That being said about call quality, there is certainly a tradeoff to getting ~164 calls in a megabit, which is extremely impressive.

G.729 is the next best from a bandwidth standpoint, and expected additional channel usage is very close to the quoted bandwidth of G.729 codec usage - ~9.6kbps in each direction. The next most efficient codec would be GSM at 14.7kbps, and third would be Speex at 17.4kbps.

Somewhat surprising were the IP/IAX2 overhead figures for ILBC - they are almost one-third that of some other protocols. I tested that particular number three times to make sure I did not make any errors, and all tests were within 3% of each other, so that is an unexpected but consistent result. Of course, my method for obtaining these estimated IP/IAX2 overhead figures may be inappropriate - I welcome comments on the method used to estimate those numbers.

{A comment on overhead for iLBC: if I'm not mistaken, iLBC uses 30millisecond frames. With the default 20millisecond trunk frequency, there is only 2/3rds of a frame to be sent at each trunkfreq interval. So about 1/3rd of the trunk packets don't have to be sent at all. Or, looking at it a different way, 60msec of audio can be sent using only 2 sets of iax/ip headers, whilst other codecs would need 3 sets of iax/ip headers. So there is at least some of the difference — Steve Davies}

{Note A comment on the above comment and iLBC. The above explanation is pretty much on target. What actually happens is this:

Trunking occurs every 20 milliseconds. iLBC packets are sent every 30 milliseconds

Thus, only every other set of iLBC packets are trunked (eg at 60ms, 120ms, 180ms, etc) when the ilBC transmission coincides with IAX2 trunking. Half of the packets will enjoy(?) trunking, while the other half won't. For the 2 call case, half of the transmission will be at 2x headers, while the other half will be at 1x headers. On average, the transmission will be incurring the cost of 3/2 x headers.
Graphically:

Quantity of IP headers in the 2 call case for iLBC with IAX2 trunking

(ratio of IP headers to the 1 call case)
     2|  ___     ___     ___     ___     ___
     1|     |___|   |___|   |___|   |___|
     0|______________________________________
      |   0  30  60  90  120 180 210 240 300
      				time (milliseconds)


Therefore the equations utilizing the above data will actually be:
28.0 kb/s = 3/2*headers + codec
49.3 kb/s = 2*headers + 2*codec

Solving simultaneously, we see that 6.7kb/s is actually 1/2 the headers! And so the IP overhead according to that data is actually 13.4kb/s. This compares favourably with the theoretical value of 12.7kb/s. This lower IP overhead is directly related to the lower transmission frequency. Since only 2/3 as many packets are transmitted (once every 30ms opposed to once every 20ms), the overhead should be 2/3 of a similar codec with 20ms transmission periods (eg compared to G.729, 2/3 x 20.3kb/s = 13.53kb/s, which again compares favourably with the test figure of 13.4kb/s)

And so we can gather the following from the data:
one call: 56134.91 bps/67.45 pps (28.0 kbps)
two calls: 98679.11 bps/102.41 pps (49.3 kbps)

Thus:
  • For every additional call: 42544 bps (14.6 kbps)
  • Est. IP/IAX2 overhead (1 call): 13590 bps ( 13.4 kbps)
  • Raw number of calls per megabit: 67

Also it is not stated if these results take into account the effects of voice activity detection (and the resulting silence suppression). Though asterisk itself does not support it at this time (28/3/05) the phone can still do this resulting in lower bandwidth in one direction. Also, in the future asterisk will support VAD :rolleyes:.
[Shameless plug: I too am a consultant. Mail me (herman.webley@blitzllc.com) for details.]

Caveats:

I only had two phones with which to test, and only a limited amount of time. More accurate testing would be achieved by putting many more lines into a trunk and watching the bits-per-second growth of the IAX2 trunk - two calls is probably not sufficient to examine true bandwidth usage in anything other than a general way. Most interesting would be to watch the growth of LPC10 channel usage, as they are supposed to be only 2.4kbps, which could possibly result in many more channels per megabit if the overhead figures stay roughly the same with more channels added. The 6.1 number for LPC10 seems a bit high, given that the quoted usage is 2.4kbps.

References:




Shameless plug:

I am a consultant, and I'm happy to do this kind of work to further develop your products or deployments of VoIP/Asterisk systems. Mail me (jtodd@loligo.com) for details.

Shameless plug - with useful info:

I too will add a shamless plug, I also do VoIP and network consulting and can be reached at trixterNOSPAM@0xdecafbad.com. Now for the useful information. Keep in mind the link layer framing that is used as well. ATM (what most of the internet backbones use) has 53 byte fixed cells. There are 5 bytes of headers. Only one packet can exist in a group of cells (a group is 1 or more cell until the whole packet is sent). This means that if you try to send a 80 byte IP packet over an ATM link it will be chopped into 2 ATM cells with 16 bytes of padding. This is only 83% efficient. By adjusing your sample size you may find that your throughput can be increased because you transfer more useful data and less padding. Make your sample size too small and you have a lot of IP overhead, make it too large and it can cause problems with call quality (think of a 30ms jitter buffer and 30ms sample sizes, in effect you have no jitter buffer because packets cant be reordered, jitter cant be controlled, etc). Its a fine balance, but something to consider. Even if you dont use ATM the internet backbones often do, so this is something that may make slightly faster transfers and better network efficiency. DSL, E1, T1, SMDS, OC1, OC3, OC12 etc generally all use ATM, so its quite common.

Real life measurements

I have set up IAX2 trunking between Paris and Reunion Island (where bandwith is expensive) and trunking 15 channels of g.729 calls shows an average of 12.5 kbps / call (using ntop to measure inbound + outbound bandwith usage). While not as good as the theoretical figures, trunking on g729 just about halves bandwith requirements (you need 25kbps / call with SIP+g729) with no noticeable loss of quality. Jean-Michel Hiver

JT


See also

IAX2 Bandwidth Study

John Todd (jtodd @loligo.com)

2003-06-28 <- Notice how old this is.

Purpose:

To obtain a better chart of actual bandwidth usage per codec as seen "on-the-wire" when using IAX2 trunking between two Asterisk telephony servers.

A word of warning

Q: Does anybody have any reasons for not using Trunking? Any disadvantages?
A: In good old days, jitterbuffer and IAX2 trunking made a bad match. Yourcall...wowould...sstartto....soundlike....this :-)
(2008-07-16 update: jitter buffering works much better these days - JT)

Discussion:

Past threads on the asterisk-dev and asterisk-users lists have indicated that the optimal way to save bandwidth on multiple calls to the same destination is to use IAX2 (Inter-Asterisk eXchange version 2) in "trunk" mode, which eliminates IP overhead to a large degree.
This trunking eliminates the IP overhead found in individual VoIP IP streams by pipelining RTP data from multiple calls into single (larger) packets, thus removing the redundancy of IP overhead for each RTP stream and more closely allowing bandwidth scaling as a function of codec usage instead of a function of (codec usage + IP
overhead.) Of course, this mode can only be used if all the calls are between two specific Asterisk servers, but this is frequently the case with toll-avoidance situations or between two branch offices where there is an Asterisk server at each location.

Various statements have been made as to which codec is the most efficient at moving traffic over limited bandwidth. Often, these statements would quote the codec usage itself, and not include the IP and/or IAX2 overhead which is vital in computing actual bandwidth usage between two endpoints on the Internet - merely measuring codec theoretical usage is insufficient for real-world costing and provisioning purposes.

I needed to actually quantify these numbers with 'real' on-the-wire examinations of traffic to build my own competence with quoting these figures, as they are frequently requested from me by clients. To this end I had never compared the major protocols side-by-side in any formal manner. Thus, on a beautiful Saturday morning when I should have been outside at the farmer's market or on my bike, I sat inside in a darkened computer room and said "blah" several hundred times at a movie theater recording.

These numbers may come as no surprise to those who have configured them before, but I have not as yet seen a back-to-back comparison here, so I post it for your examination and my record-keeping. I am a firm proponent of "don't believe everything you hear" which applies doubly-so for Internet rumors, so I had to test these codecs myself to prevent rude surprises in the future. In keeping with that spirit, you should not believe my numbers, and test these yourself and post updates to the list to double-check my methods and results in a public forum.

Experiment notes and methods:

  • Raw figures below are bi-directional, meaning that the numbers include both incoming/outgoing voice traffic. "Cooked" figures are included in parenthesis - raw numbers were divided by two and turned into kbps for your ease of reading. Packets per second not cooked, and that is left as an exercise to the reader.
  • Phones were Cisco 7960 devices
  • Calls were bi-directional with audio, using the same recording at the PSTN side, and my use of the word "blah" repeatedly spoken into the 7960 at a steady rate
  • Diagram:

7960(G.711) -> *(IAX2) -> Internet -> *(IAX2) -> T100P -> PSTN (PRI)

  • The number for "Estimated IP Overhead" was obtained by subtracting (additional channel usage) from (single channel usage.) This is possibly inaccurate.
  • Asterisk server talking to SIP phones: Asterisk CVS-06/27/03-07:29:29
  • Asterisk server talking to PRI: Asterisk CVS-05/26/03-15:30:05
  • Method used to obtain figures "rate -b -f 'host my.iax2.host and not port ssh' "
  • All tests were done with IAX2 trunking turned on ("trunk=yes" in iax.conf for each peer)
  • All tests were verified on the remote host for use of the proper codec during calls
  • All tests performed over one minute
  • All measurements performed twice, and averaged
  • Calls were allowed to complete, then testing started for one minute, then testing stopped, and calls were hung up (in other words, call setup and teardown was not factored into any of these figures, but I suspect that will not make a difference)
  • Protocols untested: G.723.1, adpcm, mp3, slinear
  • When measuring bandwidth, "kilobit" is 1000 bits per second (not 1024) and a megabit is 1,000,000 bits per second


Testing results:

  • G.711 (ulaw)
one call: 164333.75 bps/94.26 pps ( 82.1 kbps)
two calls: 296171.60 bps/101.46 pps (148.0 kbps)

Thus:
  • For every additional call: 131837 bps (65.9 kbps)
  • Est. IP/IAX2 overhead (1 call): 32495 bps (16.0 kbps)
  • Raw number of calls per megabit: 15

  • ILBC: see note below
one call: 56134.91 bps/67.45 pps (28.0 kbps)
two calls: 98679.11 bps/102.41 pps (49.3 kbps)

Thus:
  • For every additional call: 42544 bps (21.2 kbps)
  • Est. IP/IAX2 overhead (1 call): 13590 bps ( 6.7 kbps)
  • Raw number of calls per megabit: 47


  • G.729
one call: 60124.33 bps/101.26 pps (30.0 kbps)
two calls: 79496.23 bps/102.85 pps (39.7 kbps)

Thus:
  • For every additional call: 19372 bps ( 9.6 kbps)
  • Est. IP/IAX2 overhead (1 call): 40752 bps (20.3 kbps)
  • Raw number of calls per megabit: 103


one call: 70958.16 bps/102.13 pps (35.4 kbps)
two calls: 100455.23 bps/102.63 pps (50.2 kbps)

Thus:
  • For every additional call: 29497 bps (14.7 kbps)
  • Est. IP/IAX2 overhead (1 call): 41461 bps (20.7 kbps)
  • Raw number of calls per megabit: 68

one call: 43855.44 bps/89.94 pps (21.9 kbps)
two calls: 56059.18 bps/100.81 pps (28.0 kbps)

Thus:
For every additional call: 12203 bps ( 6.1 kbps)
Est. IP/IAX2 overhead (1 call): 31561 bps (15.8 kbps)
Raw number of calls per megabit: 164


  • (SPEEX):
one call: 74817.18 bps/101.06 pps (37.4 kbps)
two calls: 109692.68 bps/102.18 pps (54.8 kbps)

Thus:
  • For every additional call: 34875 bps (17.4 kbps)
  • Est. IP/IAX2 overhead (1 call): 39941 bps (19.9 kbps)
  • Raw number of calls per megabit: 57

Conclusions:

lpc10 is the clear winner for thrifty use of bandwidth. However, voice quality is on the robotic-sounding side, and may not be acceptable for users who are expecting something near toll-quality.
Calls are perfectly understandable, but nuances are lost with this codec. This was the only codec that had significant quality issues that were worth mentioning in this study; other codecs have different sound quality ratings, but their differences are minor enough that they are not relevant to the scope of this examination. That being said about call quality, there is certainly a tradeoff to getting ~164 calls in a megabit, which is extremely impressive.

G.729 is the next best from a bandwidth standpoint, and expected additional channel usage is very close to the quoted bandwidth of G.729 codec usage - ~9.6kbps in each direction. The next most efficient codec would be GSM at 14.7kbps, and third would be Speex at 17.4kbps.

Somewhat surprising were the IP/IAX2 overhead figures for ILBC - they are almost one-third that of some other protocols. I tested that particular number three times to make sure I did not make any errors, and all tests were within 3% of each other, so that is an unexpected but consistent result. Of course, my method for obtaining these estimated IP/IAX2 overhead figures may be inappropriate - I welcome comments on the method used to estimate those numbers.

{A comment on overhead for iLBC: if I'm not mistaken, iLBC uses 30millisecond frames. With the default 20millisecond trunk frequency, there is only 2/3rds of a frame to be sent at each trunkfreq interval. So about 1/3rd of the trunk packets don't have to be sent at all. Or, looking at it a different way, 60msec of audio can be sent using only 2 sets of iax/ip headers, whilst other codecs would need 3 sets of iax/ip headers. So there is at least some of the difference — Steve Davies}

{Note A comment on the above comment and iLBC. The above explanation is pretty much on target. What actually happens is this:

Trunking occurs every 20 milliseconds. iLBC packets are sent every 30 milliseconds

Thus, only every other set of iLBC packets are trunked (eg at 60ms, 120ms, 180ms, etc) when the ilBC transmission coincides with IAX2 trunking. Half of the packets will enjoy(?) trunking, while the other half won't. For the 2 call case, half of the transmission will be at 2x headers, while the other half will be at 1x headers. On average, the transmission will be incurring the cost of 3/2 x headers.
Graphically:

Quantity of IP headers in the 2 call case for iLBC with IAX2 trunking

(ratio of IP headers to the 1 call case)
     2|  ___     ___     ___     ___     ___
     1|     |___|   |___|   |___|   |___|
     0|______________________________________
      |   0  30  60  90  120 180 210 240 300
      				time (milliseconds)


Therefore the equations utilizing the above data will actually be:
28.0 kb/s = 3/2*headers + codec
49.3 kb/s = 2*headers + 2*codec

Solving simultaneously, we see that 6.7kb/s is actually 1/2 the headers! And so the IP overhead according to that data is actually 13.4kb/s. This compares favourably with the theoretical value of 12.7kb/s. This lower IP overhead is directly related to the lower transmission frequency. Since only 2/3 as many packets are transmitted (once every 30ms opposed to once every 20ms), the overhead should be 2/3 of a similar codec with 20ms transmission periods (eg compared to G.729, 2/3 x 20.3kb/s = 13.53kb/s, which again compares favourably with the test figure of 13.4kb/s)

And so we can gather the following from the data:
one call: 56134.91 bps/67.45 pps (28.0 kbps)
two calls: 98679.11 bps/102.41 pps (49.3 kbps)

Thus:
  • For every additional call: 42544 bps (14.6 kbps)
  • Est. IP/IAX2 overhead (1 call): 13590 bps ( 13.4 kbps)
  • Raw number of calls per megabit: 67

Also it is not stated if these results take into account the effects of voice activity detection (and the resulting silence suppression). Though asterisk itself does not support it at this time (28/3/05) the phone can still do this resulting in lower bandwidth in one direction. Also, in the future asterisk will support VAD :rolleyes:.
[Shameless plug: I too am a consultant. Mail me (herman.webley@blitzllc.com) for details.]

Caveats:

I only had two phones with which to test, and only a limited amount of time. More accurate testing would be achieved by putting many more lines into a trunk and watching the bits-per-second growth of the IAX2 trunk - two calls is probably not sufficient to examine true bandwidth usage in anything other than a general way. Most interesting would be to watch the growth of LPC10 channel usage, as they are supposed to be only 2.4kbps, which could possibly result in many more channels per megabit if the overhead figures stay roughly the same with more channels added. The 6.1 number for LPC10 seems a bit high, given that the quoted usage is 2.4kbps.

References:




Shameless plug:

I am a consultant, and I'm happy to do this kind of work to further develop your products or deployments of VoIP/Asterisk systems. Mail me (jtodd@loligo.com) for details.

Shameless plug - with useful info:

I too will add a shamless plug, I also do VoIP and network consulting and can be reached at trixterNOSPAM@0xdecafbad.com. Now for the useful information. Keep in mind the link layer framing that is used as well. ATM (what most of the internet backbones use) has 53 byte fixed cells. There are 5 bytes of headers. Only one packet can exist in a group of cells (a group is 1 or more cell until the whole packet is sent). This means that if you try to send a 80 byte IP packet over an ATM link it will be chopped into 2 ATM cells with 16 bytes of padding. This is only 83% efficient. By adjusing your sample size you may find that your throughput can be increased because you transfer more useful data and less padding. Make your sample size too small and you have a lot of IP overhead, make it too large and it can cause problems with call quality (think of a 30ms jitter buffer and 30ms sample sizes, in effect you have no jitter buffer because packets cant be reordered, jitter cant be controlled, etc). Its a fine balance, but something to consider. Even if you dont use ATM the internet backbones often do, so this is something that may make slightly faster transfers and better network efficiency. DSL, E1, T1, SMDS, OC1, OC3, OC12 etc generally all use ATM, so its quite common.

Real life measurements

I have set up IAX2 trunking between Paris and Reunion Island (where bandwith is expensive) and trunking 15 channels of g.729 calls shows an average of 12.5 kbps / call (using ntop to measure inbound + outbound bandwith usage). While not as good as the theoretical figures, trunking on g729 just about halves bandwith requirements (you need 25kbps / call with SIP+g729) with no noticeable loss of quality. Jean-Michel Hiver

JT


See also

Created by: oej, Last modification: Mon 15 of Sep, 2008 (16:32 UTC) by AlexandreBourget
Please update this page with new information, just login and click on the "Edit" or "Discussion" tab. Get a free login here: Register Thanks! - Find us on Google+