2 Background and Related Work

2.1 Application Context

Adaptation starts at the application’s flexibility to carry on its useful task, for example meaningful communication in a multimedia conference, in different grades of service. The network is assumed able to support the application’s highest demand at light load. One or more encoders may define the different grades of service by using different compression rates. If the application or middleware is able to switch between layers at any time, extra information needs to be maintained, either encapsulated in an RTP-type [Rtp96] packet or in a separate packet (even stream) indexed or synchronized to the data packet (or stream). This is because the application needs to be aware of the received stream characteristics. This required extra information introduces overhead. In an end-to-end architecture the applications ability to switch between different rates has to be combined with monitoring and quantifying the underlying network conditions. In a network feedback architecture these are provided by the network. Since the codec belongs to the application layer a significant part of the adaptation has to belong there, according to the RTP paradigm.

Multimedia applications are sensitive to lost packets, delayed packets and jitter. RTP defines how loss and jitter should be estimated. Their monitoring is performed along the end-to-end path and a feedback packet informs the server periodically. The server uses this past interval to adjust its future sending rate. In essence, the underlying assumption is that the near future network response is anticipated to be similar to that experienced in the near past. QoS information is therefore very time-sensitive. Another realization related to the feedback path delay is that, sadly, when we need the QoS information the most –that is when the network conditions are highly adverse- it is exactly when they are usually received late, errored or lost altogether (proportional to how symmetric are the links).

By application context we refer to an ILP architecture that implements at least an RTP thin transport layer, a presentation layer and an application layer suitable for energy efficient, mobile clients. The target applications for this work can be categorized as follows:

· According to liveliness

Live vs Playback - A source may transmit a stream as it is captured in a live session. Or otherwise a pre-recorded, pre-encoded stored stream is played back from secondary storage in a playback session. The main distinction between live and playback session is that in a live session the future data is not available whereas in a playback all the future data is available at the beginning of the session. Thus the stream can be transmitted arbitrarily faster (or slower) than its consumption rate. Data is still slightly delayed and buffered at the source before transmission to allow for a small transmission rate differences. In both cases data has to be buffered at the client side in order to compensate for the jitter in delivery times. Clearly a live session places more stringent requirements in the buffer size used because use of larger buffers limits interactivity due to the proportionally larger playback delays. Smoothing is therefore limited too in live applications. Adaptive encoding approaches can be used to smooth the bandwidth requirements of the encoding stream, for example [NgK97]

· According to content:

Audio coding / Video coding - Audio codecs usually require less bandwidth than video. Video codecs employ compression in two levels. On an image level and on a frame level. Image compression is employed to encode the image of one frame. In order to compress further usually a video codec will transmit one compressed full significant frame and a few of the next frames will be encoded only as differences from the significant frame. This implies that, an audio application can usually transmit at a constant average bit rate in a much smaller time scale than a video source. It also implies that in video codecs packets do not have a uniform QoS significance. Furthermore, most audio codecs have been designed for synchronous channels, for example the GSM codec at 13Kbps. In/tolerance to lost packets is an important attribute to audio codecs. Object coding attempts to encode audio and video (M-peg 4) by objects. A speech application is a constant bit rate with silence, on/off intervals. The Brady model is widely used for voice traffic modeling. A hypermedia application is one that combines audio, video with text and images. In this dissertation we extensively deal with Mpeg audio and video (H.263).

By introducing adaptivity to the application layer, demands placed on the network can be adjusted within the session, and user perception may be enhanced. Application models that support adaptivity have been proposed in [McI98], [Cha97], [Sis97]. An application model suitable for wireless ad-hoc networks can be found in [Kaz99]. Architectural considerations are found in [Cla90], [Ott98], [Boc96].

2.1.1 Adaptability in Multimedia Streaming

Figure 2.1. The two functions of multimedia adaptation

We conceptually classify the functions of multimedia adaptation in two categories: (i) the network transport operations i.e. the network monitor that implements the measurements and congestion control mechanisms and (ii) the presentation operations that constrained by the network resources as reported from (i) direct the coding operations. The challenge in (i) is to perform efficient, stable, distributed and accurate congestion control while the challenge in (ii) is to trade-off quality and error control for the particular content and codec capabilities.

Live and playback multimedia applications are likely to be developed over any one of the network configuration under study, independently of whether these will provide quality of service or not. An example is the Internet, where shortly after the modems reached the minimum possible bandwidth a significant number of solutions were instantly provided and deployed. Adaptive streaming over IP networks has been the focus of research for this reason.

In this section we introduce an application architecture framework that will allow us to more effectively study the application context functions.

· Session Initiation: This has special importance for adaptive multimedia application protocols, since it may include QoS negotiation and call admission as well as initial application buffering. We include both initial buffering and re-buffering in the session initiation since it is common that many protocol parameters are decided during this phase.

Figure 2.2 Application Context

· Buffering

· Server Buffering, used for source smoothing. In the server side of a live session to provide small scale access delay difference allowance between the content capture, the encoding, and the transfer. By increasing its size the application may trade off send-out delay (a portion of end to end delay) with flexibility in applying techniques such as the ones described here (packetizing, scheduling, error control etc). Buffering for transmission in the playback case is trivial to these concerns. Buffering for transmission is mainly used for source smoothing -presenting an equalized stream to the network instead of the bursty codec output. One may categorize the bulk of work on video server architectures to optimize accesses on video on demand issues here.

Client Buffering, is important in both playback and live applications as a jitter compensator. In live, however, the size of the buffer limits the liveliness of the application. For this reason a buffer overflow event may be handled differently. Both types of applications, however, may do dynamic buffering allocation. Before a buffer overflow the client might somehow denote that to the server through a transfer control mechanism. Buffer overflow effects may be partially alleviated by selective dropping. Buffering underflows result in direct interruptions on the perceived stream. Furthermore, they are cause for de-synchronization in layered approaches [Rej00]. For these they are highly undesirable. When they occur the application decides whether a re-buffering will be triggered or not.
Presentation Encoding: A multimedia application suited for a network with wireless links must be adaptive. There are several ways in which the source may be encoded so that quality adaptation of a stream is achieved:

· multiple encoded versions: in this approach the server keeps different versions of each stream with different qualities. As the available bandwidth changes the server switches encoder and/or encoder parameters in order to lower an average rate. The server usually needs a large space per stream to keep all the encoded versions.

Hierarchical encoding: with hierarchically encoding the stream, the server can separate a stream into layers. The server chooses to send more or less layers depending on bandwidth fluctuations. Layered approaches (e.g. [Sis98]) usually have the decoding constraint that a particular enhancement layer can only be decoded, if all the lower quality layers have been received. They are however, more suitable for caching by a proxy for heterogeneous clients [Rej99]. They require less storage at the server side and provide an opportunity for selective retransmission of the more important information.
Dynamic coding: the server can dynamically request from the codec to present the content within a specified rate limit. For example in Wavelet video coding a specific number of co-efficients of the wavelet transform summands may be returned. The wavelet coding may be used for selective packet dropping techniques in active networks configurations (or where the routers may check the packet header to decide dropping packets)

· Multi-resolutioned streams: when from the higher quality stream the lower may be extracted. Note that in this case, the client then has the extra choice to do the layer drop on the fly without waiting for an RTT. Assuming the errors are not evenly distributed the probability of producing a more meaningful communication with dropping the quality is higher.

Stream selection or add/drop mechanism: the main challenge that maximizes overall (congestion control) and individual (presentation quality) delivered quality while minimizing thrashing (frequent layer switches).

It is generally desirable to drop layers before they reach the client. In this way network effective utilization is increased. The can be done by en-route drop mechanisms. The application may tag the packets more suitable for dropping. Alternatively, transcoding may occur inside the network by transcoding proxies. This however, has the additional disadvantage that the re-encoding will carry impairments not only from the transcoding procedure itself, but from inclusion of network noise in the process. A transcoding mechanism for mpeg audio can be found in [Nak98]. We do not deal with en-route drop mechanisms in this thesis.

· Error detection and correction:

· One form of content dependent error detection and correction is integrated in the codec (media dependent). For example codecs compress less some data considered of increased perceptual importance, a basic example is the B and I frame distinction. One usual encoding trade-off is error resilience and compression ratio. Introducing temporal dependencies can enhance efficiency of compression. This however, results in a single error affecting the quality of the stream for a longer period of time. By using temporally independent coded blocks, packet loss effect is limited. From this discussion it becomes clear that different encoding techniques will react differently (deliver different QoS to the user) once applied the same error pattern. This in turn suggests that there are error patterns that favor one compression technique against another. By incorporating knowledge and/or prediction of error patterns, an application can either choose a more suitable encoder or, if possible, alter the source to fit a desirable error pattern (e.g. by predicting bursty interval of errors one may properly interleave packets in an MPEG source to achieve uniform in time error distribution at the client side [Yao97]. A discussion on internet packet loss and implications on end-to-end QoS can be found in [Bor98]. Since the QoS delivered to the user does not decrease similarly with different codecs and it is a highly subjective value we use two techniques in this work. One, develop hybrid simulation tools that allow viewing of the actual video delivered through large simulated networks (perceptual evaluation). Two, in order to quantify and evaluate our results we use an established QoS framework that in turn quantifies perceptual degradation as a function of loss [For98]

In absence of a standardized multimedia transport, applications perform error prevention by sending redundantly important packets, and error detection and correction by either employing FEC or ARQ or hybrid techniques outside the codec in separate packets or payload part (media independent FEC). FEC needs more bandwidth and ARQ introduces more delay often unsuitable for real time applications. Media independent FEC is less complex than media dependent and is a more suitable candidate for mobile battery limited devices even though media dependent FEC can exploit knowledge of the media compression scheme to improve efficiency of the recovery mechanism. In media independent schemes, recovery has the additional possible advantage of applying integrated error and congestion control. The idea is that since error recovery trades off bandwidth, an increase in loss rate should trigger a real-time decision of the application either to send the next packet or retransmit the same. In a FEC context, use the bandwidth for increased parity or for upcoming packets. This is the tradeoff between error control and quality adaptation [Pod99]. Furthermore, rrror control at the application context complements transport (UDP) error control. UDP is a natural choice for multimedia applications but it has been designed for wired networks. Especially for wireless network a transport needs not only to be adjustable, but adaptive as well. UDP-lite is a suitable underlying transport for this type of interaction. In both Bluetooth and 802.11 UDP-lite cannot be used due to the required medium access CRC. In this work we generally settle for using an established QoS framework as our focus is not to develop retransmission and error control schemes.
Data transfer regulation:

Congestion Control - Fitting the codec bursty transmission rate to the available bandwidth is a difficult task considering that even the available bandwidth is not known accurately and timely.
End-to-end feedback: it carries information from a monitoring process to the other end. Any type of desired information can be carried since its propagation is the result of an end-to-end protocol. For example, on an internet based audio server it can contain queued time to be used in bandwidth balancing or enforcing some fairness among its clients. The source to destination reverse path carries the feedback overhead. A serious disadvantage is the known reverse path problems (known also from TCP), for example delayed, lost, obsolete feedback, instability in the closed loop congestion control scheme, etc. RTP is the paradigm that uses an estimation of loss and jitter as end-to-end feedback.
Network Feedback: It can be either implicit or explicit. When explicit, it is the result of a pro-active network level propagation and is readily available to the application at QoS negotiation. Implicit feedback signals do not consume extra bandwidth but their interpretation requires homogeneity and knowledge of the underlying mechanisms. The desirable network feedback in QoS related work, e.g. routing, is the available bandwidth and the delay. In the case of network feedback the application has to support an network API through which it may communicate the QoS quantifications with the network.
Moving from/to application buffers to/from the network: The application may packetize according to codec suggestions (block sizes, frame boundaries etc.) and network suggestions (MDUs). It may also schedule the packets to change a predicted channel error distribution to a codec-friendly error distribution or a small time scale network response. Live applications are more limited in such approaches, as the size of the sending buffer constitutes a part of the end to end delay.
Multiplexing/Synchronization – Timestamping – Framing: Headers must support these functions but at the same time the header overhead bits should be limited. Different techniques of header compression exist that are used but are not of direct interest to this work.

Figure 2.3: General Server Architecture

Figure 2.4:General Client Architecture

2.2 Feedback

Feedback mechanisms provide the application context with the information necessary to perform adjustments to the available bandwidth. In [Fab98] feedback mechanisms are discussed under the umbrella of active networking. We describe the feedback mechanisms mostly relevant to our proposed study and mention others. The taxonomy is based on network or end-to-end and implicit or explicit feedback.

(i) Explicit network feedback: This feedback is the result of a proactive explicit propagation at the network layer. Its purpose is to reflect a given parameter (like delay or available bandwidth). A server may periodically request from the network information on available bandwidth or delay or some combination of the two on the path to his client. Since close-loop behavior is dependent on the path delay network feedback has the significant advantage of lower delay. The information is ready from the network and assuming sufficient convergence fairly accurate. However it imposes work on the network layer. This work may not be entirely extra overhead if it is combined with network routing for example. A link state routing algorithm for networks with wireless links, especially multi-hop, usually results in consuming a significant amount of the bandwidth due to frequent updates required by mobility support. As a network layer procedure it may produce information based on the segregation of flows rather than a specific flow. This is dependent on flow identification at the switches. An explicit feedback uses a specified part of the payload to convey information to the feedback receiver (e.g. DECbit). If it is binary then in effect it hides the quantitative significance from the application (congestion bit). It can be seen as a decision taken for the feedback receiving application.

(ii) Implicit network feedback: An implicit feedback is one that is obtained by monitoring the status of the data transmission. This feedback is the result of an event that implicitly propagates to the feedback recipient and captured by his monitor process. Since close-loop behavior is dependent on the path delay this feedback has the advantage of lower delay over end-to-end feedback. Since the event has to be captured by the monitoring process we may conclude that it generally has a larger delay than an explicit network feedback. This is partly true however, since the explicit network feedback is hidden by its pro-active propagation, and its availability does not imply updated information. Since the application monitors for it in its flow it can be flow related without the network having to distinguish flows. A well-studied implicit feedback is packet loss in TCP slow start. Another can be delay dispersion of packets, which packet-pair uses to evaluate max network rate, or the observed throughput in NETBLT. Reacting on implicit feedback should follow correct interpretation of the signal, which may be relevant to network implementation. For example one may correctly react on a packet drop when one knows when a packet drop may occur on the network (and this is done uniformly inside the network). For another example, observing the rate at which packets emerge from the network assumes an averaging interval adequate to reflect a sufficient value. Sufficient is depended in turn into network mechanisms like packet scheduling and application specifics like application re-action. As mentioned before however closed loop congestion control is depended on delay. Implicit feedback can be very misleading in heterogeneous networks, when sub-networks in the end-to-end path have different strategies. Due to mistrust in feedback signals applications usually adopt conservative approaches in adapting their behavior to it. An advantage of the implicit feedback is that it does not require any overhead resources from the network or the switches.

(iii) Explicit End-To-End Feedback: This feedback becomes necessary when the monitoring has to be done away from the feedback recipient, and the end-to-end connection is used to carry it back. It is the result of a defined end-to-end known protocol. In source quality adaptation for example, the client monitors and measures the RTP statistics and sends this information periodically back to the server. Values, usage and period can be application specific. It suffers most importantly from larger end-to-end delays and delay variances. It also suffers from feedback implosion when the server serves multiple clients - not necessarily multicast (problem applies to both). However, it is more flexible in that it carries a value that may be specific to the application (for example a video server may decide to learn the buffered milliseconds of his on-demand clients to decide how to apportion his resources)

2.3 Transport and Congestion Control

Today’s Internet is dominated by TCP. Its design depends on the sources to perform flow control via end-to-end measurements. In this way, routers can forward packets without knowledge of specific flows. Internet owes its scalability on this. Furthermore, according to TCP end-to-end design paradigm, minimum dependence on lower layers is allowed. Internet protocols owe their easy deployment and inter-operability on this. Transport protocols rely on implicit rather than explicit network-based congestion indicators.

In general, multimedia cannot use TCP. Even if reliability (ARQ and re-ordering) functionality is removed from TCP, it still does not present a good candidate. The trial-and-error approach converges slowly and requires many attempts. On the other hand, multimedia traffic prefers an approach that would from start operate on a fairly good estimate of the available bandwidth, incurs minimum, perceptually costly, attempts to improve quality, and adjusts smoother to network changes. Furthermore, window based techniques impose unnecessary delays and high jitter. Real time traffic should ideally be serviced (transmitted) as soon as generated by the application, or packets may not reach the destination by their playback time. Live applications are particularly intolerant of delays, especially transport delays that are always in the critical path.

In this thesis, in our attempt to improve existing end-to-end based techniques, we develop and design a multimedia transport protocol based on a new path available bandwidth measurement, more accurate than existing and suitable for heterogeneous and wireless networks. Available bandwidth is a particularly interesting quantity for transports. If a protocol knows its path available bandwidth it can accordingly adjust its sending rate. At the same time it is hard to measure available bandwidth, at least when a network model that is close to Internet reality is required [Lai99]. An active heuristic technique called cprobe [Car97] is widely used to measure available bandwidth. Passive measurements of available bandwidth using dispersion techniques are also seen in transports. We indicatively refer to the sender based packet pair equivalent in the faster recovery mechanism proposed in [Cas00], and Microsoft’s MSTFP [Zha98]. Packet pair may measure the bottleneck link bandwidth in FIFO queuing networks by using for example the filter proposed in [Lai01]. Its result is the available bandwidth only in WFQ networks [Kes94]. Therefore, those measurements are only as accurate as the weighted fair queuing model approximates the Internet. It is conceivable, however, that in low hop connections the traffic at the bottleneck or minimum available bandwidth link statistically approximates the competing traffic from a fair queue (for example, think of a single link connection accepting traffic from many flows). In [Dov01] it is shown that such techniques converge to the Asymptotic Dispersion Rate that is lower than the capacity but is not the available bandwidth in the path. Our work is in agreement with this study. We note, however, that the classic bytes-over-dispersion model assumed in previous work can be significantly improved. The packet pairs and train still suffer cross traffic related problems.

The design of the protocol is motivated by the fundamental question of whether end-to-end stability, fairness, and TCP friendliness can be supported with a pure passive measurement technique, rather than observed loss. We say pure passive as opposed to semi-passive as for example used in [Mas01]. A transport protocol in general may not use active measurements due to the overhead imposed on the network, but it may perform what we call semi-passive measurements. It does so, by using existing packets from the application but shaping them appropriately. Unfortunately, a real time transport has limited shaping flexibility due to delay constraints. PLM ([Mas01]), for example, pairs up all application data to packet pairs in order to increase their potential bandwidth ([Lai99]). Our requirement to handle bursty VBR, delay and delay-jitter sensitive traffic does not allow us to do this. Consider also that bursts usually represent significant frames in video, and pairing increases loss probability as well.

As mentioned above, inter-operability with TCP, or TCP-friendliness, is important. It is defined as having throughput proportional to TCP’s throughput over sufficiently small time scales. Note, there is no requirement that similar techniques (measurements), should be employed. However, TCP-friendly mechanisms typically observe the loss. We have noticed two types of achieving TCP friendliness a) by using simplified steady state TCP throughput models [Flo99] or b) by actually calculating TCP parameters along with protocol parameters (i.e. TCP emulation) [Ses97]. Recent work shows that the class of binomial algorithms [Ban01] with increase-decrease factors summing to 1 and synchronized feedback is stable and TCP friendly. We consider this for rate based rather than window based protocols and use it for developing our pab-based TCP-friendly mechanism.

Transports in general, may tradeoff accuracy for fast converging measurements and filtering. Window based protocols may apply packet pair techniques more efficiently than rate based. Packet pair can only measure bottleneck links that have bandwidths less than the sender bandwidth of the pair [Lai99]. This means that the packets should be large enough, and their separation in time small enough to cause queuing at the bottleneck. In principle they should be sent back-to-back. Window-based protocols naturally send packets with high sender bandwidths (although application interactivity is a limiting factor).

A rather sophisticated filtering mechanism uses packet pair for bottleneck link measurement in the tool nettimer [Lai01]. This work has progressed from [Lai99] and has also produced [Laib00]. The latter is a technique that uses a train of packets to measure multiple bandwidths along a path. Nettimer, uses a kernel density estimator in order to statistically compute the bottleneck link bandwidth from a collection of samples. It finds the highest density point by effectively dealing with well-known problems arising from constant bin size histograms. The basic assumption under which this estimation is correct, is that more packet pairs end up exhibiting the bottleneck link bandwidth than any other point. While this is mostly true, since the bottleneck link bandwidth is a sustaining quantity, it requires a large number of good samples. More importantly, an end-to-end transport is not directly interested in bottleneck link bandwidth, but available bandwidth. We use this bottleneck link bandwidth measurement in aid of an available bandwidth estimate.

When a packet dispersion mechanism is used for available bandwidth estimation, the filtering has an averaging effect [Car97], rather than discovering modes [Dov01] or densities [Lai99]. The samples are typically computed using the simple bytes-over-dispersion formula at the observer side. However, we find that this simple model may be significantly improved, especially in presence of intense cross traffic. We have developed a model that is somewhat more complicated than the simple division, but produces samples that are closer to the actual available bandwidth. Providing tangible proof for that is especially difficult, since the actual available bandwidth at routers changes frequently and cannot be known to observers. However, we provide equations that we believe prove our point. The sampling is naturally distorted by errors in bottleneck link bandwidth estimation as well as known packet pair distorting effects.

In order to use our measurement passively in a multimedia transport, we develop an appropriate filtering technique based on the confidence of the sample and digital filter theory. We analyze events in the source traffic and overlapping packet pair sequence to evaluate our confidence on the sample. The confidence appropriately weights a digitized low pass filter, by effectively ranging its cut-off frequency between two values. The filter is a low pass, Tustin estimator with the inter-sample variance anti-distortion found in [Pou00].

After this introduction we are ready to set the requirements that direct the design decisions for our measurement based multimedia transport protocol (MMTP)

(1) MMTP should not rely on specific services by other layers (i.e. the network does not offer any explicit congestion feedback, proxy help cannot be assumed etc.). It should not violate the end-to-end design principle (i.e. wrongly make use of split connection etc.)

(2) MMTP should have good performance in networks with wired and/or wireless (mobile and lossy) links.

(3) MMTP should have good performance in networks with different (high or low) delay-bandwidth products.

(4) MMTP should incur minimum overhead to the network in terms of packets.

(5) MMTP should support both live and streaming media and therefore should incur minimal delay overhead. Ideally it should service the packets as soon as they are ready from the application.

(6) MMTP should be designed so that it is stable, intra-protocol fair and TCP friendly. Friendliness to other protocols should also be possible.

(7) MMTP should support a lightweight implementation especially at the receiver side.

(8) MMTP may be placed in the application context (or equivalently assume some limited application level interaction). It generally follows the RTP architecture paradigm. It should work well independent of specific codec used.

Requirements (1), (7) and (8) describe the general design space. Requirements (2) and (3) are the ones that drive the important decision, to rely on measurements, since ‘trial-and-error’ techniques do not suffice in convergence speed. Fast and accurate measurement of available bandwidth would therefore be a key to potential MMTP success. This in turn drives to the following decisions:

- Apply passive measurement techniques to the VBR flow that obtain as accurate measurements as possible as fast as possible

- Apply efficient, effective, control theory supported filtering techniques.

- Perform measurements on one-way rather than round trip, but measure both directions (mainly for TCP-friendliness, req. (6)). This reduces the errors in the measurements in half, makes the errors more detectable and provides better results much faster (see measurement tables in [Lai01]).

- Requirements (4) and (5) may be considered conflicting. The former disallows active measurement while the latter limits our shaping of existing traffic. We will not incur any additional delays by shaping our traffic. We accept the traffic as it is shaped from any video smoother etc.

- For stability and TCP friendliness feedback should be per RTT (per packet) [Ban00].

Furthermore, adaptation and call admission have often been studied in parallel because of their close relation. Call admission in wireless networks can only be handled through adaptive contracts. A quantity of interest for these functions is available bandwidth [Lai99], [Dov01]. Unfortunately, it is very difficult to measure and filter using end-to-end techniques even in wired networks, because the observed samples follow a multi-modal distribution [Car96]. Techniques for measuring related quantities, like bottleneck link bandwidth are more feasible [Car97], [Lai00]. For 802.11 networks the situation is more difficult since more transient parameters affect the measurements. This is due to mobility, random loss, virtual carrier sense timers and unique neighborhood head-of-line problems.

In other transport related work [Ban01] generalizes the TCP AIMD control algorithm to include non-linear algorithms, i.e. ones that the amount of window increase is a faction of the window itself. This work is important to multimedia because it shows that any increase/decrease algorithm (e.g. SQRT) with certain characteristics is stable, efficient and TCP friendly, according to a well known set of assumptions [Chiu-Jain].

2.4 Networks

Wireless and embedded computing technologies are expected to become a basic building block in future intra and inter-networks. Two basic trends can be distinguished. An increasing number of small computing and communication devices are being equipped with wireless capability and ad-hoc protocols (see [Haa98], [Haa99] on Bluetooth, [Neg98] on HomeRF, [Est99] on sensor networks). These devices have typically short range due to power constraints but support the bandwidth needed for voice communication. On the other hand, higher bandwidth (1 to 10 Mbps) wireless LAN NICs are provided for laptops and palmtops as the moving client node of the network. Application level integration for these two devices promises an integration of services as well.

Ad hoc multi-hop wireless networks are self-organizing, self-configuring, instantly deployable in response to application needs and independent of a fixed infrastructure existence. As such, they are very attractive to multimedia applications in disaster recovery situations (flood, fire, earthquakes etc), law enforcement (crowd control, border patrol etc), search and rescue in remote areas, sport events, festivals, ad hoc nomadic, collaborative computing, indoor network appliances, and battlefield [Tsa95]. As such, it would be desirable to allow nodes in such networks IP access. Adaptive multimedia for multi-hop networks is discussed in [Alw96].

Both of these types can be used for providing IP based services on a multimedia capable device. As such, it is a natural desire to use them as clients for internet or intranet multimedia services. We choose as target an IP based network, which will be used to deliver multimedia to last hop (or hops) wireless network organized either with access points or in ad-hoc configurations.

2.5 Performance Evaluation

2.5.1 Platforms

The concept of using a hybrid simulation platform for evaluating speech in large wireless networks has been introduced during the WAMIS project at [Hwu98]. The hybrid simulation platform introduced and used here is a new platform built from scratch. It is, however, based on the concept introduced in [Hwu98], [Bag91], [Liu94] and has been extended to support adaptive application layer techniques over the new GlomoSim platform. It enables us to run large scale adaptive experiments using the large set of options for the lower layers supported by GlomoSim.

Detailed simulation models of complex heterogeneous systems can be extremely resource intensive to develop. An alternative is to use a hybrid model i.e., one where some components exist as simulation or analytic models and others as operational subsystems realized in hardware or in software [Bag91], [Liu94]. Use of hybrid models allows an analyst to ascertain the impact of design changes in one subsystem without developing detailed simulation models of the entire system; instead certain system modules can be directly replaced by simulation models of alternative designs, considerably reducing the modeling overheads. The hybrid simulation platform described is designed specifically to support adaptive multimedia experiments.

On the input side of our hybrid simulator we have the ability to feed into a simulated subsystem, packet traces produced in an external system. Since the interface to the simulator is the trace file format, the external system can be just about anything that can produce those traces. The traces may come directly from a codec or a streaming application. Alternatively the traces may be captured after travelling a real network subsystem or a different simulation platform subsystem. On the output side we may feed the traces on a next real subsystem. In the case of application layer adaptation to differently encoded streams the process may be significantly simplified by pre-capturing the traces and choosing which to use at run-time based on the adaptation mechanism. Under development, is an application that by reading the output traces and plays them out in order to perform end user perception evaluation.

2.5.2 Accuracy of Measurement versus QoS

In this section we model the impact of adaptation to QoS by describing a simple model and running computer simulations to obtain numerical results. The purpose is to show how the accuracy of a measurement can impact the connection and network QoS.

Assume that the time is divided in periods by the feedback arrival times. In every period a connection received a feedback of a quantity of interest such as available bandwidth (AB). Say, the actual available bandwidth is AB and the measured a*AB. The connection will perform a function Choose(a*AB) in order to decide how many bits of content it can fit in the channel during the next period. For simplicity we will assume that all rates are available. The bits sent will be received at the client side if the bandwidth was indeed available. We assume that in the steady state, i.e. when no flows enter any part of the path and no route changes occur during the current period, the AB does not change. This means that we are dealing with congestion rather than wireless random packet loss. Our goal is to provide the available bandwidth to the application based on which the application will decide how to use it. Therefore in the steady state the sending throughput ST, the client throughput ST and the QoS for period I are:

Now assume that a flow that shares or contests with part of our path is leaving. It was sending Choose(ABa’) assuming the same Choose() function, therefore the quantities in this case become:

When a flow enters, we similarly have

When a route change occurs assuming a blackout handoff loss L, and uniformly distributed and independent new route available bandwidth we get:

Now assume we can calculate the probability of route change, entering flow, leaving flow and steady state by observing the route and noting the event that takes place in the above order (mutually exclusive events). Then the overall QoS for one connection would be

Let us now use a QoS model behaving like the one before but normalized to 1. We also use a simple Choose() function where Choose(AB)=C*AB

We have expressed the QoS of one connection in relation to the link bandwidth, the handoff blackout loss, the average Available Bandwidth, the probabilities of the state of the system and parameters from the probability distribution function of the accuracy of measurements. Note the state probabilities are dependent on the feedback cycle d and can be usually expressed as Poisson with rate d*t. The model can be extended using this in order to associate actual mobility and other network parameters using the capacity models in [Gup00]. However, here we are interested in exploring the accuracy of measurement/QoS trade off for the congestion case. The accuracies of measurement a_i s can be modeled simply using a single mode distribution function as for example, Gamma. Note that the product AB*a has been shown to follow multi-modal distribution using packet dispersion techniques in [Dov01] and not the accuracy variable.

As an intuitive example we now attempt to apply numbers to the above equation and draw the result. For simplicity we assume that the accuracies are normally distributed around 1 (=accurate measurement) and look at the QoS while shifting their mean and variance. Since we are interested to test in middle to high load scenarios let us use: P(no change)=0.6;P(entering flow)=0.15;P(leaving flow)=0.15; P(route change)=0.1; for bandwidth of 1Mbps, C of 0.9, loss of bandwidth due to handoff=0.01 and average AB 250Kbps. Note that in the perfect case of a=1 we get an average QoS for the connection of C(1-AB/B) for B/AB-1 connections (.675 for 3 connections)

Figure 2.5 Mean of Accuracy Normal(mean, 0.1) versus QoS

Figure 2.6. Accuracy Variance versus QoS

2.5.3 Metrics

In our study we wish to maximize the overall QoS provided by the network as well as individual connections QoS. The term QoS however is a very broad term. Besides developing experimental infrastructure that allows for perceptual evaluation we need to define our metrics of evaluation of the large scale simulation experiments.

The loss rate of the connection gives an incomplete idea of the QoS provided to that connection. We are interested in both aggregate loss rates or RTP-based loss rates. The former is the division of the packets sent by the packets played out (not received) during the time of the connection. The latter averages the loss rates in time windows and then reports the average of those. There are differences in the above two methods, but they are both reporting higher loss rates than simply considering sent and received packets.

The Server Throughput is another metric of interest. It shows the effect of the adaptation to the server throughput. The Client received Throughput is the successfully received and played out throughput.

For the issue of fairness whether inter or intra protocol we use the co-efficient of variation.

Assume a situation where the content can be sent in N encoding rates of linear degradation { L(n), L(n-1), … , Lo } and that the connection receives a bit rate of R bps that belongs to a grade of service Li bps. Let us see how an application adapts. Assume the actual available bandwidth, averaged over the desirable time i.e. the feedback cycle, is AB and the estimation we get has a relative error e. On each cycle k, we get an estimate (1+/-e) * AB_k . Based on this we decide what to send in cycle k+1. For the moment, assume we may fill up the available bandwidth, and therefore the application always selects the maximal layer that fits in the available bandwidth estimate ( Li<(1+/-e) AB_k ) . The possible scenarios are:

(i) AB_k+1 >= (1+/-e) AB_k

a. All Li < (1+/-e) AB_k < AB_k+1

b. There is Li < (1+/-e) AB_k < AB_k+1

i. In this case we may be sending less that we should so that QoS is maximized. We are sending AB_k+1 - (1+/-e) AB_kless at most depending on layer granularity.

c. There is no Li < (1+/-e) AB_k < AB_k+1 : this is just here for completeness as it is considered a call admission issue.

(ii) AB_k+1 < (1+/-e) AB_k

a. All Li < (1+/-e) AB_k < AB_k+1

In this case we are over-sending (1+/-e) AB_k- AB_k+1 , and therefore either experience a loss rate or cause loss rate to some other connection.

b. There is Li < (1+/-e) AB_k > AB_k+1

i. Li > AB_k+1 : in this case we are over-sending as before.

ii. Li < AB_k+1 :

From the above simple analysis it is evident that in order to maximize the QoS from equations (I) and (II) the cases 1b, 2a and 2a(i) should be avoided. There are types of error that lead to these cases. The inevitable errors and the measurement errors. The former is due to the non-correlation of AB_k and AB_k+1 due to mobility, flow changes, fading and other variant conditions. The latter, is due to measurement method inaccuracy and noise and it is what we are attempting to improve on.

The ultimate measure of performance in multimedia communications is the QoS delivered to the user. Due to the prohibitive cost and time to have human objects evaluate QoS, there has been extensive research in developing suboptimal quantitative methods, especially in the coding research arena. Some of these measures are mathematical or perceptual, objective or subjective. In many applications, the Mean Square Error of a delivered image is expressed in terms of a signal-to-noise ratio (SNR), which is defined in decibels (dB). Another definition of the SNR, called the PSNR, is used commonly in image coding applications. In this case, the MSE is reported to the peak-to-peak value of the reference image. The PSNR is one of the most commonly used metrics. This mathematical measure is computed for each frame of the video sequence, but does not take into account at any level the human visual system comportment. Many studies have shown that this metric is poorly correlated with human perception since it does not take visual masking into consideration. In other words, every errored pixel contributes to a decrease in PSNR even if the error cannot be perceived. Recent research has therefore addressed the issue of video quality assessment by means of metrics based on the properties of the human visual system. The signal to weighted noise ratio, WSNR, is very close to the PSNR but makes possible to take into account some human visual system properties through the weighting functions such as range of the luminance, weighted noise power density and others defined by the ITU-R 21 451-2 recommendation as a function of the eye sensitivity. Another measure has been proposed by the ITS called SHAT. It combines spatio-temporal filtering of images with factors optimized through subjective tests.

In this study we use the MPQM metric [Van96]. The model is based on the properties of human vision. It also considers visual masking techniques to effectively take into account error concealment techniques in the QoS result and targets video, moving picture quality rather than image quality. It has been shown to behave consistently with human judgments [Van96]. Its quality rating is scaled from 1 to 5 as described in [Ard94]. The quality being Excellent (=5), Good, Fair, Poor and Bad (=1), and the Impairment being Imperceptible (=5), Perceptible, Slightly Annoying, Annoying and Very Annoying (=1). [Dal96] studies the impact of losses on a video sequence. The weakness of this method is that it does not consider the encoding quality, but only the transmission degradations. Other attempts have been made to measure loss degradations, but none of them has accomplished widespread use.

In MPEG video the MQUANT parameter largely defines the rate of transmission. It has been shown that the PSNR exponentially decreases with increasing MQUANT. The MPQM metric exhibits a linear relationship with MQUANT [Fro98]. The same characteristic has been observed through user’s subjective evaluation [Fuk97]. The perceptual quality saturates at high bit rates. Increasing the bit rate may thus result, at some point, in a waste of bandwidth since the end-user does not perceive an improvement in quality after a certain bit rate. The average bit rate after which the quality does not significantly increase may be shifted to the left by means, for instance, of an adaptive quantization scheme [Ver96]. Therefore, the same perceptual quality may be reached at a lower average bit rate. For lower bit rates the function can be approximated by a linear function. The actual values for the parameters in the above equation have to do with the encoding complexity of the scheme. In our study we want to push the QoS evaluation so we will develop a model that corresponds to high encoding complexity.

The next issue that is very important in our evaluation is how QoS is affected by loss. It is shown [Fro98] that, on a semi-logarithmic scale and for a given MQUANT (average bit rate), first the video quality remains constant with the PLR (packet loss rate). Beyond that, perceptual quality quickly drops. The higher the MQUANT value, the lower the rate and the higher the point after which the quality drops. Also, the lower the packet loss rate, the lower the number of lost packets per frame on average. Therefore, the higher the MQUANT, the higher the PLR for an equivalent perceived degradation.

In wireless the bit rates are pushed to the lower limits allowed by encoding techniques and high error control (e.g. FEC) is employed. In general for wireless we have low rates, higher loss, increased error control vs quality and QE becomes linear to rate for low rates, rrror Control/Quality ratio decreases with rate for high error control/quality ratios and Q is proportional to a higher order PLR for high PLRs. Therefore the model we use for our QoS evaluation, for which we have picked numbers that correspond in high motion and encoding complexity video, is

For 5 Layers of 256,128,80,64,48 Kbps of high Motion, Frequent scene change clip, we have Qe’s decreasing from 5 to 3 linearly with rate and get the model depicted in the following Figure:

Figure 2.7 QoS MPQM related evaluation model used

1 Network Technology

In this section we are presenting a Bluetooth Scatternet enabling solution. The reason we deal with creating an efficient ad-hoc multi-hop Bluetooth scatternet is that we have chosen Bluetooth to be one of our work platforms and when we started the work there was no scatternet research available. We identified the problem of Inter Piconet Scheduling as central to creating an efficient Scatternet, one that may have the capacity necessary for low rate multimedia communication. Our contribution is the MaxMin Optimal Rendezvous Point Based IPS Algorithm that locally optimizes the gateway’s forwarding throughput. Very recent work has focused on random techniques of assigning rendezvous points which are simpler and scale better, however have more limited throughput. This may prove crucial for multimedia communication in these Scatternets.

The PAN is defined as the collection of devices carried by a mobile, networked individual (e.g., a professional on the move, an Internet-wise tourist, a student attending “virtual classes”, an avid Internet game player, etc). The devices include any subset of: cell phone, laptop, earphones, GPS navigator, palm pilot, beeper, portable scanner, etc. These devices form his/her PAN (a.k.a. personal “bubble”). The connectivity within the bubble is wireless (using for example a low cost, low transmit power wireless LAN such as Bluetooth). The bubble can expand and contract dynamically depending on needs and may be used for ad-hoc communication or as Internet access technology. In relation to our study it should provide efficient multimedia access from the PAN to the Internet and enable voice/data intra and inter-PAN networking. Our key challenge is to design adaptive application protocols that provide smooth transition between different bandwidth, connectivity and mobility configurations. At first, we will assume that each PAN corresponds to a single user and consists of a portable device (eg, laptop, PDA, etc.), and limit ourselves, for simplicity, to single hop communications, i.e. sources and destinations are within the same Piconet in the Bluetooth case. In this simplified, single hop setting the performance of the network will be for the most part determined by the MAC layer. Currently, there are two leading candidates for such role: (a) the IEEE 802.11 MAC protocol and (b) the Bluetooth MAC protocol [Bluet]. The IEEE 802.11 protocol is a rather sophisticated protocol that includes a fairly broad range of options. In particular, it includes the PCF (Point Coordination Function) mode which permits a “base station” to poll various terminal in a cellular-type environment. It also includes the DCF (Distributed Coordination Function) mode, which supports peer-to-peer, ad hoc type communications as well as access point infrastructures. The DCF version is a random access protocol similar to CSMA, with the addition of RTS and CTS (for collision avoidance) and of an ACK returned by the receiver after successful transmission. In our study we will assume the use of the DCF mode, which is the mode implemented in the WaveLAN cards (even for infrastructure configurations). Recently a new radio technology and MAC protocol has emerged as a strong PAN candidate. The Bluetooth MAC protocol is a major departure from the IEEE802.11 protocol. To start with, it uses Frequency Hopping with separate frequencies chosen ad-hoc, dynamically for each Piconet rather than Direct Sequence Spread Spectrum or configuration based Frequency Hopping, thus exhibiting better protection from co-channel interference. Secondly, it uses time/slot synchronization. Moreover, it uses a centralized access scheme, separating the nodes to “masters” and “slaves. Bluetooth is expected to become very popular due to its low cost potential (in the order of a few dollars per interface). It operates in the worldwide unlicensed 2.4 GHz Industrial-Scientific-Medical (ISM) frequency band, which makes it unable to co-exist with 802.11 radios. To minimize complexity and to reduce the cost of the transceiver, a simple binary Gaussian frequency shift keying modulation is adopted. In order to allow efficient wideband data transmission the bit rate is 1 Mbit/s. Two or more Bluetooth units sharing the same channel form a piconet. Within a piconet a Bluetooth unit can be either master or slave. Within each piconet there may be only one master (and there must always be one) and up to seven active slaves. Any Bluetooth unit can become a master in a piconet. Furthermore, two or more piconets can be interconnected, forming what is called a scatternet, see Figure 3.1. The connection point between two piconets consists of a Bluetooth unit that is a member of both piconets. A Bluetooth unit can simultaneously be a slave member of multiple piconets, but a master in only one, and can only transmit and receive data in one piconet at a time, so participation in multiple piconets has to be on a time division multiplex basis. The Bluetooth system provides full-duplex transmission using a slotted time division duplex (TDD) scheme where each slot is 0.625 ms long. Master-to-slave transmissions always start in an even-numbered time slot, while slave-to-master transmissions always start in an odd-numbered time slot. An even-numbered time slot and its subsequent odd-numbered time slot together are called a frame. There is no direct transmission between slaves in a Bluetooth piconet; transmission is only between a master and a slave, and vice versa. The communication within a piconet is organized such that the master polls each slave. A slave is only allowed to transmit after the master has polled it. The slave will then start its transmission in the slave-to-master time slot immediately following the packet received from the master. Each Bluetooth unit has a globally unique 48-bit IEEE 802 address. This address is permanently assigned when the unit is manufactured. In addition to this, the master of a piconet assigns a local active member address (AM ADDR) to each active member of the piconet. The AM ADDR is three bits long, is dynamically assigned and reassigned, and is unique only within a single piconet. The master uses the AM ADDR when polling a slave in a piconet. Bluetooth packets can carry either synchronous data on synchronous connection oriented (SCO) links mainly intended for voice traffic, or asynchronous data on asynchronous connection-less (ACL) links. To ensure reliable transfer of data, a fast acknowledgment and retransmission scheme is used, only for ACL links. In addition, a forward error correction (FEC) scheme may be used to further improve reliable packet transmission.

Bluetooth Scatternet enabled technology today exists only in the standards not the actual chips. Even though the standard defines the primitives to support scatterneting, the architectural options are left open to research. We use a number of proposed solutions and implemented models to explore application and transport behavior in Bluetooth scatternets. Since mobility support for Scatternets is minimal today, we only regard cases where mobility does not effectively change the network structure. We take our study one step further by including a strictly, as much as possible, MAC layer comparison between the structured Bluetooth MAC and the totally asynchronous 802.11/DCF MAC, both suggested for infrastructure-less networks. Our study suggests that in Bluetooth even though gateways effectively limit the network capacity at a fraction of the link data rate, closed loop end-to-end adaptation can be effective in controlling congestion and improving user perceived QoS. This is attributed to the very controlled master centric polling MAC layer in combination with the time invariant inter-piconet scheduling architecture we use. On the other hand, the 802.11 limited to Bluetooth characteristics for the shake of our comparison, shows significantly less effective throughput in all cases, and a more inefficient response to adaptive traffic. We studied the traffic behavior of such applications in scatternets of up to 16 piconets and 64 nodes. We conclude that simple end-to-end adaptive closed loop congestion control can be very effective in Bluetooth piconets and scatternets, and more effective than in more asynchronous unstructured MAC layers when mobility is low. The choice between a structured and an asynchronous MAC layer should be made based on how well the more efficient structured MAC can support mobility.

Bluetooth scatternets are essential to a wide range of applications ranging from single personal area networks (PAN) to larger ad hoc networking environments e.g. sensors, large events, conferences etc. The nodes in a Bluetooth network are organized in piconets, a star formation of up to 8 time-slot and frequency synchronized nodes. Larger networks are supported through the designation of nodes that may synchronize to more than one piconets, and therefore may receive and send data between them. Any node may become a gateway if it senses the presence of a foreign master node, through the INQUIRY/PAGING procedure [Bluet]. The gateway nodes then time-divide their presence in the piconets. An interesting situation occurs, where slave nodes in a piconet may exchange data at full bit rates, taking advantage of the frequency reuse from frequency hopping on different sequences, while nodes in different piconets may communicate at a fraction of this rate (and increased delays), as defined by the presence intervals of gateway nodes in the path.

At the network layer, the favorite routing algorithms are the On-Demand routing algorithms DSR [Jon96] and AODV [Per99]. The 802.11 technology is dominating the wireless LAN market today. The leading wireless LAN application is point to multipoint Internet access. In 802.11 ad-hoc mode, all the nodes can hear each other, and the interference between different conversations can become overwhelming. Moreover, it is difficult to exercise call acceptance control (on IP telephony, for example) or congestion control. In summary, today 802.11 solution provides low latency of access, no need for a priory network configuration, robustness to mobility. These characteristics are ideal for users willing to exchange quick messages among each other, or to interactively browse the web. On the negative side, there is the problem of congestion and the inability to provide efficient QoS support to real time users. Active research is currently going with the goal of providing some form of congestion control [Luo00], fairness [Tan99] and QoS support [Lin99] in 802.11 ad hoc networks. But these efforts are in the very preliminary research testbed stage and still far from commercialization. Another novel wireless LAN technology, Bluetooth, soon to be widely commercialized, can be viewed in part as the complement, and in part as the competitor of 802.11, depending on the applications. Our focus here will be the ad hoc networking environment and the RTP end-to-end adaptive multimedia applications mentioned above. Namely, we want to examine the efficacy of Bluetooth scatternets in supporting a simple adaptation policy based on loss rate for a VBR (video) stream.

In general, as more Piconets are added the overall system capacity is increased due to the ad-hoc frequency hopping mechanism, reaching capacities comparable to 802.11/DCF based ad-hoc systems. In scatternets, however, the situation may be different due the time division policy at gateway nodes. The gateway naturally operates at a fraction of the link bandwidth, and has different delay characteristics due to absence intervals. The Bluetooth Scatternet architecture we have used is described in detail next, and is based on the assignment of rendezvous points. Rendezvous point assignment is an efficient way to define the gateway presence intervals. The rendezvous point is a pre-arranged slot that a master agrees to poll a gateway node, and the gateway node agrees to accept the poll. The master decides after how many slots it will poll the gateway and lets the gateway know. The gateway then arranges to switch back right before that poll slot. In this way poll slots are not wasted. After establishment of a rendezvous point it will be periodically honored for the duration of the connection. Rendezvous points are supported in the Bluetooth specification [Bluet] by using the SNIFF primitive. The establishment of rendezvous points allows the easy development and application of inter-piconet scheduling algorithms (IPS). For example, we have devised algorithms that pick points that locally optimize the gateway forwarding throughput and avoids the creation of low throughput connections.

Most of the published work on Bluetooth so far has focused on the single piconet and issues related to scheduling of units within one piconet, e.g., [Cap01] [Das01]. Some work has also looked at the task of actual formation of Bluetooth piconets and scatternets, [Sal01], [Bas01]. In a scatternet resulting from any formation algorithm, the inter-piconet units still need to be scheduled in an efficient way. Thus, the inter-piconet scheduler (IPS) is crucial for the operation of scatternets. Few pieces of work on inter-piconet scheduling issues in scatternets can so far be found in the literature. In [Joh01] an inter-piconet scheduler is analyzed for a case where central control of the entire scatternet is assumed, which is costly to achieve. The work herein will focus on analyzing inter-piconet schedulers that operate in a distributed manner in scatternets and thus eliminate the need for scatternet-wide coordination. Recently [Rac01] has proposed a pseudo-random algorithm for IPS that is based on check-points (similar to rendezvous-point but random). In the same paper, a theoretical algorithm is described called Ideal Coordinated Scatternet Scheduling (ICSS). The pseudo random algorithm in undoubtedly advantageous is situations where scalability is important and sustained bandwidth of lesser importance, as in non-real time TCP communication. Even though we cannot test these algorithms (the simulation models used in that Ericsson paper were not available to us) our results are in accordance with them. In fact ICSS, seems to have very similar performance to our RV-maxmin optimal forwarding throughput algorithm. RV-maxmin deals with practical optimality issues by dealing with the forwarding throughput equations that allow an efficient local optimization. [Jum01] is another algorithm that recently emerged based on rendezvous points. It is based on pseudo-random rendezvous windows and focuses on minimizing RV-point collisions. Unfortunately there is no presentation of any performance results and therefore comparison with that algorithm is not possible.

Some architectures that have been proposed already ([Sal01] and [Agg00]) select the Gateways and attempt to create the links on application and routing demand, but in order to test the IPS we attempt all possible gateway connections under imposed limits of number of gateways per piconets (GP) and number of piconets per gateway (PG). We are studying the IPS performance rather than the selection of the Gateways per se. It is a more challenging problem of IPS when all the possible connections are attempted.

How should then gateways across the scatternet adjust and coordinate their presence intervals to each piconet they act as forwarders? Ideally a gateway can forward traffic at half the link bandwidth (if no traffic has the gateway as destination, it ideally spends half its time receiving traffic from one master and the other half forwarding it to the other master). However, what are the best slots for a gateway to rendezvous with each piconet master, given that each piconet has different traffic requirements, different number of slaves and different gateways present? In order to understand the impact of this decision, first consider a totally random slot selection. The presence interval may then be arbitrarily small and therefore may result in wasted slots, very low data forwarding rates and highly variant delays. A more intelligent inter-piconet scheduling (IPS) is therefore needed. Now consider a simple algorithm that plainly equally distributes the gateways presence among the piconets, as proposed in [Joh00], the maximum distance rendezvous point allocation algorithm. It ensures that the presence interval as large as it can be but the forwarding data rate may still be very low. Consider, for example, when a gateway switches during very heavy traffic from other nodes or when all other gateways are also present during the same time. This may happen e.g. when a five-node piconet has three gateways and all decide to switch in the piconet in successive slots. Then, there will be a very busy interval in which the master will have to exchange traffic with all four nodes and an underutilized interval that the master may only exchange data with the one slave. In a network that this occurs often, throughput will be low. In this paper, we are presenting a rendezvous point based IPS algorithm that coordinates so that this occurs minimally, and maximizes overall network throughput by maximizing the forwarding throughput between two adjacent piconets. It represents an improvement of approximately 30% over the maximum distance scheme, while avoiding the creation of bottleneck gateways.

Figure 3.1 A scatternet with one inter-piconet unit that divides its time between the two piconets

1.1 Bluetooth Simulation Model

We base our study on our Bluetooth model implementation on the NS platform. Most of our experiments have also been duplicated in Glomosim. The NS basic Bluetooth model has been programmed by Rohit Kapoor. I have extended this model with some of the Scatternet functionality and have programmed ARP, AODV, related link layer and memory management functions so that intensive multimedia communication can be supported. The model implements most of the features of the Bluetooth baseband layer like Frequency Hopping, Time Division Duplexing, ARQ (Automatic Retransmission Query), Multi-Slot Packets, Fragmentation and Reassembly of Packets. Frequency hopping is modeled as a pure pseudo-random sequence. If two or more transmissions occur on the same frequency, the SIR (Signal-to-Interference Ratio) is evaluated using the gain factor g of each radio channel. The factor g is considered constant during the packet transmission and its value is obtained by considering the path loss due to distance with an exponent between 2 to 4, log-normal Gaussian random shadowing and fading with unity mean. For each portion of the packet of length L in which the SIR is considered constant the information bit error probability is evaluated by taking into consideration the modulation adopted and the FEC coding (if adopted). Details on the channel model can be found in [NMag01]. The polling scheme used in these experiments is pure round robin. Scatternets are supported through gateway nodes exactly as described in the architecture below. In reality, piconets are not be synchronized and this results in the loss of 1 or 2 time slots when a gateway makes a switch between asynchronous piconets. In our model, we do not consider this here for clarity of results. Issues related to the scatternet architecture like “Inquiry” and “Paging” phases [Bluet] and inter-piconet scheduling are presented later.

Ad-hoc routing protocols like AODV rely on a broadcast at the MAC layer to propagate the routing packets. In Bluetooth, packets can be broadcast inside a single piconet using the piconet broadcast address of –1 [Bluet]. In order to support broadcast of routing packets to outside of a piconet, we define the rules: (i) If a non-gateway slave node needs to broadcast a packet, it sends the packet to the Master of the Piconet it belongs to; (ii) If a Master node needs to broadcast a packet, it broadcasts the packet to all the non-gateway slaves in its piconet using the piconet broadcast address of -1 and unicasts the packet to each of the Gateway slaves when they visit the piconet. A Gateway slave node wanting to broadcast a packet sends it in a unicast manner to the Master of each piconet it belongs to when it visits the piconet.

1.1.1 Scatternets and Inter-Piconet Scheduling Simulation

In this paragraph we introduce our inter-piconet scheduling work. We wanted to produce a realistic and implement-able Scatternet model rather than just assume perfect inter-piconet gateways, which is not possible in an ad-hoc mobile real environment. Gateways are in reality established one after another, without possibly having the future, whole picture.

A slave of piconet i, that recognizes another master’s (piconet j) INQUIRY, may become a gateway between its home piconet and the visiting piconet. There may be other gateways already present in the home piconet as well as in the visiting piconet. Also, the prospective gateway node may already be a gateway (i.e. connecting its home piconet with some other piconets). Therefore, the parameters that affect the forwarding throughput (FT) of the gateway are the existing rendezvous point in home, visiting piconet and gateway. Note that a rendezvous point is the number of slots after which the master will poll. In order to use absolute numbers to refer to rendezvous points (rather than relative to the establishment time) we also introduce the notion of the superframe cycle (measured in slots). A superframe cycle defines the period after which the rendezvous point will re-occur. Therefore all operations are assumed to be in modulo superframe arithmetic. Note that the superframe notion is logical only, and does not imply any piconet synchronization. The possible forwarding throughput from one piconet to another based on the RV points of its gateways. An intuitive depiction of the situation is attempted in Fig []. It is defined by the presence interval of the gateway as a fraction of the superframe, and the number of other slaves and gateways that it has to share the date rate during that time. The algorithm we use calculates the possible FTs between the visiting piconet and all piconets it connects to, as if the proposed connection had already been established. For each possible rendezvous slot it looks at the minimum among the FTs. It then keeps the point that results in the maximum of the minimum FTs. For details and equations in this complex problem please see [Kaz01t].

We have modeled in C++ a set of objects that describe the nodes, piconets and gateways and interface both with NS (see Figure 3.2). We define an event as a new node that is to become a gateway between a new pair of Piconets (home Piconet and some other one). The list of events is sorted based on a random eventId. This simulates the Beaconing process described in the standard, as it is random which node pair becomes aware of their proximity first. Then the RVs are calculated based on the algorithm proposed.

Figure 3.2 Scatternet Simulator and Interface with NS and GlomoSim Bluetooth model

1.2 Bluetooth Scatternet Architecture

The current Bluetooth specification defines the notion of interconnected piconets, called scatternets, but does not define the actual mechanisms and algorithms necessary to set up and maintain them. A scatternet will typically be set up in an ad-hoc fashion to interconnect piconets. For instance, a typical personal area network (PAN) would comprise of a user’s wearable and portable electronic devices in a piconet formation, while scatterneting enables networking between many PANs. The inter-piconet Bluetooth units, i.e. the gateways interconnecting the piconets in a scatternet, need to time division multiplex their presence in each of their piconets. The time division in gateways effectively limits the network capacity and may introduce bottleneck points in the network. Therefore an inter-piconet scheduling (IPS) algorithm is required in order to efficiently coordinate the inter-piconet Bluetooth units and maximize the network capacity. This may be achieved, for example, by using rendezvous points –pre-negotiated slots that a piconet master agrees to poll the gateway, and the gateway agrees to have switched to that master’s frequency hopping sequence and accept the poll. The impact of the rendezvous IPS algorithm to the overall scatternet is studied. We consider this an important question because the inter-piconet scheduling functions require support from low layers usually implemented in silicon [Ger01]. We assume a distributed approach is required and evaluate performance of a range of different rendezvous scheduling algorithm. On one extreme we have a minimum intelligence/silicon space totally random assignment and on the other a maxmin optimum forwarding throughput assignment. This is achieved by maximizing a forwarding throughput function whenever a node is connecting to a new piconet. The function depends on locally known information and therefore the algorithm is naturally a distributed one. The algorithms presented can be extended to support re-negotiation of the RV-points and traffic requirements. We argue that a totally random one is inappropriate for most applications. The maxmin optimum algorithm on the other hand is shown by simulation experiments to have a total network throughput of more than a 30% than simpler non-random solutions without creating bottleneck gateways. This improvement may be considered important to some applications, especially for the range of typically expected scatternet data rates, but comes at the cost of a higher design complexity/cost.

The inter-piconet scheduler (IPS) is crucial for the operation of scatternets. Very little work on inter-piconet scheduling issues in scatternets can be found in the literature. In [NJo01] an inter-piconet scheduler is analyzed for a case where central control of the entire scatternet is assumed, which is costly to achieve. The work herein will focus on analyzing inter-piconet schedulers that operate in a distributed manner in scatternets and thus eliminate the need for scatternet-wide coordination.

We next define the general problem including the calculation of forwarding throughput of a gateway between two piconets, and in par. 2 the allocation schemes. In 3 we describe the simulation models and in 4 the results. We conclude in 5.

1.2.1 Background

Bluetooth nodes perform periodic INQUIRY/PAGING in order to sense the existence of other nodes. When a master senses a new node they exchange configuration parameters so that the node may join the piconet and vice versa [Bluet]. When a node already belongs other piconet(s) (as e.g. the home piconet, the piconet it first joined) it may join the new piconet and become a gateway between the two (or more). In an rendezvous point based IPS, if the node was not a gateway previously, two rendezvous points need be established, otherwise one new point for the new piconet is enough. Whether a node should actually attempt to become a gateway or not, is decided by a gateway selection algorithm as in [Sal01], [Agg00]. Existing rendezvous points of the new master and the gateway should be exchanged during this gateway establishment phase, so that the new rendezvous slot does not conflict with others. Our algorithm uses this information, not only to choose non-conflicting new rendezvous point(s) but also to decide the optimal new rendezvous point(s). Note that the establishment of a new rendezvous point in a visiting piconet, does not only affect the forwarding throughput of the (home, visiting) piconet pair. It also affects the forwarding throughput of all connections of the visiting piconet. Our algorithm picks the point so that the minimum forwarding throughput (f.t.) from the visiting piconet to any of its connected piconets is maximum. In this way bottleneck links are not created in the network and overall forwarding throughput is close to optimal. We also experiment with a greedy algorithm –one that maximizes the f.t between visiting and home piconet and found that it may create bottleneck piconet pairs. Simpler algorithms are also used in simulations that require less computation, but have worse performance and/or flexibility.

Let us assume that a number of piconets is formed and as they move within a terrain they come into proximity to each other and may form scatternets. This is for example, similar to a conference PAN scenario where one person’s devices are arranged in a piconet and people move about a conference show room and communicate. A slave of piconet i, that recognizes another master’s (piconet j) INQUIRY, may become a gateway between its home piconet and the visiting piconet. There may be other gateways already present in the home piconet as well as in the visiting piconet. Also, the prospective gateway node may already be a gateway (i.e. connecting its home piconet with some other piconets). Therefore, all the parameters that come into play are:

Note that we have previously defined a rendezvous point as the number of slots after which the master will poll. In order to use absolute numbers for rendezvous points (rather than relative to the establishment time) we also introduce the notion of the superframe cycle. A superframe cycle defines the period (measured in slots) after which the rendezvous point will re-occur. Therefore all operations from now on may be assumed to be in modulo superframe arithmetic. Note that the superframe notion is logical only, and does not imply any inter-piconet coordination or synchronization. It simply defines the RV point in absolute terms. Say for example the superframe is 90 slots and a gateway’s RV point is 30. The gateway knows after how many slots to switch at any slot t (that is 30-t if t mod 90<=30 or 90-t+30 if t mod 90>30). In the implementation the superframe cycle length and current slot may become known to the gateway when a link is established.

Now we are ready to calculate the possible f.t. from one piconet to another based on the RV points of its gateways. An intuitive depiction of the situation is attempted in Figure 3.3.

Figure 3.3 The number of nodes active in the polling cycle during the superframe cycle of a piconet with s non-gateway slaves and 3 gateway nodes.

The fraction f that a gateway G_g spends with master M_k:

The average number of nodes that actually participate in the polling cycle depends on the polling scheme used. Let us call s the number of active nodes in the polling cycle and its average in the superframe. For example, in round robin non-exhaustive, if the masters know when gateways are present, it becomes:

For a pure round robin scheme, for example, the fraction f of a slave that is not a gateway is 1. Let us now calculate the f.t. for a scatternet link from (adjacent) piconets i to j interfaced by a gateway g.

(1)

In the above formula b is the Bluetooth bandwidth (1Mbps). The formula is validated using our simulation models in the next section. It can be extended for more than one hop (not adjacent piconets).

1.2.2 Validation

We validated the above equation using our GlomoSim Bluetooth model and our NS model. The part of the topology involved in our validation is shown in Fig 3. In order to calculate the f.t. of gateways we had every node shown participating in a high bandwidth (640Kbps) CBR intra-piconet or inter-piconet connections. In this way, the network is saturated, the queues fill up and the gateways, being the bottleneck points, work at their f.t.s. Gateway 10 in the experiment forwards packets from master 8 to master 24. The simulation gave us a goodput of 99Kbps for this connection. According to our equation let us first calculate the average number of active slaves in the piconet of master 8 when gateway 10 is present; that is from 80 to 20 in a 120 slot superframe. The two slaves S11 and S12 are present through the whole gateway presence interval. Since this is a pure round-robin run we count a fraction of 1 for each. A fraction of 1 is also counted for the gateway itself and 20/60 for gateway 9. This is because the G10 presence interval is 60 slots and for 20 of those, slot 80 to 100, G9 is also present. Similarly we get 20/60 for gateway 7. This gives an of 3.66. Similarly for master 24 we get an average number of slaves of 3. The G10 presence fraction f is 60/120=0.5 for both piconets and so the f.t. is defined by the piconet of master 8 (with the minimum FT_i and the max ) as 0.5*721 / 3.66 = 98.3Kbps, which is very accurate compared with the simulations!

1.2.3 Rendezvous Points Allocation Schemes

The proposed algorithm calculates the possible FTs between the visiting piconet and all piconets it connects to, as if the proposed connection had already been established. For each possible rendezvous slot it looks at the minimum among the FTs. It then keeps the point that results in the maximum of the minimum FTs. If the node has not been a gateway so far, then all slot pairs are examined and the best (maximum) pair is chosen. The logic behind it is to avoid establishing RV points that take away forwarding throughput from other piconet pairs. We also experimented with different RV assignment algorithms. We found that the overall average possible scatternet throughput with this algorithm is close to the greedy one (see 2 below), which has optimal average throughput but creates bottleneck links. We refer to this algorithm as RV-maxmin. It can be extended to handle traffic requirements when those are known at the IPS module at RV assignment time. In that case, instead of simply choosing the RV point that gives the maximum minimum f.t., the choice can be limited in the RV points that satisfy the given traffic requirements, again based on equation 1.

Other possible algorithms that represent the range of options in this type of IPS algorithms are:

RV-Separation: The new RV point returned is the midpoint of the two more distant already established points.

RV-Greedy: The new RV point returned is the one that maximizes the FT between the home and visiting piconet (directly from eq. 1)

RV-tree: The RV-tree algorithm chooses RV points from a predefined global list of available RV points. In our implementation each master divides its superframe in 3 intervals and one and only gateway may be present during such an interval. For example, if the superframe is 90 slots the RV-points 0, 30 and 60 are given out. This scheme is very simple and has optimal performance when exactly 3 gateway nodes are assigned per piconet. When more than 3 gateway nodes are needed they have to share the RV slot and take turns.

Figure 3.4 The topology for the validation of the equation.

1.2.4 Scatternet Model & Results

We have modeled in C++ a set of objects that describe the nodes, piconets and gateways and interface both with NS and GlomoSim [Kaz02] (see Figure 3.2). Initially the terrains dimensions are decided based on the number of piconets, the range of 10m and a topology expand factor (0.7). The latter can be used to have super-imposed or totally expanded piconets. Both dimensions then are:

(2)

Then, the piconets are formed by uniformly distributing the master nodes and a random number of slave nodes around them. Next a list of all the possible gateway nodes is produced according to the distances from foreign masters. This list is sorted based on a random eventId. This simulates the Beaconing process described in the Bluetooth specification, as it is random which node pair becomes aware of their proximity first. Each gateway is tried in turn against system limitations (number of slaves in visiting piconets) and imposed limitations. The imposed limitations are parameters that significantly determine IPS performance. These are the Maximum Gateway per Piconet (GP) allowed limit, and the Maximum Piconet per Gateway (PG) allowed limit. The limits affect greatly the resulting topology and therefore we examine all possible values of them. That is 2 to 7 GP and 2 to 7 PG (for the RV-tree only more than 2PG are not tested). Note that the (2 GP, 2 PG) case is the totally serial connection and the (3GP,2PG) is the tree structure. After a gateway node is decided the RVs are calculated based on the algorithm proposed. After the scenario is simulated, bandwidth, delay and other statistics of the specific assignment are collected.

The following results are obtained from the Scatternet Object Simulator. The number of piconets tested ranges from 3 to 123. We run one experiment for each configuration incrementing the number of piconets by 3. The nodes ranged from 10 to 513. The terrain ranged from 20*20 to 140*140 square meters according to (2) with a 0.7 expand factor. Each point in the aggregate graphs represents the average over all gateways and over all the experiments performed for this number of piconets (this is 41 -123/3- experiments per point). In all the experiments presented, the average f.t. is not generally affected by the number of piconets but only from the imposed limits. Therefore each point in this graph shows approximately the average f.t. for each experiment as well. In the graphs we have used the 1Mbps as the link throughput.

The connectivity achieved by a scheme is an important attribute of the network created. It affects the network performance through alternate routing paths. Assume each piconet is a node. One edge exists on the graph if there is one gateway connecting the two piconets. The average connectivity degree of the graph then is affected by the expand factor of the topology and the imposed limits. Connectivity ranges from 2 to 9 in our experiments as shown in Figure 3.5. The RV-scheme is independent of the connectivity degree since in our runs the decision of whether a node becomes a gateway or not depends on the topology limitations and the imposed limits only.

We present results from the case of pure round-robin. We have also obtained results from a gateway priority polling scheme. On each polling cycle the master polls the present gateways and one slave only, instead of all slaves. In a gateway priority scheme an IPS algorithm has an even larger impact, and our algorithm appears even more beneficial.

In Figure 3.6 and Figure 3.7 we present the minimum forwarding throughput that each experiment had over all its piconet pairs connected by a gateway. Note that each point in the graph is the average of the 41 experiments performed for 3 to 123 piconets. The two axes are the imposed limit GP (x axis) and PG (y axis). Figure 3.6 shows the RV-maxmin against the RV-separation scheme. The RV-separation did an equal or worse job than RV-maxmin. However, as shown below, RV-separation has poor average f.t. performance. Figure 3.7 shows RV-maxmin, compared with the two other schemes RV-tree and RV-greedy. The difference there is even more notable. The RV-maxmin scheme allocated RV-points so that the minimum forwarding throughput across any two directly connected piconets was double than the other schemes.

Figure 3.8 and Figure 3.9 compare the average, instead of the minimum, performance of the RV-maxmin scheme to the three others. The average on one experiment is obtained by averaging the forwarding throughput across all piconet pairs directly connected through a gateway. One point is shown on the graph for each imposed limits pair (GP, PG). This is the average over the 41 experiments (from 3 to 123 participating piconets). The RV-maxmin has average performance very close to RV-greedy. Since it also has much better minimum f.t. performance it is the ideal scheme for Bluetooth Scatternets based on rendezvous points.

Note that the minimum f.t.’s are similar between RV-greedy and RV-maxmin in the case of allowing only two piconets per gateway. The RV-greedy problem of creating bottlenecks exists particularly when a node is allowed to become a gateway of more than two piconets. Specifically, assume a gateway already connects two (or more) piconets. Assume that RV-greedy assigns RV-points that are superframe/2 apart. When a third piconet is attempted by this gateway the RV-points that maximize the f.t. for one pair may turn out to cause a bad f.t. for another pair. The RV-maxmin scheme deals effectively with this problem because it is maximizing the minimum f.t. that may occur for any pair from a new RV-point assignment.

Let us now look more closely at the f.t.’s accomplished by the schemes and how they distributed across the piconet pairs. Figure 3.10, Figure 3.11 and Figure 3.12 show all f.t.’s for each piconet pair connected through a gateway node. For illustration purposes we ordered the f.t.’s. The x-axis is an incrementing number of a connected piconet pair. Each point in the graph is the f.t. of a piconet pair in the 123 piconet case (less piconets exhibit the same shape in the graphs). Figure 3.10 shows a characteristic case. We may observe the performance of the three schemes (RV-Separation –noted NOOB1, RV-greedy –noted OPTB1, and RV-maxmin– noted OPTMB1), for imposed limits (GP, PG) equal to (2,3). Note how the RV-greedy scheme on average has connected the piconet pairs with better f.t., but there are some pairs (14 in number) that only allow an f.t. of about 10Kbps. The RV-maxmin scheme, in contrast, does not have these tails. All pairs have more than 100Kbps f.t. Similar trends are show in Figure 3.11 – the (3,3) (GP,PG) case and in Figure 3.12 – the no limits case.

Figure 3.5 Average connectivity degree versus PG (x axis) and GP (y axis) limits.

Figure 3.6 Round Robin f.t. without gateway priority, RV-maxmin and RV-Separation : Minimum averaged F.T. against GP (x axis) and PG (y axis) limit

Figure 3.7 Round Robin f.t. without gateway priority, RV-maxmin, RV-greedy, RV-tree : Minimum averaged F.T. against GP (x axis) and PG (y axis) limits.

Figure 3.8 . Round Robin f.t. without gateway priority, RV-maxmin and RV-Separation : Averaged F.T. against GP (x axis) and PG (y axis) limits.

Figure 3.9 . Round Robin f.t. without gateway priority, RV-maxmin, RV-greedy and RV-tree: Average F.T. against GP (x axis) and PG (y axis) limits

Figure 3.10 Piconet pair F.t. distribution for 123 piconets (2,3) (GP,PG).

Figure 3.11 . Piconet pair F.t. distribution for 123 piconets (3,3) (GP,PG).

Figure 3.12 Piconet pair F.t. distribution for 123 piconets and no imposed limits - (7,7) (GP,PG).

Next we look at scatternet end-to-end results. The graphs presented here are obtained by running several simulations of a scatternet consisting of 123 piconets. In order to obtain end-to-end results we run a Dijkstra algorithm to get routes formed with (i) the minimum hop distance path between all Piconet pairs and (ii) the maximum forwarding throughput path. In Figure 3.13 we show the min hop distances achieved for each combination of (GP, PG) limit. The min hop distance depends on the actual distance of the piconets, the applied limits and the random order that nodes sense foreign masters. The graph shows that the no limit ([GP,PG]=[7,7] ) min hop distance –which is the best average min hop- can be achieved for any limit combination that allows 3 or more gateways per piconet. In fact full connectivity is achieved for limits higher than (3,3) as well. In Figure 3.14 we see the hop distribution for the (3,3) fully connected case. In Figure 3.15 we see the average end-to-end bandwidth that can be achieved for each limit pair. The rendezvous point allocation scheme used is RV-MaxMin-Optimal. We see that the highest end-to-end throughput can be achieved for the (3,2) case and then for the (3,3) case. The worst case is the serial connection (2,2). In Figure 3.16 we look at the fully connected case (3,3) and show the distribution of the end-to-end throughputs for each pair of piconets. We note that while the RV-optimal case has slightly higher average throughput, few piconet pairs can only communicate with less than 25Kbps. The RV-MaxMin-Optimal does not create these bottleneck points and the minimum end-to-end throughput pairs can communicate with 100Kbps instead.

Figure 3.13 Hop distance averaged over all piconet pairs in scatternet. (x axis is the GP limit)

Figure 3.14 Hop distance distribution for the (3,3) (GP,PG) limit (x axis is hop distance, y axis is number of piconet pairs that are connected with the hop distance with a min-hop route)

Figure 3.15 End-to-end route throughput averaged over all piconet pairs for the RV-maxmin case. (x axis is the GP limit, y axis is the route mimimum f.t. )

Figure 3.16 Route throughput distribution for the (3,3) (GP,PG) limit (x axis is route throughput in bps, y axis is an increasing number that represents a piconet pair, it is ordered by route throughput) .

1.2.5 Conclusion

The introduction of Bluetooth scatternets enables a wider and more flexible use of Bluetooth network as the means to create local and short range connectivity. However, the scatternet also presents a technical challenge with respect to network formation and scheduling of traffic within a Bluetooth network. In this paper the focus was on rendezvous inter-piconet scheduling (IPS) issues in particular. We studied the impact of the IPS algorithm on the scatternet performance. We consider this an important question because the inter-piconet scheduling functions require support from low layers usually implemented in silicon [Monet2]. We argued that some intelligence in the inter-piconet scheduling is necessary and used a general architecture for scheduling in scatternets that enables the deployment of IPS optimizing algorithms.

We specifically studied the rendezvous point assignment issues in Bluetooth scatternet architectures that are based in rendezvous points established between master and gateway nodes. Establishing honored RV-points is a promising approach because it sets the ground for possible QoS guarantees and does not waste slots on failed master/gateway attempts to communicate. We found that the ideal algorithm in terms of gateway forwarding throughput is the RV-maxmin. This algorithm has average performance very close to the one provided by RV-greedy and does not have the disadvantage of creating low throughput gateway connections. This is highly likely to happen in RV-greedy when more than 2 piconets are allowed per gateway. Note that optimality here is local in time and space, i.e. as the problem is known at the time and place of solution. When an event occurs – a node is to become the gateway between a new pair of piconets – it optimally solves this local problem. We structured the design space according to imposed limits on the number of Piconets allowed per Gateway (PG) and on the number of Gateway allowed per Piconet (GP) and explored all possibilities of a scatternet architecture based in fixed (for the life of a gateway connection) Rendezvous Points that are always honored by master and gateway. Intelligence added to the IPS increases the space (silicon) and time cost but improves overall scatternet performance by 30%, a percentage that is crucial for some applications especially in the scatternet expected throughput range. The algorithm can be extended to support RV point re-negotiation and take into account traffic requirements, if those can be made known to the IPS module. We showed that RV-maxmin, based on maximizing the minimum forwarding throughput across the involved piconet pairs is ideal for scatternets. In a nutshell, we argue that a totally random one is inappropriate for most applications. The maxmin optimum algorithm on the other hand is shown by simulation experiments to have a total network throughput of more than a 30% than simpler non-random solutions without creating bottleneck gateways. This improvement may be considered important to some applications, especially for the range of typically expected scatternet data rates, but comes at the cost of a higher design complexity/cost.

2 The Network Feedback Architecture

Traditionally, transport protocols rely on end-to-end measurements since this is a simple, easily deployable, scalable and transparent solution. The common approach is that each session employs end-to-end monitoring to estimate quantities of interest, like delay, delay jitter and available bandwidth. However, it is understood that in last mile and multihop wireless networks end-to-end solutions may be less efficient. At the same time, higher layer protocols in wireless networks need to dynamically adapt to observed network response. A less conventional approach is to employ lower layer explicit feedback mechanisms in place or in aid of end-to-end efforts. Available bandwidth measurements are known to follow multi-modal distributions and therefore are especially difficult to measure and filter, even in wired networks. In wireless, especially 802.11-based multi-hop networks obtaining usable end-to-end measurements is questionable. They are affected by a combination of a large number of transient variables due to the virtual carrier sense, head of line problems on each link and mobility. Motivated by this, we are developing a network explicit feedback mechanism. Our study of this accurate network feedback architecture aids in the cost/benefit analysis of an important trade-off: deployment of network support mechanisms for transports and QoS, versus the simple, scalable and easily deployable end-to-end solution. We use a single link, source-destination pair specific measurement, suitable for multi-hop networks. The measurement can easily be provided with MAC layer co-operation but we also provide and experiment with a self-contained network layer implementation. This measurement is then propagated throughout the network and made available to the sources using the routing messages. It is then used at the sources for two purposes: (i) direct multimedia adaptation and (ii) perform measurement based call admission. Loss rates of end-to-end adaptive video and audio connections have been more than 4 times higher than in the network feedback case. A simple call admission strategy has also proved very effective using the feedback. In our experiments it led the network to a maximal performance and stable operating point.

The main disadvantage of lower level support of measurements is that nodes in order to operate in the network need to provide that support. This limits scalability, ease of deployment and increases the node cost. Note that the network support requirement may be limited in local or last mile networks by use of middleware architectures and nodes in part of the path may in this way be unaware of it. Also this scalability issue is not of the magnitude of the traditional per flow QoS problem. The aggregation is per node link and route. Two or three extra fields per route contain the necessary information and distance vector, link state or on demand propagation is feasible. But, how much would be the benefit of deploying such network support versus its cost in networks with wireless links? In setting the grounds to answer this question we explore a lower level support architecture. We then use the explicit measurements for a) multimedia adaptation b) measurement based call admission.

First, we focus the study on asynchronous 802.11 networks. We find this timely also because of additional pressure for 802.11 to support voice and multimedia calls in the infrastructure-less mode (DCF) due to Bluetooth advent. The DCF mode is the only mode currently supported by 802.11 in the infrastructure-less ad-hoc network and is also the only mode currently implemented in Wavelan even in infrastructure mode (access point support). 802.11 based networks do not lend themselves to accurate end-to-end measurements and ‘trial and error’ approaches are tardy in supporting efficient adaptation.

The 802.11 standard allows for link performance measurements. Here we use the fragment acknowledgement and link-fail message used for unicast traffic in DCF to produce link-by-link (source, destination) pair throughput measurement. Link layer queue utilization measurements are then combined to produce a available bandwidth measurement, in a suitable way for a wireless multi-hop network (see also contention models in [Nad00]). Propagation of the available bandwidth values can be accomplished using piggybacking to routing messages or as a separate application. The former constrains the two mechanisms, with possibly different cost functions, to work on relevant time scales. Their propagation may be distance vector based, link state based or on demand. In this paper we are using distance vector propagation. In [Kaz01b] we have developed an event-driven propagation based on AODV [Per99].

Link bandwidth measurement has been attempted in the past for networks with wireless links (including multi-hop) using a variety of approaches; link or end-to-end, passive or active. However, there is no study that is either a) based on an implementable approach b) has been experimented with in multi-hop networks c) uses MAC layer and link layer features and d) provides a network layer measurement mechanism. In [Can99] a maximum available bandwidth measurement per node is proposed for QoS support in MANETs, but is only expressed in a formula. Translation of the formula to a distributed algorithm would result in a complex scheme because it would require up-to-date neighborhood knowledge.

Figure 4.1 Node block diagram architecture for network feedback support.

Figure 4.2 The network feedback case: accurate measurement and better propagation.

2.1 Propagation of Available bandwidth

We have implemented a distance vector based propagation that piggybacks its values to existing routing broadcasts. In the propagation we are interested in discovering the minimum available bandwidth along the path (bottleneck link). In order to allow for efficient updating of the tables, we use both the measurement (expressed as negative available bandwidth as a cost) and the hop count to the discovered bottleneck link. This is necessary to detect bottleneck point changes and cancellations.

The effectiveness of the routing algorithm is central to most network functions. In wireless networks routing QoS support requires significant bandwidth consumption. This is due to the link state nature of QoS routing algorithms and mobility support. Clearly, it is a central issue for feedback systems as it affects both forward and reverse path. Network feedback provision is closely related to the routing algorithm. Further discussion on routing can be found below in the feedback mechanisms context.

In a QoS routing algorithm, the QoS information is propagated regularly through the network as integral parts of the routing table. This information is then readily available to the application in the form of network feedback. More importantly, it can used to perform admission control, necessary for bounding excessive delays and excessive congestion. A QoS routing algorithm can also search for alternative routes and distribute the traffic in such a way that avoids oscillations of service. A feedback mechanism not based on such techniques can misinterpret such oscillations since these are likely to happen with a larger period than the closed loop round trip delay.

2.2 QM-AODV QoS propagation and aggregation support

We have designed and implemented a propagation of the measurement based on AODV [Per99] routing algorithm for MANETs. We use the recommendations in the QAODV RFC [Per00]. This RFC defines the formats that augment the AODV data packets with the propagation and support of QoS metrics, namely delay and bandwidth. Our first implementation, published in [Kaz01b] was prior to QAODV. The latter augments AODV with functionality that supports call admission based on QoS values. For example a node sends a route reply message, not only if it knows the path to a destination, but also if that path satisfies the QoS constrains contained in the route request message. It does not deal however at all, with how these values get obtained or aggregated in the routing tables, and therefore measurement based call admission needs extensions in order to be supported. We do exactly that; add those extensions to QAODV, namely a QoS aggregation method. Since QoS information needs to be up to date we use a special route reply message that is used to update these measurements. When a node measures a significant change to its QoS with the neighbors it can use such a packet to propagate this information. This way we accomplish an event driven measurement update method, in accordance with the on demand nature of AODV.

In AODV only routes to destinations each node communicates with are recorded. The uniqueness of AODV is its two dimensional routing metric. The hop count metric accounts for building optimal routes, and the sequence number ensures the freshness of the route. When a source has data packets to transmit to the destination but does not have any routing information, it floods the entire network with the Route Request (RREQ) packet to discover a route. A node can respond to this RREQ by sending a Route Reply (RREP) to the source if one of the following two conditions is satisfied. The node must either be the destination itself, or it must have the ``fresh enough'' routing information to the destination. When the source receives a RREP, it learns the route to the destination and can hence send data packets. When a node detects a route disconnection, the upstream node of the broken link sends a Route Error (RERR) message to the upstream direction and to the source to notify a route break. Nodes that receive this packet remove the routing table entries that use the broken link. The source can re-discover a route when needed. QAODV as mentioned extends the packet format to add QoS requirements to the route request messages and the functionality of route replies messages, so that only nodes that have QoS-adequate paths reply.

We made implementation changes to QAODV in order to support the efficient propagation of the measurement values. We have kept the AODV philosophy in all the decisions taken for the propagation of available bandwidth, by using an event-driven approach for the measurement updates as well. In this paragraph, we assume that the available bandwidth and delay values are estimated and available from each node to each neighbor.

First, one extra field, other than the ones added by QAODV i.e. available bandwidth and delay, needs to be added to the routing tables: the hops to the link that defines the available bandwidth i.e. the bottleneck link. It is necessary to identify the distance of the current node to the bottleneck link to enable detection of upgrade of the route's bottleneck link without route change.

Additionally, hop based AODV naturally reacts only to path changes (link failure messages), whereas our setting requires a more frequent update to detect changes in available bandwidth measurements along an unchanged path. In order to accommodate this while keeping the on-demand philosophy we added the "event triggered RREP". When the measurement values is changed for more than RELAY_FACTOR per cent (%) than the previously relayed measurement for the link (source-destination pair) then an RREP is triggered. A RELAY_FACTOR of 1.0 (never trigger RREP from change in measurement) naturally minimizes the control overhead but allows learning of new measurements only following route changes. A RELAY_FACTOR of e.g. 0.2 (trigger an RREP when the measurement has changed by at least 20%) increases control overhead but results in learning about a change along a path within at most a round-trip time. During an RREP the node sends this information to the upstream node of the route. Upon receiving this event triggered RREP, a node will not relay the measurement packet unless the event triggering condition holds between his own last transmitted value for the link and the newly received.

The changes in AODV are:

· When a node receives a Request it should naturally update the reverse path with the QoS information as well, not just the hop. Since the Requests are received from neighboring nodes it is possible to know the QoS values of the reverse path. If the reverse path is not known it is added and the fact that the bottleneck hop is one is recorded in the tables. If it was known from before the new values are obtained and compared to the existing ones. If the bottleneck hop was the first hop, the new values are updated, else the new values are entered in the routing table only if they denote a smaller available bandwidth. Delay is replaced for the destination.

· After updating the reverse path the node should attempt to reply, if it knows of a suitable and fresh path. Before replying the node needs to check the QoS values for the next hop on his path and update this in the route reply message as before.

· When a node receives a Reply it first distinguishes whether it is a Route Reply or a Measurement Reply.

o It then looks in the routing table to check whether it already has a route for the data packet destination. The node then decides whether it should forward the Route Reply.

§ If it is a measurement reply then it forwards according to the RELAY FACTOR rule, which requires it to check its current routing tables for the data destination.

§ If it is a normal route reply then it will forward if the route packet’s destination is not itself.

o If the node does not have a route entry to the destination, the node checks its QoS to the source address of the route message. If it knows of a smaller available bandwidth it will update the route reply with it and forward.

o In the above, it might happen that the node needs to forward a Measurement Reply but does not have a fresh route to the destination (the route might have expired, or an MREP arrived first than an RREP). In that case it may mark that the next measurement update should be forwarded in any case, overriding the RELAY FACTOR rule.

In our experiments with adaptation, the routing still operates without being affected by the propagation of these values, other than the increase in the size of messages, and calculates the minimum hop routes. The pseudo-code follows:

Table 4‑1 Pseudo code for QM-AODV modifications to AODV

RT_RESOLVE {

If route is up {

If (! toforward || I am Not the source) {

If ( Mac->AB [ Rt->NextHop ] > Rt->AB AND This is the bottleneck hop )

OR ( Mac->AB [Rt->NextHop] < Rt->AB * Relay_Factor ) {

UpdateRoutingTable;

If I am not the Source {

Send MREP To the Reverse Path

}

Aodv Resolve

}

RECV_REQUEST {

//Opportunity to Refresh/Update Source of Rpacket from Local Measurement

Update Reverse Route (

If It was not Up Update it with MyAB

If MyNextHop is the RPacket Source OR

Request->Bhop==1

OR MyAB< Reverse Route AB {

Update with MyAB

}

Handle buffered packets as AODV does

)

Attempt to Send Route Reply

If I am the destination send AB=Inf

If I am not but have a fresh route {

If MyNextHop is the source of Request OR

Rq->Bhop=1 OR MyAB < Rt->AB {

Update Bandwidth and Bottleneck Hop in Routing

}

SendReply with My RT->AB and Bhop+1

} Else Just Forward the RQ

}

RECV_REPLY {

// When ToForward=1 will trigger an MREP to the previous hop,it is next hop now since I am on the reverse path to source. The next packet from the source will find the reverse route updated and forward

Decide what to forward and update my table (

If I have a route to dest {

If Rp->AB<Rt->AB(1-Relay_Factor) {

If it is an MREP ToForward=1;

} Else {

Rp->Bhop++; Rp->AB=Rt->AB

}

} Else {

If Rp->AB > My_AB(Source of Reply) {

Rp->AB=My_AB, Bhop=1; If it MREP ToForward=1;

} Else {

Rp->Bhop++;

}

Update or Add to My RTable As in AODV

)

SEND_REPLY(ipdst) {// Support MREP }

GET_AB(dst) { if (Rt->hops==1) return AB(dst); else return Rt->AB }

Table 4‑2 Sample Run of QM-AODV operation

1.02400 7 Sending Request for 14

1.02633 4 Received Request ( 7 asking for 14 ) from 7 hops 0

4 New Route to 7 on Reverse Path (313516.098457,0.000173,1, hops 0)

4 had No Route to 14 (7 was asking) REQ received from 4 hops 1, just forwarding.

4 Forwarding with Source Address updated from 4 to 4

RTABLE ->7 Nexthop:7 Hops:0 B:313516 D:0.00017 Bhop:1

1.03195 5 Received Request ( 7 asking for 14 ) from 4 hops 1

5 New Route to 4 on Reverse Path (313452.155716,0.000173,1, hops 0)

5 New Route to 7 on Reverse Path (313452.155716,0.000173,1, hops 0)

5 had No Route to 14 (7 was asking) REQ received from 5 hops 2, just forwarding.

5 Forwarding with Source Address updated from 5 to 5

RTABLE ->4 Nexthop:4 Hops:0 B:313452 D:0.00017 Bhop:1

RTABLE ->7 Nexthop:4 Hops:0 B:313452 D:0.00017 Bhop:1

1.03643 6 Received Request ( 7 asking for 14 ) from 4 hops 1

6 New Route to 4 on Reverse Path (313516.098459,0.000173,1, hops 0)

6 New Route to 7 on Reverse Path (313516.098459,0.000173,1, hops 0)

6 had No Route to 14 (7 was asking) REQ received from 6 hops 2, just forwarding.

6 Forwarding with Source Address updated from 6 to 6

RTABLE ->4 Nexthop:4 Hops:0 B:313516 D:0.00017 Bhop:1

RTABLE ->7 Nexthop:4 Hops:0 B:313516 D:0.00017 Bhop:1

1.05498 14 Received Request ( 7 asking for 14 ) from 4 hops 1

14 New Route to 4 on Reverse Path (224807.386372,0.000277,1, hops 0)

14 New Route to 7 on Reverse Path (224807.386372,0.000277,1, hops 0)

14 Reply is out to 7. hops 0, BHOP is 0

14 Forwarding with Source Address updated from 14 to 14

RTABLE ->4 Nexthop:4 Hops:0 B:224807 D:0.00028 Bhop:1

RTABLE ->7 Nexthop:4 Hops:0 B:224807 D:0.00028 Bhop:1

1.06126 4 Received Reply from 14 about 14 :(1000000.000000,0.000000, 0, hops 0)

4 Looking at my QoS to 14 (191657.018441,0.000339)

4 Decided that Qos to 14 would be (191657.018441,0.000339,1, hops 0)

4 Now Forwarding the Reply to 7 (191657.018441,0.000339,2,hops 0)

4 Forwarding with Source Address updated from 14 to 4

RTABLE ->14 Nexthop:14 Hops:0 B:191657 D:0.00034 Bhop:1

RTABLE ->1 Nexthop:1 Hops:0 B:204282 D:0.02103 Bhop:1

RTABLE ->0 Nexthop:1 Hops:0 B:204282 D:0.02103 Bhop:1

RTABLE ->7 Nexthop:7 Hops:0 B:191657 D:0.00034 Bhop:2

1.06939 7 Received Reply from 4 about 14 :(191657.018441,0.000339, 1, hops 1)

7 Looking at my QoS to 4 (314012.000383,0.000173)

7 Decided that Qos to 14 would be (191657.018441,0.000512,2, hops 1)

7 Finally not forwarding reply. It was for 7

RTABLE ->4 Nexthop:4 Hops:0 B:314143 D:0.00017 Bhop:1

RTABLE ->0 Nexthop:4 Hops:0 B:314143 D:0.00017 Bhop:1

RTABLE ->14 Nexthop:4 Hops:1 B:191657 D:0.00051 Bhop:2

Figure 4.3 A scenario of AODV QoS value aggregation and propagation.

2.3 802.11 network layer available bandwidth support

In multi-hop networks we measure the link performance for each (source, destination) link pair. In a multi-hop network each node has a unique neighborhood and its traffic suffers unique competition. The competition can be graphed for each node as in [Kok00]. It is therefore absolutely necessary to define a source destination dependent throughput rather a node dependent throughput. We define our available bandwidth using a typical residual bandwidth measurement as

where u is the link queue utilization. Therefore, in order to calculate the available bandwidth we first need to estimate the link throughput. Each node passively estimates its throughput to each neighbor. The throughput seen by one packet of S bits can be calculated as

The throughput we measure using this formula includes fading, internal, external noise and contention. If measured above the link layer, it normally contains (and cannot distinguish) queuing time, 802.11 related overhead and actual bit transmission time. In order to understand what we are measuring with the above equation, let us express it in 802.11 terms:

where t_q is the L2 queuing time, ts the transmission time of the S bits, t_CA the collision avoidance phase time, t_overhead the control overhead time (e.g. RTS/CTS, ACK, Header, 4 propagation delays), R the necessary retransmissions and TB_r the back-off time for retransmission r. This formula shows that the measurement on a source destination basis may take into account the distributed contention and hidden terminal topology formed in multi-hop wireless networks. The 802.11 virtual carrier sense affects the throughput to a destination according exactly to the contention seen by the protocol. When combined with the utilization it gives the permissible throughput that applications may use to adjust their network requirements. The formula reveals some undesirable characteristics that are however common to measurements

- Packet size dependence

- High variance due to random per packet effects.

- Queuing delay has to be properly factored out, if transmission time is marked at or above the link layer

Figure 4.4 (top) a CBR source uses different packet. (mid) packet delays (bottom) the throughput observed per packet.

On the other hand, it has significant benefits compared to end-to-end metrics since those problems are only relevant to a single link. We can identify three distinct areas of operation. A) low load - in this area we have very rare retransmissions, mainly due to random channel errors, the collision avoidance phase is rarely prolonged and the overhead is almost constant and paid once per frame. B) medium load - retransmissions are still rare and the variance in the measurements is introduced mainly by variation of the collision avoidance phase duration. C) high load - collisions and retransmissions, cause high delays and dominate the throughput measurement. Strong hidden terminal effects further complicate filtering in case of multi-hop networks. Detailed specific analyses on 802.11 throughput and delay can be found in [Cro97] and [Bia00].

In Figure 4.4 we show the effect of the packet size on the throughput measurement by simulation. A different packet size is used for every 20 seconds of a one hop connection over a 2Mbps link. The throughput measured as above (with a window of a single packet) starts at barely over 500Kbps (25% of link bandwidth) for a payload of 100 bytes! The different packet sizes are a significant source of noise to the measurements. For illustration purposes we show that by subtracting c as in:

Figure 4.5 Under low load the throughput measurement becomes independent of packet size by substracting a constant c

the effect of the packet size can be measured in simple environments. It includes the transmission times of the overhead bits, RTS, CTS, Header, ACK, the hold for ACK time (see also [Ieee80211]) the extra propagation times and SIFS for the RTS and the CTS and one DIFS that the transmission will be delayed. In our implementation we use a filter that averages these effects over the packets used by the applications.

We use a packet window in order to increase statistical robustness of the measurements. A packet window of 32 samples (fragments) is adequate to produce fast enough, noise immune measurements. For an illustration of this see Figure 4.6 where the high variance of per packets measurements is filtered using the window. The window idle time and window duration are also calculated per window in order to produce the link utilization factor and finally the available bandwidth measurement as

The use of the packet window, in the case that the function resides in the network layer, requires special handling though. Queuing delays should be appropriately factored out. When a packet has waited in queue, the correct delay contribution can be calculated at the time that the corresponding ACK (or NACK) has been received as .

Figure 4.6 Window operation

When it has not suffered any queuing the contributed delay is . In the following section we provide and discuss the network layer algorithm that implement these measurements. One sample is collected per window and it is directed to a digital low pass filter. A Tustin approximation [17] is suitable for this digital sampling of throughput.

In summary, the measurement requires:

- Per Neighbor maintenance of Throughput Estimates, AB estimates

- Per Node maintenance of Busy_time, start_time, n, iQin, iQout

- Per Measurement Window maintenance of Sent, Recv, Dest, Bytes

` The implementation can be as simple as follows:

On Unicast Packet Send

Sent [iQout_ % MEASURE_WINDOW ] = NOW;

Dest [iQout_ % MEASURE_WINDOW ] = destination;

Bytes [iQout_ % MEASURE_WINDOW ] = size; iQout_++;

On ACK Receive

Recv[iQin_ % MEASURE_WINDOW ]=NOW; iQin_++;

On Each Measurement Window

busy_time=Σ(Recv_[idx]-Sent_[idx]);

u = 1.0 - busy_time/(NOW-start_time)

Tsample[dest]=Σbytes[idx] / Σ(Recv_[idx]-Sent[idx]), for idx that Dest[idx]=dest

Asample[dest]=(1-u)Tsample[dest];

Weight=Count(idx that Dest[idx]=dest)/WINDOW

The weight is used in the following filter to weight the current sample against the current estimate:

Figure 4.7 The 802.11 measurement block diagram

2.3.1 A Network Layer Implementation

In this section we propose an implementation that requires minimal support and knowledge of the link layer, and is suitable for the network layer. Errors are introduced in the measurement due to unknown parameters for the network layer. Most errors however can be filtered out or tolerated. One of the main issues dealt with here is the subtraction of queuing delays in the window operation (see Figure 4.8). In order to simplify the implementation description we first introduce the assumption that a single notification per packet is provided, and later we discuss how to deal and/or detect violations of this assumption (802.11 may have duplicate ACKs and L2 drops packets that cause overflow without notification).

The necessary variables and data structures are

· LastPacketTimestamp[WINDOW_SIZE] : a vector used to keep the timestamp when a packet leaves the network layer

· LastPacketPayload[WINDOW_SIZE]: a vector used to keep the useful bits of the above packets

· VQ: Contains the number of packets sent. It is incremented (modulo at least WINDOW_SIZE) each time a packet is sent.

· NTF: Contains the number of notifications received. It is incremented (modulo at least WINDOW_SIZE) each time a notification is received,

· PrevNTFTimestamp: Contains the timestamp of the last notification,

· SampleDelays[RECEIVERS]: Contains the sum of delays measured for packets to RECEIVER,

· SampleBits[RECEIVERS]: Contains the sum of bits for the above packets.

We also use Window_duration: the duration of the current window and Idle_time: the idle time in the current window for utilization measurements.

ON PACKET SENT()

VQ++;

LastPacketPayload[VQ]= <packet bits>;

LastPacketTimestamp[VQ]= now;

If Queue was empty =(VQ - NTF==1) {

Channel has been idle since last notification

Idle_time += now - PrevNTFTimestamp;

Window_duration+=now-PrevNTFTimestamp;

}

ON_NOTIFICATION_RECEIVED()

NTF++;

If (ACK received)

SampleBits[Notif.source]+=LastPacketPayload[NTF];

This_Packet_Suffered_Queueing =

(LastPacketTimestamp[NTF] < prevNTFTimestamp);

If (This_Packet_Suffered_Queueing)

Delay = Now - prevNTFTimestamp - C;

Else

Delay= Now-LastPacketTimestamp[NTF]-C;

SampleDelays[Notif.Source] += Delay - C;

Window_duration += Delay;

MEASURE()

For All Measured Receivers R:

Link_Throughput=SampleBits[R]/SampleDelays[R];

If ( Test_Measurement() ) {

PermissibleT[R]= idle_time/Window_duration *

Link_Throughput;

Update_Tables();

}

Figure 4.8 An example packet arrival/ acknowledgement sequence. The network calculates the contributed delays by using the notification series to emulute the link queue.

Note that the implementation is simple and does not require significant memory. The per-destination vectors can be constant in size regardless of the size of the network while the packet payload and timestamp vectors depend on the window size. At worst, information that cannot be held into the data structures can be discarded since this just deprives the measurement of the current sample.

In order to make the algorithm work efficiently outside the link layer we need to be robust in situations where a notification is sent more than once or when a notification is not sent at all. The former is rare but may occur in 802.11(duplicate acknowledgement), while the latter occurs when the link layer queues are full. Both of these situations are detectable but only after few packets. Even if sanity checks do not fail for erroneous measurements they can be eventually filtered out.

For the propagation of values, we use a distance vector based propagation that piggybacks its values to existing periodic routing broadcasts. In the propagation we are interested in discovering the minimum available bandwidth along the path (bottleneck link). In order to allow for efficient updating of the tables, we use both the measurement (expressed as negative available bandwidth as a cost) and the hop count to the discovered bottleneck link. This is necessary to detect bottleneck point changes and cancellations. In [Kaz01b] we extended AODV to support event-driven on demand propagation of measurements.

2.3.2 Simulation Correctness

In this section we use GlomoSim [Tak99] for two simple simulations that show the correctness of our measurements. We show the measured throughput, the available bandwidth and the consumed, defined as the difference between the two. The latter must always match the source throughput if the measurement is correct. The environment is the same as for the remaining experiments (1Mbps links).

In the first experiment we use a CBR connection of 182Kbps rate using packets of 480 bytes. In Figure 4.9 we see the throughput measurement to be close to 1Mbps and can even distinguish the periodic broadcast messages in the throughput measurement. Combining with the utilization measurement we get a available bandwidth that is 200Kbps less than the measured throughput. This shows that our measurement is very accurate under low load conditions (the 18Kbps difference is transport overhead, RTP header etc.)

The above experiment was with one hop CBR traffic and consequently no link queuing. In the next experiment we use a video source to examine our VBR case. In Figure 4.10 we see the source rate averaged every 32 packets. The video traces are taken from the Star Wars trailer [Star00] encoded in Intel's implementation of H.263 at 100Kbps target rate.

Figure 4.9 Exact measurements for CBR

Figure 4.10 The VBR source rates in time averaged over 32 packets

Figure 4.11 Accurate measurements for VBR traffic.

As we see in Figure 4.11 the measurement has very similar magnitude and shape and the consumed bandwidth follows accurately the offered traffic. In general these simulations show that we have a measurement accurate, fast and robust. We go on exploring the behavior of such a feedback, and show that its attributes are good enough to perform both adaptation of multimedia sources and call acceptance control.

2.3.3 Adaptive Multimedia Simulation

In this section we use video and audio adaptive connections adapting either to periodic 1 second end-to-end feedback containing the RTP loss, or to the network feedback. The video connections are able to adapt to 5 rates of an H263 encoded video stream. The traces have been gathered from the Star Wars trailer clip encoded in 256, 128, 80, 64 and 48 Kbps average target rates. The smoothing process is limited to one frame time and uniformly spreads the frame bytes into 200 byte payloads into the inter-frame interval. The audio is encoded at 9 encodings of 256 to 8 Kbps (as in [Kaz99b]). The initial rate is set to a low rate of 64Kbps. When adaptation is based on network feedback the server tries to match half of the reported available bandwidth by choosing the appropriate rate for the CODEC. In general the two mechanisms use values of different nature. We have attempted to use as similar add/drop strategies as possible. In this comparison, we are downgrading the CODEC rate proportionally to the loss rate observed. For example, if we observe a loss of 60% while the maximum tolerable loss is 15% we will attempt to downgrade 4 layers. This is a pessimistic approach that avoids over-sending and limits the loss rates further for the end-to-end case, resulting in a more fair comparison. The emphasis is in the measurement rather than the add/drop strategy in this way. As we see, even so, the end-to-end loss rates remain high.

In a mesh topology and connectivity of 36 (6 by 6) nodes we distribute connection pairs throughout the network. The hop distribution is (20%, 25%, 25%, 15%, 10%, 5%) for 1 to 6 hops respectively. Each experiment has 12 connections either active throughout the experiment or starting/ending 20, 40 or 60 seconds late/early. 4 experiments that contain adaptive audio and video connections are reported. The aggregate loss percentages are shown in Figure 4.12 for the network and end-to-end cases. In most cases, network stability has been maintained, but only the network mechanism managed perceptually acceptable loss rates (ranging from 1 to 8 percent). The end-to-end case in a few cases has even failed to uphold its maximum loss rate threshold (15%).

2.3.4 Call Admission Simulation

Next, we use similar scenarios but using higher rate connections so that all connections cannot be allowed in the network even with adaptation. We also use only video connections since passive measurements with VBR sources are more challenging. We do this to test whether the stability of the network will be maintained by a simple call acceptance strategy. On these experiments, the adaptive connections request a rate before entering the network or when a layer addition or drop is performed. Once they start they periodically query the path available bandwidth (through an API). They initially request the rate of the second lower rate (64Kbps plus their overhead). The network allows them to enter if there is at least double the bandwidth requested in order to maintain stability and fairness.

Figure 4.12 Overall loss rates (%) per experiment

Figure 4.13 Total bytes sent vs total bytes received with call acceptance and without.

The experiments show clearly that even a simple strategy and implementation of call acceptance based on our measurement can significantly improve the overall network throughput. Figure 4.13 shows that the amount of bytes received in the network is equivalent between the cases of call acceptance or no call acceptance. This, and the fact that loss rates for the call-acceptance case adaptive connections were small, shows that the call acceptance process found a good and stable operation point. The network resources were correctly utilized close to their limits. Based on available bandwidth and its propagation and working in parallel with source adaptation, it was precise in finding the network limits, since it managed to sent as much data as would be received.

In all of the above experiments we see that the network feedback mechanism provides strong control capabilities at the end points of the network. The values of available bandwidth propagated are robust and accurate enough for performing not only source adaptation but also call acceptance control.

2.3.5 Conclusion

We have successfully developed an architecture that supports accurate available bandwidth explicit feedback to multimedia transports and call admission applications. The motivation is for the architecture to base a cost/benefit analysis of end-to-end measurements versus lower level explicit feedback in last mile 802.11 networks. We also dealt with deployment issues by providing a network layer implementation of the link-by-link measurement. We tackled common measurement problems effectively, by dealing with them on a link-by-link source-destination specific basis. Only simple techniques were necessary to produce a statistically robust, correct and interpretable measurement, in contrast to common end-to-end experience. After making available the bottleneck available bandwidths at endpoints of a multi-hop ad-hoc network, we compared the network feedback adaptation to end-to-end and found it four times more effective in reducing the RTP loss rates in our experiments. Finally, we used them to direct a simple call acceptance strategy and showed that it very accurately found a stable, high throughput and low loss operating point. Therefore, single link measurements and feedback present a promising approach for soft QoS support in multi-hop ad-hoc networks.

Scalable, simple, deployable and cost-effective end-to-end approaches should be further studied. Direct end-to-end measurements, particularly other than loss observation and ‘trial-and-error’ based approached, should be evaluated in accuracy and resulting network goodput in networks with wireless links. The architecture studied here may provide a reference for such a comparison.

2.4 The Bluetooth Available Bandwidth Support

In this paragraph we develop a link by link measurement for Bluetooth nodes. Our goal is to to develop a measurement that is orthogonal to other chip functionality and would work equally well in all Piconets, Scatternets, Masters and Slaves. We accomplish this by relying on measurement of the Inter-Poll time and the type of packets used for the data communication (e.g. DH3, DH5 etc.). A Master node keeps one inter-poll delay estimate per slave and a slave one per Piconet it belongs to. It also keeps the average length of the packets sent, the Masters to the slaves, and the slaves to each Piconet Master that may belong to. For example, if there is no data a zero is counted or else a DH1, DH3, DH5 packet 27, 185 or 339 bytes are counted respectively, according to the Bluetooth specification. In this way we get an estimate of the available bandwidth that is based on the actual traffic of our neighbors that is causing a delay in the interpoll time. The available bandwidth to a node then is:

The first term represents the full throughput between a node and a master or vice versa. In that case one 5-slot packet can be sent at best every interpoll interval. From this throughput, the average length of packets sent per poll cycle is the consumed part. The above estimates are input to a digitization filter as before.

With the above mechanism any polling mechanism and IPS algorithm can be accurately supported. Implementation of this mechanism requires Masters to keep information for 7 Slaves, and Slaves for few Masters as many as they are allowed to gateway to. One entry of the above is 2 timestamps, 3 estimates, 2 samples, and a few lines of code as shown below:

As is evident from the above the available bandwidth lower layer measurement is very simple in Bluetooth both in space and time.

3 The End-to-End RTP based Architecture

This option is the most commonly used one, when congestion related multimedia adaptation is performed. It is implied in the RTP RFC and is a ‘trial and error’ approach. The application follows the principles of Application Level Framing (ALF) [Cla90]. Here is a reference architecture for this model.

A network-independent QoS notification programming model is implemented to provide the server with the ability to monitor network conditions at the client end and react adequately when congestion occurs. The model is designed so that future evolution of network layer services will not affect its generality and is comprised of three main parts: 1. the network layer API which provides the interface to network services, 2. the Network Monitor module which collects and analyses QoS information from the network, and 3. the Application QoS Notification API which accepts this QoS information from the Network Monitor and processes it. Such separation of the model into three modules allows the application to achieve the desired network independence.

The Network Monitor provides a quality of service abstraction for the application, so that it can always assume that the network provides QoS support, while in reality it may not. Its activity consists of the following three parts: 1) it monitors the multimedia stream from the server to the receiver; 2) it measures QoS parameters of this stream; and 3) it reports this QoS information to the application.

The protocol used to stream data to the receiver is similar to RTP but contains only the necessary functionality. The server generates the sequence number and timestamp information when it sends a packet over to the client. It stores the two numbers in the audio packet’s transport header. When the packet arrives to the destination, the receiver extracts these parameters from the header and passes them to the Network Monitor.

3.1 Development of an Adaptive Audio Client/Server

This work was inherited from two master students that have worked on the application architecture and implementation in the past. In an ILP manner an audio server and client has been programmed in Visual C++ to work over IP and support rate adaptation by keeping multiple files of the same recording, recorded in different qualities. Another student, who, under the supervision of TsuWei Chen, added the notion of text captioning, continued this work. The text was to be used at the client side in order to provide robust communication in case of high error rates. Since then this application became the basis for developing the end-to-end models for this work. After adding a more standard conforming RTP programming and real time speech recognition I tested different end-to-end feedback mechanisms replacing the simple TCP-based feedback which very frequently posed problems.

3.1.1 Speech Recognition Extensions

In order to improve end user perception, an ultimate encoding layer is introduced when AudioTool is used to stream speech audio. A text transcription is associated with the speech stream, either real-time using a speech recognition engine or from a caption file. The text traverses the path easier (smaller packets, very low bit rate) and more reliably (e.g. using redundancy). The data can be displayed at the receiver side in a caption window or, more importantly, when the QoS feedback suggests switching to this bottom layer, the speech can be reproduced using the transcription with a text-to-speech synthesizer. In this way the speech stream can still be comprehensible at the client side. Switching into this layer must be properly indicated not by the adaptation process at the server but from the client monitoring process, locally. It must be triggered at the point where the layer above it, would not produce comprehensible speech. Since it is a layer much more uncorrelated with the rest of the layers it presents in full all the issues related to synchronization and thrashing (frequent switching). Experiments in our wireless multi-hop testbed show that the communication quality is substantially upgraded even when adverse network conditions persist for a long time.

Our application allows the receiver to be inside a moving vehicle. The driver can listen to the transmission without ever having to look at the display. Such application becomes useful in a combat or emergency situation when information lines are down but wireless LANs can be quickly deployed. Think of the ambulance driver receiving traffic information from other drivers and/or helicopter.

When the audio stream is real-time speech, a text transcription is generated using a speech recognition engine (or read from a file in case of pre-transcribed speech) and sent redundantly piggybacked to the audio packets. Every audio packet, sent within a time window of 2 seconds contains the text recognized in the last 2 seconds. Even if only one audio packet gets through every 2 seconds the text -to-speech synthesizer will be able to produce meaningful and continuous speech at the client side.

In order to support real-time transcription of speech Microsoft Speech Recognition Engine version 3 is used to transcribe discrete speech on the fly. Discrete speech, unlike continuous, requires a speaker to pause between words. A speech recognizer cannot be expected to work perfectly for our application without training and fine-tuning of its parameters. Our system ideally requires continuous dictation and infinite vocabulary, but speech recognition technology is not in a position to supply this yet.

In order to get sufficient performance out of the speech recognizer we use a limited domain grammar [MicSR] made up of approximately 1000 words and no grammar rules. Currently user-defined vocabulary training is not supported. Because the speech recognizer works much better for the trained words, our grammar includes words that are used by Microsoft’s test sets. In general, application specific grammars and/or vocabularies can be used to limit the speech recognition engine’s possible responses so that the resulting accuracy is acceptable and our captioning scheme can depend on them to reproduce the speech. For example a system used to transmit routing information in a specific city has a naturally limited vocabulary.

We use Microsoft's text-to-speech synthesizer contained in the Microsoft Speech 3.0. The synchronization between the bottom layer and the above is important. Since a real speaker may speak with highly variable speed, a speed adaptation algorithm is needed to keep the speaker and the synthesizer in sync. In our implementation, we calculate a step to increase or decrease the speed of the synthesizer every 2 seconds. When the speech is pre-transcribed the synchronization can be performed manually (pre-synchronization). In our application we insert a special character in the caption file to denote the desired 2-second windows. This simple solution produced good results and decreased our development time. Some further issues relevant to TTS synthesizer use are:

· How to set the threshold: we expect that a dynamic computation of the threshold will produce better results. It must be set in such a way that the switching between the speech synthesizer and the speaker is not frequent. We currently use a run-time user defined threshold to avoid extensive oscillations of the measured loss rate around it and thus avoiding switching on and off the TTS. Smoothing of the loss rate measurement is also applied before comparison to this threshold. For a more detailed discussion see [Kaz99c].

How to measure the loss rate in the presence of silence: it is desirable not to send "silent" packets. In real-time speech this means however that measuring the loss rate will not be straightforward since we do not have an expected rate. The first packet received may be used to indicate the start of valid audio transmission. In our implementation, since we are primarily interested in observing the quality of the speech, we simply ignore the 100% loss rate measurement whether it comes from silence or real loss. This results in less accurate QoS measurements but it is much simpler.
The text-to-speech synthesizer has features that can help in the case of a multimember conference (multicast). A receiver can recognize or distinguish the speakers, when the text-to-speech synthesizer is on, by having the application manipulate parameters like the pitch of the speaking voice, personality of the voice (e.g. business, casual, computer, excited, singsong), gender of the voice and age of the voice.

· Multiple Encoding Synchronization - The synchronization between the two very different encodings brought about difficulty that was initially overcome by the introduction of a speed adaptation algorithm. The idea was that since we already intrinsically have window granularity synchronization applying a dynamic speed (Words Per Minute) adaptation of the text to speech within the window would produce good enough results. When we switch down to the bottom layer in real-time speech though, things are not so simple. Every packet contains the words recognized in the last 2 seconds. It is evident that the TTS will normally try to speak parts of the transmitted sentence more than once. Ideally, the TTS engine must be able to speak a word in the list exactly when its audio packets finish. In our early implementation we tokenize the phrase into words and feed to the TTS only what “appears” to be a newly spoken word. This produces fully comprehensible speech except for the case of communicating discrete digits with audio (e.g. one-one-three will sometimes be spoken one-three). Later, I collaborated with two graduate students who took interest in this problem and implemented a synchronization algorithm by direct association of the two heterogeneous streams. It required an end-to-end protocol co-ordination. Using low level speech engine functionality we extracted offsets in the audio stream for each of the words recognized. This offset was then sent piggybacked in the corresponding audio and text packets and used at the client side for the switching in the layer and out of it. The client, because of the media scaling properties of the PCM codec was able to use this information even in an adaptive application with variable bit rates by properly scaling the offsets.

3.1.2 Testbed and Environment

Our multi-hop wireless testbed consists of a server, a client, a gateway and one or more interfering stations as shown in Figure 8. Both server and client run on NT/95 platform, whereas the gateway runs on Linux. A WaveLan I is used for wireless networking. In order to get reproducible network conditions, both server and client can run on the same machine (without a gateway) and packet loss and delay jitter values can be simulated. In this paper we present experiments with a real multi-hop testbed.

Figure 5.1 The topology used in the experiments. Three subnets are used to create a multihop connection. Two subnetworks are used for the server and the client respectively. One is used for the interference. The gateway has virtual interfaces to all subnets.

The server takes input from a microphone and sends speech in real time to the client via the gateway. Audio packets are sent to the client via UDP. The client application on the other end receives the audio packets and plays them out on the audio I/O device. Alternatively, the server can get the input audio stream from a file. This allows for conducting experiments easily. The playback takes place in real-time following an initial buffering delay. When the audio stream is speech, a text transcription is generated using a speech recognition engine and sent redundantly piggybacked to the audio packets. When the server streams a pre-recorded audio file, the caption is taken from a file (pre-transcribed speech). Every audio packet sent within a time window of 2 seconds contains the text recognized in the last 2 seconds. In order to easily conduct experiments a pre-recorded speech sample is manually transcribed in an accompanied text file. In this way the same input text stream can be used in different experiments. The text is separated in 2 seconds time windows.

When the speech is pre-transcribed the text is broken into 2 seconds windows and the whole window is sent with any audio packet generated in this time frame. Even if only one audio packet gets through (uniformly over time) every 2 seconds the text-to-speech synthesizer will be able to produce meaningful and continuous speech at the client side by using the text stream. When the audio stream is sent at 8KHz (8000 samples per second) mono (8-bits per sample) (7 Kbytes/sec) packetized every 240 bytes –as our adaptation process indicates for constantly high loss rates- approximately 50 packets are sent in the window of 2 seconds. This means that the caption is replicated 50 times. With an (arbitrary) average of 18 bytes of text per 2 seconds the replicated text stream reaches approximately 520 bytes/sec. These numbers can be changed to meet any audio stream to text stream bandwidth ratio or reliability requirements.

The client constantly monitors number of packets lost as well as the delay jitter, and periodically sends QoS feedback to the server via TCP. The server switches between layers (sampling rates) of the audio stream and changes the packet size dynamically based on the QoS information received. When the packet loss rate is over a defined threshold the client uses the text-to-speech (TTS) synthesizer to reproduce the speech. Note that the switching between TTS and the audio stream is performed at the client side using local, up-to-date information. It does not depend on the QoS feedback channel. Its behavior and efficiency is affected by the QoS adaptation.

Any adaptation procedure can be easily incorporated in our programming model, as well as any layering/encoding. The purpose of these experiments is to validate that adaptation is important in improving quality of perception of multimedia data in wireless networks, and, that when the audio stream is voice our caption embedding scheme further improves end user perception.

3.2 Real 802.11 / Wavelan experiments

In the experiments we have used WaveLan NICs to connect portable devices through PCMCIA. As in ad-hoc networks the nodes operate on a peer-to-peer level with no access point or wired system.

Table 5‑1 Real Testbed parameters

Frequency	902-928 MHz (915 Mhz)
Modulation Technique	SS DQPSK
Data Rate	2Mb/s
Media Access Protocol	CSMA/CA
Bit Error Rate	<10^-8
Range:
Open configuration.	180-240m 600-800ft
Semi-Open configuration	60-120m 200-400ft
Closed configuration	27-33m 90-110ft

Carrier Sense and Multiple Access with collision avoidance (CSMA/CA) is used in WaveLAN I as the MAC layer protocol. If the channel has been idle for longer than t_long, seconds, the sender waits for random time, t_rand and then transmits immediately, otherwise, it waits until the channel becomes idle for t_long + t_rand. After transmitting a packet the sender monitors the channel to make sure its transmission does not collide with others. A collision may still occur if two stations transmit a packet simultaneously. If a collision is detected, the stations involved will back off for a random period of time, according to the binary exponential algorithm. In addition to the collisions caused by two nodes transmitting simultaneously, the limit of radio transmission range unveils the effect of hidden terminal which causes collisions that can only by detected by the receiver but not by the sender. Without using a more sophisticated collision avoidance scheme (e.g. RTS/CTS used in 802.11) in MAC layer, this type of collisions further degrade the channel throughput, since the damaged packets must be handled by retransmission in either the MAC or the transport layer

Currently Microsoft Speech Recognition Engine and API version 3 is used to transcribe discrete speech on the fly. Discrete speech, unlike continuous, requires a speaker to pause between words. Experiments with continuous speech are currently under way. Speech recognition technology is rapidly improving. However, current state of the art Automatic Speech Recognition cannot be expected to work sufficiently without training and fine-tuning of its parameters. The most important performance boost for the speech recognizer can be achieved by training or better yet, training for a specific grammar and/or vocabulary. In a speech recognizer, it is desirable that parameters like the following are programmable through an API. Recognition accuracy and response is affected by those. This indicates that ASR accuracy is highly dependent on the specific environment it operates: microphone type and auto-gain, expectation of echo canceling, expended energy of noise, percentage of processor time that the engine expects to use during constant speech, speaker recognition, confidence level, time that the speech-recognition engine waits before discarding an incomplete phrase or regarding a phrase as complete after the user has stopped speaking, etc. In order to get sufficient performance out of the speech recognizer we use a limited domain grammar [MicSR] made up of app. 1000 words and no grammar rules. Because the speech recognizer works much better for the trained words, and user-defined vocabulary training is not supported, our grammar includes words that are used by Microsoft’s test sets. In this way, we can achieve an almost perfect (100%) hit rate. In general, application specific grammars and/or vocabularies can be used to limit the speech recognition engine possible responses so that the resulting accuracy is acceptable and our captioning scheme can depend on them to reproduce the speech. For example a system used to broadcast routing information in a city has a naturally limited vocabulary.

We use Microsoft's text-to-speech synthesizer contained in the Microsoft Speech Suite 3.0. In a text-to-speech synthesizer, parameters like the following may be used to improve speech comprehension and synthesizer to real speaker synchronization: percentage of processor time that the engine expects to use during constant speech, baseline average speed, the word that is currently being played and/or the exact byte in the audio stream that is currently being played, etc. Features that support security and multimember conferencing in a TTS synthesizer are programmable parameters such as pitch of the speaking voice, personality of the voice, gender of the voice and age of the voice. By encoding the text and these parameters, listeners can determine whether the information comes from a known trusted source. The synchronization between switching the reproduction of speech from the text stream and the audio stream is very important in improving end-user perception. Some of the problems encountered are different when the speech is pre-transcribed rather than real-time. When the speech is pre-transcribed, recall that every packet in the 2 second window contains the caption for these 2 seconds. When the speech is real time the text piggybacked to each packet is the text recognized in the last 2 seconds of speech. The synchronization problem is particularly difficult when using pre-transcribed speech. We use pre-transcribed speech for easily contacting our experiments but the system is more likely to be intended for real-time speech. However, by using a TTS speed adaptation algorithm, the synchronization between the two layers is sufficient for meaningful speech even with pre-transcribed speech. Another issue relevant to TTS synthesizer use is setting the threshold; we expect that a dynamic computation of the threshold will produce better results. It must be set in such a way that frequent switching is avoided. Due to the CSMA/CA MAC protocol intense traffic will cause highly variable loss rates over time in a real-time application (see next section).

3.2.1 CSMA Speech Scheme Real Testbed Results

Figure 5.2 : Loss Rate when no adaptation is performed, and with standard experiment interference. Audio stream is packetized in 960 byte packets and sampled at 22000 samples per second and 8 bit per sample. Interference is ‘ping’ packets of 40 bytes attempting to fill the channel

Figure 5.3 Visualization of text synthesizer use: switching between audio (even number) and text stream (odd number) with different TTS-thresholds in first experiment

Extensive experiments with pre-recorded and pre-transcribed speech have been performed. They show that incorporating transcription and using a text-to-speech synthesizer at the receiver end allows for more comprehensible speech streaming. These successful experiments indicate that similar results can be expected when the transcription is produced by a speech recognition engine on the fly. Our goal is to sustain an audio quality high enough for meaningful communication of real-time speech. In our lab we can hear the differences. In this paper for each experiment we show the measured loss rate over time. A graph indicating the switching between the audio and the text stream accompanies the experiment. We show three experiments, one without using any adaptation mechanism which would require a QoS feedback path, one with using adaptation in the same environment and one with adaptation and more intense interference to emulate behavior in extremely adverse conditions. In this last environment adaptation is not enough to prevent corruption of the audio stream. Even at 8Khz and 240 bytes payload the loss rates are more than 70% most of the time. In these experiments the interference –generated by a node as seen in Figure 5.1- is intense in order to better evaluate our scheme. The interfering station is trying to fill the channel with ping packets of 40 bytes in the first two experiments. In the third experiment in order to achieve greater interference a ping packet of 160 bytes is used to fill the channel. Furthermore the distance between the stations is increased so that the increased propagation delay will prevent the CSMA/CA from successfully detecting the busy channel. In this way the number of unsuccessful packet transmissions is increased, resulting in a highly lossy environment. In Figure 5.2 we can observe the loss rates measured by the application’s QoS monitor at the receiver side. The MAC layer prevents any node from constantly transmitting, resulting in a highly variable and oscillatory loss rate for our real-time application. When the loss rate is high, the ping packets use the channel. The audio server senses the channel, finds it busy, and according to CSMA/CA delays sending the packets to the client. The client considers the packets lost when these arrive later than the playback point. In Figure 5.4 we show the loss rate when adaptation in sampling rate and payload size is on. Since the interference is intense, our adaptation process switches to sending 8Khz audio packetized in 240 bytes, as opposed to 22Khz and 960 byte packets. The adaptation process used is similar to the one we describe in [Kaz99c]. Here the loss rates are smaller (averaging 13 as opposed to 40) and the oscillatory nature is not present. The latter is due to the change in payload size for the audio stream. The ‘ping’ (interference) packet of 40 bytes and the audio packet of 240 bytes are now comparable and have a comparable probability of getting the channel and being transmitted. Furthermore the 8Khz audio stream will require the channel less frequently than the 21Khz in the first experiment. This results in more packets reaching the client before the playback point. In Figure 5.6 the loss rate observed with the high interference. The 8Khz, 240 payload audio stream is corrupted at these high loss rates averaging 77%. In this environment our captioning scheme can sustain meaningful communication. With almost cut-off loss rates of 99%, still one packet goes through in the 2 seconds time window and the conference is kept alive.

The remaining graphs (Figure 5.3, Figure 5.5 and Figure 5.7) contain the same information. They show clearer the switching between the audio and the text stream for different threshold values. A ‘high’ on each line indicates that the speech is reproduced from the TTS synthesizer. Different lines are shown corresponding to different thresholds from 10% (lower line) to 40%. In Figure 5.3 we can observe the behavior of multiplexing the layers when the loss is oscillatory (experiment with no adaptation). In Figure 5.5 the switching is much more frequent resulting in a degradation of the level of perception. Figure 5.7, where the loss rate is constantly very high even with the audio at 8KHz and 240 byte payload, shows that our scheme produced a very high level of perception in an environment where the audio stream would not comprehensible at all.

Figure 5.4 : Loss rates when adapting to QoS, and with standard experiment interference. Audio stream is packetized in 240 byte packets and sampled at 8000 samples per second and 8 bit per sample (due to the adaptation) . Interference is ‘ping’ packets of 40 bytes

Figure 5.5 Visualization of text synthesizer use: switching between audio (even number) and text stream (odd number) with different TTS-thresholds insecond experiment

Figure 5.6 : Loss rates when adapting to QoS with extremely adverse network conditions. Audio stream is packetized in 240 byte packets and sampled at 8000 samples per second and 8 bit per sample. Interference is ‘ping’ packets of 160 bytes attempting to fill the channel. The distance between the stations here is larger and the larger propagation delays result in higher loss rates

Figure 5.7 Visualization of text synthesizer use: switching between audio (even number) and text stream (odd number) with different TTS-thresholds in third experiment

We presented a way to stream real-time speech over very high and highly variable loss rates as those observed in networks with wireless links. Our goal is to keep a meaningful voice communication even when the sender, the receiver or intermediate hop stations are in a high interference zone. Our experiments with pre-transcribed speech show a very significant improvement in user perception. When the loss rate is constantly very high the voice conference is kept alive by the redundantly sent captioning of the speech. As speech recognition technology improves, using a text stream as an ultimate encoding layer of real-time speech will be possible without the present restrictions.

3.2.2 Issues in Payload adaptation

Adapting on packet size was another focus of experiments because transport packet sizes affect quality and network response. The quality is affected in two ways: a) codecs oftentimes need block information to start decoding. This implies that the packetization may take this into account so that blocks arrive as coordinated as possible. b) with a UDP base transport the packet size determines directly the amount of information missing from the decoder on one packet miss. This is even more important when one considers that a missing packet from the decoder will have further effects on the perception due to temporal dependencies in the compression (typical codec trade-off). According to ALF the application unit should be in accordance with transport and mac unit. The packetization at the application layer of a coded stream is done in accordance with lower layer transfer units. In fact, in our experiments we off-line noted the maximum layer units and assured no fragmentation.

Let us mention a few observations from the results. Graphs and more details can be found in the respective publications [Kaz99c][Rom98][Sla97]. The adaptive schemes react promptly to packet loss and jitter, leading to adjustments in sampling rate and packet size which improve performance and drive the system to the most effective operational regime.

· The heavy loss periods are much longer in the non-adaptive case than in the adaptive. Long audio ``black outs'' (several seconds) are extremely disruptive.

· Packet size adjustment appears to be particularly effective in reducing loss rate This was expected since random channel interference is the major problem in our experiments. In these conditions, the shorter the packet, the lower the probability of corruption. Recall that we have limited control on external interference in our experiments.

· The adaptation of both packet size and sampling rate provides the best performance. Height and width of loss bursts are the smallest in this case.

Interestingly, the traces also seem to show an apparent correlation between the observed jitter and packet loss at the receiver. The jitter follows the same tendencies as the packet loss, so that its increases and decreases follow those of the packet loss curve. This observation adds extra justification for our adaptation scheme. Although our scheme explicitly manages only the packet loss on the channel and does not directly attempt to decrease the jitter, it does, in fact, appear to improve both of these parameters because of the correlation that exists between the two. We noted a considerable upgrade in sustained user perception. The speech synthesizer could take over and deliver meaningful speech to our ears continuously even when the loss rate was insisting on 99%. Theoretically we need at least one packet to go through in every 2-second window and this was what we experienced in practice too. Only when the interfering station was attempting to fill the ‘ether’ by pinging the gateway (using the -f option and 960 byte packets) did it completely capture the channel and noticeable intervals and “black-outs” occurred in our speech stream. The loss rate at those times was reported 100%.

3.2.3 Larger Scale Real Testbed Experiments

Figure 5.8 Topology in Large Scale Real Experiments

Figure 5.9 Loss Rates in Clients with FTP interfering traffic

Figure 5.10 Loss Rates in Non-Adaptive Clients with FTP interfering traffic

Figure 5.11 Loss Rates in Clients with only non-TCP intefering traffic

These experiments show results that are both in keeping with our previous in the lab shorter scale experiment result and in our simulation experiments. Specifically, notice the first case where client1 and 2 both adapt but compete with ftp traffic in some routers. The two connections receive an unfair part of the bandwidth. One manages to have considerable lower loss rates than the other. This is in keeping with hybrid simulation results, which suggest that when real time traffic is mixed with TCP traffic adaptive flows get an unfair portion of bandwidth. However, when these do not adapt fairness in apportioning effective bandwidth is increased. In contrast, when the network is mixed with rate adaptive flows exclusively fairness is increased (see respective graphs). Another important observation is that besides average loss rates or averaged effective bandwidth the RTP loss rates will always exhibit a highly oscillatory shape. These results are presented here on order to validate the hybrid simulation results. During these larger scale real experiments a considerable amount of data was collected to be processed soon. During these two day experiments besides RTP statistics, we collected ip dumps (using tcpdump) in a number of stations as well as delay distributions measured by ping throughout the experiments. A typical ping delay distribution is shown in the following graph, where the delays of a wireless link are shown when no traffic exists and when traffic is pumped into the network. Additional traffic exists in the interval clearly distinguishing itself by larger delays. The x-axis is the increasing number of packets sent one per second at default size 100 bytes and the y-axis is the delay measured in msec. Ping uses ICMP echo packets and expects an echo-reply answer from the remote host. It is worth noting that a processing delay is always added to the link propagation delay since each packet requires the host to compute a different IP and ICMP checksums.

Figure 5.12 A single hop link ping delay graph

3.2.4 Intra-Protocol Fairness

In these experiments the server is able to encode the data in 9 different layers. These layers differ in bandwidth requirements, inter-send times and quality of the encoded audio as they are encoded at different sampling rates, using different compression ratios and possibly using different encoders. We have shown the payload size to be an application parameter that can affect lower layer response. From the same experience we note that accordance of payload size with codec application layer buffers may significantly simplify processing for some codecs that process data in blocks (for example ADPCM, CELP). The payload size also affects perception of audio, especially when the audio application runs on top of non specific lower layer configurations that normally apply their own error control mechanism and may not deliver partially damaged packets. In this work, in order to keep complexity low, we chose a useful payload size of 480 bytes to carry the audio in all layers. The 9 layers used are 180, 128, 88, 64,32, 24, 20, 16, 8 Kbps. As mentioned previously, this gives us a wide range of rates to adapt to and allow for generalization. We use a combination of robust uncompressed scheme with a widely accepted, audio-perceptual modeled low bit rate codec. They also allow us to use static values for the loss rate threshold of our adaptation mechanism as discussed in the next paragraph (and as opposed to a speech specific codec, for example, where quality degrades differently and in complex relations to FEC schemes etc).

In order to set the loss rate thresholds for our experiments we performed perceptual evaluation of the codec perceptual degradation to loss rates. Measurement of sound perception quality, in general, is a difficult problem. This is due to the inability of formalized methods to reflect the highly subjective perception of human ear. MPEG compresses audio by removing the redundancy of the audio signal using statistical correlation and by reducing the irrelevancy of the signal by considering psycho-acoustical phenomena like spectral and temporal masking. As such, it is not based on any specific assumptions for the audio stream –as the CELP codec for example assumes speech content. Based on the widely used metric of SNR (signal to noise ratio), for source coded PCM and MPEG-1 audio the perceptual degradation can be assumed linear to an evenly distributed bit loss rate in the time domain. There, this provides a base for their quality loss measurements for a processor-optimized MPEG codec. We carried out our own experiments that allowed us to listen performance degradation of the different layers used in our experiments due to packet loss. These experiments helped us decide on the maximum tolerable loss rate for our adaptation mechanism. As expected perception is significantly impaired at a 20% loss rate for the audio coding we use. The need to simplify our adaptation technique with a single static maximum tolerable loss rate influenced this last decision. In fact, a 20% loss rate in packets caused significantly lower quality in the lower bandwidth streams independently of codec used. This is due to the quantization caused by the 480 payload size which, in turn, caused an uneven bit loss distribution in the time domain as the number of packets expected per second decreased. This indicates that adaptation techniques developed for specific types of networks can gain in perception by introducing soft and dynamic rather than hard and static thresholds.

An important perceptual degradation when audio data are streamed over a network occurs from the non-uniform distribution of errored packets. In fact, it has been shown that significant quality is lost due the bursty nature of errors when non-adaptive MPEG is streamed over the internet [Yao97]. In that same paper a technique to interleave audio packets at the server is presented specifically to deal with this type of quality degradation.

The server packetizes (in RTP payload) and sends the audio according to parameters of the currently selected layer. In our implementation each packet contains a number (from 1 to 9), which indicates all the necessary parameters to the client who will perform the playback. The client uses this information to play out the packets using the specified parameters and calculate the RTP statistics as well. The loss rate is measured over an interval of one second and sent back to the server. The server uses the loss rate information to adapt its rate as in the procedure shown in Figure 5.13.

The QoS feedback packet itself, has to traverse the network back to the server, and the probability of it actually making it there on time is inversely proportional to its importance (we assume a QoS feedback packet to be important when it carries information about a congested network). In experiments performed in a real wireless testbed we concluded that a UDP transport is much more appropriate to carry the feedback than a transport using retransmissions like TCP. TCP retransmissions result in stale QoS information especially on a congested network. With UDP, a mechanism can be employed at the server so that when a few QoS packets are lost the server assumes a congested network. In our implementation we choose to downgrade the service whenever we detect TOTimesThreshold lost QoS packets every TOMaxTicks (see Table 5‑2 and Figure 5.13). A soft timeout mechanism is proved to be necessary because at high congestion feedback packets have a low probability of reaching the server. In a more complex adaptation mechanism, intuitively, an improvement in efficiency can be expected by dynamically computing these thresholds based on current bandwidth consumption and past experience. As mentioned, a more complex adaptation mechanism is not fitted for this work.

Figure 5.13 Adaptation mechanism

Table 5‑2 Adaptation mechanism parameter values

MaxLossThreshold	20%
MinLossThreshold	5%
TOTimesThreshold	5
TOMaxTicks	7

Table 5‑3 Configurations tested

*MAC*	*Network*	*Application*	*Mobility*
802.11	STATIC	NO-ADAPT	NO
CSMA	BELLMAN FORD	MIXED (ADAPT5)	RANDOM WAYPOINT 2.9km/hour
		ADAPTIVE (ADAPT10)

We experimented with non-adaptive audio and mixed traffic of adaptive and non-adaptive in order to produce clearer results. The different options tested are shown inTable 5‑3. A fair number of nodes is used in these experiments. Specifically 25 nodes were randomly placed in a 60 by 60 unit area, each equipped with a 2Mbps radio of 30 units range. A large number of experiments with randomly placed nodes was needed to increase confidence, applicability and generalization of the results. This instructed the 30 unit range provides reasonable probability of the generated topology being disjoint and connections will interfere with each other and share bottlenecks. 100 topologies were produced. The same 100 topologies were used in each experiment. All of them were inspected using our topology visualization tool –based on gnuplot. 10 different servers and 10 different clients were defined, either producing adaptive or non-adaptive audio. The only traffic in the network is these 10 audio connections. The resulting routes are mostly 1, 2, or 3 hop averaging at 1.8 hops. A large interval (100 seconds) in the beginning of the simulation allows the free exchange of bellman ford tables among the nodes. After that, the audio traffic is generated. In case of static topologies the routing table exchange is stopped. In the experiments with mobility the bellman-ford exchange of routing tables is allowed to go on. The mobility model is the random waypoint mobility model [Jon96]. It is a model widely accepted and used. The nodes randomly choose a destination point and move 1 distance unit towards it by randomly choosing an axis on which to move. In our experiments they take 5 seconds to complete the one move. Then they wait for 10 seconds and choose another destination. This gives an average speed of 720 units/hour. In all, this scenario resembles a real “low-mobility” situation. By accepting one unit to be 4 meters then we get a “real” open transmission range of 120 meters, where people (nodes) walk (move) at 2.9km/hour within an area of 240 by 240 meters. Initial experiments showed that the traffic cannot be carried sufficiently by the network at full rate (180Kbps each). In this way the effectiveness of adaptivity may be clearly shown while specific adaptivity issues are easily addressed. The configuration options across layers are tabulated in Table 5‑4.

Table 5‑4 Simulation Parameters

Simulation Time	400 sec
Propagation Function	Free Space
Radio Bandwidth	2Mbps
Radio Type	No Capture
MAC Queuing	FIFO

The validation of the first phase of the hybrid simulator is based on validating GlomoSim and the software built on top of GlomoSim. Validation of the application components has been performed by performing small scale experiments in both real and simulated networks. RTP loss rates in both have same shape and magnitude.

Figure 5.14Averaged client loss rates vs adaptivity

Figure 5.15 Averaged effective bandwidth vs adaptivity

Figure 5.16 Averaged server consumed bandwidth vs adaptivity

In general we note (for more details see [Kaz99b])

· Reducing the transmission rate at the server side increases dramatically effective bandwidth.

· With mobility, an average drop in a server rate of 60 Kbps (33%) results in a 100 Kbps more (500%) effective bandwidth (802.11, adapt all, mobility).

· For CSMA, a 100Kbps drop (55%) results in a 47Kbps (265%) increase in effective bandwidth.

Figure 5.17 Averaged Coefficient of Variations in Effective Client Bandwidth

In general we note:

· As connections become adaptive, fairness in effective bandwidth distribution among the real-time connections is persistently increasing.

· As mobility is introduced fairness both in effective client bandwidth and in server rates (not shown here) is increased.

· An 802.11 MAC results in a persistently more fair distribution both in effective client bandwidth and in server rates (not shown here) than CSMA.

In this work we adopted a simple application layer adaptive architecture for multimedia applications. The adaptation mechanism -validated and used in previous work on real multi-hop testbed- was fine tuned according to audio perceptual results. By using our hybrid simulation platform, we run extensive experiments in order to explore the behavior and effect of adaptive audio and speech connections in ad-hoc multi-hop wireless networks. 1200 experiments formed the basis for the conclusions in this paper. 100 randomly generated topologies with similar characteristics were used to clearly denote the trends. The traffic included mixes of non-adaptive and adaptive audio connections running over CSMA and 802.11. No mobility and a low mobility scenarios were used. Audio connections borrowed their characteristics from known widely used codecs. They adapted to 9 layers of different codecs and a rate range from robust PCM of 180Kbps to low bit rate MPEG audio at 8Kbps. We looked at loss rates, client and server rates and used their COV’s to explore fairness issues. The results were aggregated in 3 different aggregation levels.

At the configuration level we clearly noted the effect of adaptation on the networks under study. When all the audio connections adapt there is significant improvement when compared to non-adaptive traffic both in loss rates and effective bandwidth. At the experiment level, apart from looking at the absolute averaged loss rates, client effective bandwidth and server rates, we looked at fairness issues by looking at the COV’s of these in each experiment. Based on the similarity constrains of the experiments we compared those and found: (i) in multihop mobile networks over 802.11 our simple adaptation mechanism fairly distributed the rate at which servers should transmit. This became particularly evident when mobility was introduced. CSMA exhibited similar behavior but double COV’s than 802.11 (ii) as connections become adaptive, the distribution of effective bandwidth was consistently fairer especially for 802.11 networks. At individual connection level, we noticed closely scenarios with competing adaptive and non-adaptive connections and the end-to-end feedback effectiveness. In general, when a timeout mechanism exists to drop layers in the adaptation mechanism, the feedback can deliver a consistent view of the RTP loss rates at the server at low mobility and low hop count. The highly oscillatory nature of the loss rates in all cases studied indicated the necessity and effectiveness of an additional, local to the client layer switching. It has been shown to enhance speech perception and be necessary to sustain meaningful communications in such network conditions.

In general, we explored issues related to end-to-end adaptation for multimedia applications over multihop wireless networks, concluded on the very promising effects on perception, efficient bandwidth usage, and fairness, and formed a basis for comparison and understanding of adaptive connections in multihop networks.

3.2.5 Tcp and Udp Experiments with hybrid simulator

The same experiments described above have been performed with additional TCP traffic as well. Preliminary results show that there exist complex interactions in the mix of adaptive flows with TCP. In the next graph we note that effective bandwidth is not fairly distributed to adaptive multimedia connections in most cases. In fact, the coefficient of variation of effective bandwidth among receivers is oscillating in a high range among experiments.

Figure 5.18 Coefficient of Variation of Effective Bandwidth among clients (y axis) for 100 generated topologies (x axis) when TCP/FTP traffic exists

In Figure 5.19, Figure 5.20 and Figure 5.21 fairness in client effective bandwidth is increased (COV is decreased) when we adapt. These results are based on 1200 experiments performed and the COVs have a clear trend in all 100 experiments of the same type. In 802.11 with no mobility adapting in half the connections results in improved fairness (similarly as adapting in all). This is the only exception, since in other cases we see an improved fairness only when we adapt to all connections. This is not an “out of trend” result though. Consider that the adaptation mechanism works well when the feedback works well. When the feedback travels in a congested –not releaved from adaptive connections- network, the result is a less responsive and less network-aware servers. The case of 802.11 is the one that constantly throughout the experiments delivered increased effective throughput. This happened naturally in the non-mobile experiments as compared to the mobile. The case of 802.11 with no mobility is the case where the adaptation mechanism works best. This is why when half the audio connections adapt we were able to notice the fairness effects noticed usually when all the connections were adaptive. See also Figure 5.14 (client rates) where 802.11, non mobile is by far better in the ADAPT5 case of interest. These graphs also show that CSMA is in general more unfair and less efficient than 802.11 (dominant COV in CSMA is 3 whereas in 802.11, 2 in non-adaptive cases). However making the connections adaptive has an equal proportionally positive effect on fairness in both MACs.

Figure 5.19 Coefficient of variation of client rates of the 10 connections (y axis) for all 100 generated topologies (x axis) with 802.11 MAC and no mobility

Figure 5.20 Coefficient of variation of client rates of the 10 connections (y axis) for all 100 generated topologies (x axis) with 802.11 MAC and mobility

Figure 5.21 Coefficient of variation of client rates of the 10 connections (y axis) for all 100 generated topologies (x axis) with CSMA MAC and no mobility

Figure 5.22 Coefficient of variation of client rates of the 10 connections (y axis) for all 100 generated topologies (x axis) with CSMA MAC and mobility

In all experiments, the server and the client had a similar view of the loss rates throughout our experiments. The feedback mechanism worked well even at the presence of low mobility and high loads.

3.2.6 Conclusions

When all the audio connections adapt there is significant improvement when compared to non-adaptive traffic both in loss rates and effective bandwidth. At the experiment level, apart from looking at the absolute averaged loss rates, client effective bandwidth and server rates, we looked at fairness issues by examining the COV’s of these in each experiment. Based on the similarity constrains of the experiments we compared those and found: (i) in multihop mobile networks over 802.11 our simple adaptation mechanism fairly distributed the rate at which servers should transmit. This became particularly evident when mobility was introduced. CSMA exhibited similar behavior but double COV’s than 802.11, (ii) as connections become adaptive, the distribution of effective bandwidth was consistently fairer (especially for 802.11 networks.) At individual connection level, we noticed closely scenarios with competing adaptive and non-adaptive connections and the end-to-end feedback effectiveness. In general, when a timeout mechanism exists to drop layers in the adaptation mechanism, the feedback can deliver a consistent view of the RTP loss rates at the server at low mobility and low hop count. The highly oscillatory nature of the loss rates in all cases studied indicated the necessity and effectiveness of an additional, local to the client layer switching. It has been shown to enhance speech perception and be necessary to sustain meaningful communications in such network conditions.

3.3 Adaptive Multimedia in Bluetooth Piconets

We explore the ability to support multimedia traffic in indoor, wireless ad hoc PANs (Personal Area Networks) using the Bluetooth technology. As we mentioned Bluetooth is an alternative technology to 802.11. The simulation study based on a set of representative traffic scenarios. One of the main goals was to evaluate achievable Bluetooth throughput taking into account interference between different coexisting piconets. The simulation environment used in our experiments is NS-2 [Nsm02]. NS-2 already includes several wireless network models. In particular, it supports the IEEE 802.11 WaveLAN standard. We have augmented NS-2 with the Bluetooth model. The Bluetooth model has support for defining multiple piconets which may overlap with each other causing interference. The model contains most of the standard features of Bluetooth like Frequency Hopping, Multi-Slot Packets, Fast ARQ (Automatic Retransmission Query). It also contains a channel and collision model for an indoor environment.

Our aim here is to compare the performance of Bluetooth and WaveLAN in a totally adhoc environment, where no infrastructure in the form of access points is available. This would typically model the scenario of a large conference, where a number of Bluetooth or WaveLAN devices may be talking to each other. The traffic in such a scenario is heterogeneous and multimedia in nature, i.e., TCP, voice and video. It is assumed here that any two devices wanting to communicate are close enough to be in the same piconet and thus communicate through the master. This will be a realistic model for ad-hoc group collaboration where members of the same team will be sitting nearby and will interact with each other by exchanging files and engaging in videoconference.

In the experiment, we consider a 50m * 100m room, in which nodes are distributed according to a uniform random distribution. In the case of Bluetooth, piconets are formed by clustering the nodes close enough to each other. The number of slaves present in each piconet is chosen randomly. Also, some piconets overlap with each other incurring a certain fraction of collisions. The traffic consists of a mix of TCP, Voice and Video. The TCP data connections are always active large file backlogs, with 500-byte packets. The voice connections are modeled according to the Brady model [Brad99]. In particular, the voice connections are "on-off" sources. The on and off times are exponentially distributed, with mean 1 s and 1.35 s respectively. The voice coding rate is 8 kbit/s and the packetisation period is 20 ms, which gives a payload size of 20 bytes. Header compression is assumed for voice packets in Bluetooth and the total packet size is 30 bytes. Voice packets are sent using RTP over UDP. Each experiment lasts 32 seconds of simulation time. In order to probe the sensitivity of performance to population size and to the number of simultaneous connections, we perform different experiments choosing different values of number of nodes and connections. The topology is totally static, which means that nodes are not mobile and piconets are set up at the beginning of the simulation and do not dynamically change. Again, it is important to note that connections are only 1 or 2-hops, as in intra-piconet communication. No inter-piconet communication takes place for now.

The traffic consists of a mix of video, voice and TCP. The video sources are represented by real traces. We use the Star Wars trailer clip encoded using Intel's H.263 compatible codec. The traces have been smoothed using a simple technique, namely a frame as returned by the codec is distributed uniformly in time within the frame interval using a target of 200 byte packets. There is no other smaller time scale transport mechanism and the generated packets are simply sent through the network with UDP. A few seconds from the resulting sources for the codec is shown in Fig 2. A description of the adaptive traffic framework used in the experiments can be found in [Kaz99b].

Figure 5.23 A few seconds from the H263 source trace (sec, bytes)

We investigate adaptive as well as non-adaptive video streaming. The former uses average rates of 48, 64, 80, 128 and 256Kbps for the two codecs, while the non-adaptive cases use the 256Kbps trace. The adaptation mechanism is based on an end-to-end, periodic (1 sec) feedback that contains the number of packets received during the feedback interval. This feedback is used by the server to compute the RTP loss rates. The server then changes its rate using a min/max loss threshold. Below the minimum packet loss rate (5%) the server attempts to additively increase its rate. When the loss rate is above the maximum loss threshold (15%) the server reduces the sending rate, choosing a rate among the 5 available rates that is appropriate to support the reported loss rate. For example, if the current rate is 128Kbps and the loss rate is 50% the sending rate drops to 64Kbps.

The following graphs (Fig 3(a) and (b)) illustrate this behavior in a random 30-node 60-connection experiment. This initial experiment targets at showing the adaptive behavior with the two MAC protocols. The scenario is generated as mentioned above, but now 90% of the voice, video and TCP connections are created at 8.6s and finish at the 16.6s. The goal is to study the adaptive behavior of a video connection that lives throughout the experiment (i.e. from 0s to 32s). As the feedback indicates, the server downgrades the sending rate or attempts a higher rate. We show the loss rates as calculated when a feedback packet is received, the per-packet delays and the server selected average rates for the two cases. First, we note that when the additional connections enter the network (from 8.6s to 16.6s) in WaveLAN, the video connection downgrades to the lower possible transmission rate because the loss feedback goes beyond the threshold. On the other hand, in Bluetooth the loss rates are lower, the transmission rates remain higher and the downgrading is in all cases gradual (one layer at a time). These indicate that the network response is more regular allowing for efficient feedback control with less oscillations. This is true not only for the connection shown but for the other competing adaptive connections as well. It is interesting to note that in the congested network, less packets get lost in the Bluetooth case, but their delay is significantly increased, mainly due to the creation of longer link queues. Since in this experiment we are using a loss feedback this is exactly what the network is expected to do, send as much as possible by trading off bandwidth with increased delay. In WaveLAN the delay experienced by packets is doubled immediately when the new connections enter and does not increase gradually as in Bluetooth. This is because in WaveLAN the connections have downgraded to the lowest rate and the 10Mbps bandwidth does not allow the link queues to fill up. Instead, packets are dropped due to collisions.

3.3.1 Video and TCP

The aggregate throughput is the same for all experiments for WaveLAN. Bluetooth, however, manages to grow the aggregate throughput as the number of nodes increases. The smaller range and formation of additional piconets adds capacity to the network.

Figure 5.24 Bluetooth end to end adaptation

generic difference in the way the two source types, TCP and Video, share the network bandwidth is illustrated in Fig 4. In WaveLAN, individual TCP connections are allowed to grow their window and 'capture' the channel. When this happens, video connections suffer increased loss rates. On the other hand, the presence of polling in Bluetooth allows the video connections to share the channel with the TCP. In fact, with Bluetooth, the video achieves its full rate for different configurations. Several measurements are reported in Figure 5.25 as shown by the caption below each bar (Number of nodes; number of connections; WL vs BT). It can also be noted from Figure 5.25 that the total throughput for WaveLAN is higher than that of Bluetooth fot the 30 nodes, 30 connections case. As the number of connections increases, the total throughput for Bluetooth increase with respect to WaveLAN since larger connections means more collisions in WaveLAN. Also, a larger number of nodes also causes the total throughput for Bluetooth to be higher since it leads to formation of more piconets and hence, addition to system capacity.

Figure 5.25 H.263 Non adaptive video and TCP connections aggregate throughput.

Figure 5.26 Loss Rates for video connections for H.263.

From Figure 5.26, we see that the loss rates for video are much higher in the WaveLAN case where the contention between the connections, especially TCP and video, allows some TCP connections to increase their window and capture the channel, locking out packets from the 256Kbps video connections. On the other hand Bluetooth shows less than 5% packet loss in all cases. Due to less packet retransmissions the Bluetooth case will save a significant amount of power which is particularly important in battery powered devices.

3.3.2 Voice

The significant parameter that needs to be studied for voice is the delay. The complementary cumulative delay distributions for voice in Bluetooth and WaveLAN for 30 nodes and 60 connections with non-adaptive MPEG Video are shown in Figure 5.27 and Figure 5.28.

Figure 5.27 Voice Delay Distribution for WaveLAN

Figure 5.28 Voice Delay Distribution for Bluetooth

Figure 5.29 H.263 aggregate server sent rates

It is seen that the delays suffered are much lower in Bluetooth than WaveLAN. From the complementary cumulative distribution graph, we note that a packet loss ratio of less than 5% can be obtained for a play-out buffer of about 80 ms in the case of Bluetooth, whereas a play-out buffer of more than 350 ms is required to achieve the same effect with WaveLAN. Typically, a delay in excess of 300ms is considered unsuitable for interactive voice communications. To explain the very high WaveLAN delays, recall that the scenario considered here is of a very congested network with large number of connections. In such a network, the uncontrolled access to the channel and the large number of collisions and retransmissions in case of WaveLAN leads to large delays. Bluetooth, on the other hand, has a very controlled access to the channel determined by the polling scheme. This keeps the delays low and well-bounded.

3.3.3 Adaptive Video and TCP

In this section we repeat the experiments of Section --- with adaptive in place of non-adaptive video. The video sources adapt through the use of a periodic end-to-end feedback containing the RTP loss rates, as described above.

Figure 5.30 Loss Rates for Adaptive H.263 Video

First, we show the aggregate source sending rates in Figure 5.30. This quantity is influenced by the loss rates reported to the server and represents the application requested bandwidth. If high loss rates are reported the server drops layers and uses less bandwidth. If loss rates are misreported or not reported then the server continues streaming at the present rate unaware of adverse network conditions. The sum of the feedback packets received in both WaveLAN and Bluetooth configurations was the same (1215). WaveLAN tends to transmit less, especially at high connection density and load. Next we look at the loss rates with adaptive video in Figure 5.31. The controlled, adaptive polling environment of Bluetooth, with less reverse channel problems managed to eliminate the video loss rates almost completely in the adaptive case in our experiments. The highest aggregate loss rate in Bluetooth is 1.32%. In WaveLAN too the loss rates are reduced to half with respect to the non-adaptive case shown in Figure 5.25.

Figure 5.31 H.263 adaptive video and TCP connections aggregate throughput.

Next we examine adaptive video throughput in Fig 9. 1Mbps Bluetooth throughputs are again comparable to the 10Mbps WaveLAN total throughput. As video connections adapt they allow TCP connections to get more bandwidth. In Bluetooth video adaptation reduces loss rate to less than 1% in most cases whereas in WaveLAN adaptive video connections suffer 25% to 30% loss rates.

The total throughput is higher in WaveLAN than in Bluetooth for lower number of nodes. As more piconets are formed, Bluetooth adds bandwidth and surpasses the constant WaveLAN bandwidth, which is independent of the number of nodes.

3.3.4 Conclusion

We have evaluated the efficacy of the Bluetooth technology in supporting ad hoc indoor communications. The simulation results show that Bluetooth performs very well in mixed data and real time traffic scenarios typical of such environments. In particular, it guarantees service quality to multimedia streams while providing fair share of capacity to TCP users. It does not suffer from the TCP capture behavior exhibited by WaveLAN. Though the total system throughput is larger for WaveLAN for small number of nodes, Bluetooth can exceed the WaveLAN throughput when number of nodes becomes large, by using multiple, overlaid piconets. Adaptive video applications fare better with Bluetooth than WaveLAN, in part because the polling schedule of Bluetooth seems to offer a more stable service to adaptive video, precluding oscillations. It is to be noted again that these experiments were performed with the DCF mode of 802.11. In the future, we plan to repeat some of the experiments using the PCF mode.

3.4 Bluetooth Scatternet End-to-End Adaptation

The Bluetooth/Scatternet environment is very different from the conventional 802.11 ad hoc network environment. We explore how a congestion control algorithm, in the context of multimedia adaptation, may perform in this new environment and how equivalent scenarios behave in the Wavelan environment. What are the performance characteristics of adaptive video in Bluetooth scatternets? Is the congestion control indeed easily handled and is it predictable in Bluetooth scatternets as in piconets? How does the asynchronous 802.11/DCF solution compares to the Bluetooth scatternet MAC with regard to effective throughput?

In these experiments we will use the canonical recursive scatternet topology shown in Figure 5.32. Each piconet (e.g. person in the PAN paradigm) has a master and three slaves, as distant as possible. If the slaves are numbered starting at the top and clockwise, then slave 0 may connect to the piconet on top, and slave 1 may connect to the piconet to the right, while slave 2 will not become a gateway. This recursively may result in piconets with up to 5 slaves, 4 of which gateways. In our experiments we give an address (IP or MAC) to each node starting at the piconet to the left and bottom and continuing in rows. The Master is assigned first and then the slaves clockwise starting at the top.

We experiment with up to 16 piconets, in 1 piconet, 2 by 2, 3 by 3 and 4 by 4 configurations. The connections we throw in go vertically and horizontally. In the 3 by 3 case for example we have 6 connections. Our application is adaptive or non-adaptive VBR video based on real traces gathered from the Star Wars IV trailer clip, encoded in Mpeg of 256, 128, 80, 64 and 48 Kbps average target rates. The smoothing process is limited to one frame time and uniformly spreads the frame bytes into apprx. 220 byte payloads into the inter-frame interval. The adaptation is performed through monitoring of loss rates at the client from sequence numbers. The monitoring and the reporting goes on in 1 second intervals and according to RTP. The QoS feedback packet, has to traverse the network back to the server. We use a UDP based protocol so that we can implement delay and timeout algorithms. The timeout we use is employed at the server. When a few QoS packets are lost the server assumes a congested reverse path and will react pessimistically lowering the rates. In our implementation whenever we detect TOTimesThreshold lost QoS packets every TOMaxTicks (5 and 7 respectively). Hysterisis is also implemented, having different low and high loss thresholds (5 and 20%).

Figure 5.32 The canonical recursive Bluetooth scatternet topology

3.4.1 Results

In this section we are presenting the results obtained with the previously described simulation models and scenarios. First for the 16 piconet case and the 8, 4-piconet hop connections we show the server throughput and the goodput (client received throughput) in the case of adaptive and non-adaptive stream and in Bluetooth and 802.11.

In the latter case the 8, 256Kbps average coding rate connections do not even need to adapt as no loss rate occurs (in Wavelan at 11Mbps with the 100m range all nodes are within a single hop). In Bluetooth the connections need to adapt. Two connections are carried on each middle piconet and forwarded to the edge piconets by the gateways in the middle. The gateways can forward at half the rate ideally and looking at the topology we see that piconets in the middle have to forward two streams. As bottleneck links, they define the received throughput to be 25% of the link data rate, with ideal IPS. Two questions remain to be answered in order to understand the rates achieved in the graphs. One, how much is the link data rate? It is well known that when connections are two-way the effective data rate in Bluetooth is 340bytes * 8 / .625msec*5 slots= 870.4Kbps. Two, how close to the ideal IPS case are we? Because of the normality of our topology (same number of slaves per piconets) the max-min optimal IPS we used will almost ideally assigned rendezvous points. Looking at the rendezvous points these are all indeed from 0-4, 24-26, 50-54 and 74-76 which are quite close to the optimal 0,25,50 and 75 for our topology. (If we wished to calculate this we would do the following, looking at the topology and the presence intervals of the gateways: for 4 slots, 1 more gateway will be present than in the optimal case. This is a 4 slots/100 slots that will be divided among 3 gateways instead of 2, so this inefficiency can be roughly calculated at 2.7% lost throughput.) There is also one slave present that wastes two slots per polling round in the bottleneck middle piconets. This at least another 2/(2+40) = 4.8% decrease (assuming always perfect 5 slot packets both ways case). The above means that if our connections were CBR they would ideally each get at most 201Kbps (92.5% of 217Kbps). There are more wasted slots because when gateways are close to switching they cannot fit the packets to the available number of slots. The VBR shape and the packet size of the traffic causes a significant amount of the aforementioned fragmentation. As mentioned, our video smoothing results in many packets around 210 bytes. The 5-slot packets (full utilization 870Kbps case) correspond to 340 bytes payload. In fact our packet size happens to be a ‘bad’ size for the slotted Mac since it needs a 3-slot packet and many packets have a remainder of few bytes. Also in the few first seconds of the 36 of the connections the buffer utilization is not full. These cause the lower throughput, in the orders of 125-175Kbps, seen here.

The important result, however, from these experiments is that the simple end-to-end adaptation worked very efficiently and achieved only slightly lower rates. If we compare the goodput of non-adaptive connections and the server throughput of the adaptive connection we see that they are almost equal! Furthermore, the loss rates are less than 3% in all cases, which means that the adaptation was also timely.

Figure 5.33 Per Connection Server Send Throughput and Goodput for Bluetooth and 802.11, adaptive and non-adaptive for the 4 by 4 piconet case.

Figure 5.34 Loss rates

Figure 5.35 Jitter and delays for a typical connection. Left: from the 3 by 3 piconets case, Right: From the single piconet case.

Let us look closer at a connection in the 3 by 3 non-adaptive case. As we calculated above the forwarding throughput of the gateways is less than the (256Kbps) connection rate and therefore, the link layer buffers get filled and delay is increasing up to the point that the link layers will be full and start dropping packets (we have experimented with large link buffers in order to observe delays). When the same connection does not go through gateways (source and destination are within the same piconet) its rate is acceptable. The slave to slave delays seen range from 1.2 msec to 16 msec or 2 to 26 slots.

Now let us look at the experiments that the video connections adapt. We look at typical connections (remember our topology and traffic is almost symmetrical so all connections are typical) from 2 by2, 3 by 3 and 4 by 4 piconets case. Besides delays now we also look at the reported loss rates, the layer from which each frame is picked and transmitted and the received rate averaged over 1 sec.

The first thing we note is that the delays and the delay jitters are determined by the link layer buffering rather than the superframe size and IPS related parameters. This is because they have similar shape and magnitude in all cases. The second notable thing is the very low frequency oscillatory nature of the delays. Because the connections operate at capacity the receiving rates are not affected by our oscillation in choosing layer. Our simple control, even if it implements hysterisis (different low and high loss thresholds) cannot avoid oscillations. As the loss rate is low a higher rate will be attempted. This fills up the buffers until packets are dropped. Since inter-piconet interference is not significant in our experiments it does not affect loss rates. Packets are dropped from the link layer at the gateways. The application notices the drop and lowers its sending rate. Following that, delays for subsequent packets start falling, as the queues are emptied in the master round robin. This indicates that the closed control system has a strong and immediate impact on the scatternet. By time invariant, we refer to the fact that the rendezvous points do not change (and are not random) in our architecture.

3.5 Ad-hoc Bluetooth and 802.11 Comparison

In this section we are attempting a direct comparison of the 802.11 DCF MAC layer with the Bluetooth MAC layer. This comparison is interesting because these two MACs are intended to work both in infrastructure-less environments and represent two essentially different approaches. The former is a totally asynchronous approach, which uses physical and virtual carrier sense and collision avoidance techniques relying on a common code or frequency hopping pattern physical layer. The latter is a structured synchronous polling MAC layer. It is natural that in face of mobility 802.11/DCF, not requiring on any structure to define its spreading codes or hopping patterns, will have an a-priori advantage over Bluetooth. Since the mechanisms under which Bluetooth is going to support mobility are not yet researched and developed (again, the functions are defined in the standard but their efficient use is a new research problem) we will look at the case without mobility: Given a network topology and adaptive traffic which of the two MAC approaches are more efficient? Since in this section we wish to focus on the MAC layer mechanics directly, we are equalizing physical layer variables. Namely, we lower the 802.11 bit rate to the 1Mbps Bluetooth bit rate, and adjust the radio power to effect a maximum of 10meters range.

In the 11Mbps, 100 meters 802.11/DCF case all nodes (64 in the 16 piconet, 4 by 4 case) connections are within one hop. 8 connections of 256Kbps on average, need 2 Mbps and therefore adaptation is not issue in 802.11 when the exact same scenario is tried. We have studied adaptation in 802.11, end-to-end in [Kaz99b], and using MAC level, AODV propagated bandwidth measurements as application feedback in [Kaz01b]. We have found that even though end-to-end adaptation is mostly effective and lowers loss rates, due to the large end-to-end variance in response it can easily become unstable. End-to-end measurements are also not accurate in presence of hidden terminals. MAC and network feedback based on link by link measurements is therefore suggested there. This is our motivation to compare the two systems on how well they allow a scalable end-to-end transport architecture.

Here, we merely want to look at the 802.11 equivalent of our Bluetooth experiment. In keeping with our target, we only look at traffic, not power consumption issues, which are obviously very different in the two wireless technologies. Due to the higher capacity and the single hop situation, 802.11 does not even have hidden terminal problems in the experiments Bluetooth had multiple hops, gateway fractioned bandwidth and needed adaptation. The delays are also only due to the virtual carrier sense, rarely reaching a back-off situation. Therefore, as the traffic is more, in the higher node/connection number experiments delays are proportionally higher. We can see that all delays are lower than 70 msec, 35 msec, 12 msec and 2 msec in the 64, 32, 16 and 4 node experiments (which actually correspond to the exact Bluetooth topology of 4 by 4, 3 by 3, 2 by 2 and 1 piconet arrangement studied here).

The topology we use in this section is still the same (Figure 5.32), only now the 802.11 case requires as many wireless hops as Bluetooth and has to deal with hidden terminals in its collision avoidance functionality. We use 4 and 9 piconets, which is 16 and 36 nodes respectively. The traffic we use is CBR, instead of VBR so that the results are clearer. We progressively (in different experiments) throw in the network more connections, up to the number of nodes in number. The connections are either adaptive in 7 layers (180, 128, 64, 32, 16, 8 and 3 Kbps) or non-adaptive at 180Kbps. We use connections that follow an even hop distribution; in Bluetooth terms for example in the 9 piconet case, this is one/third within the same piconet, one/third within neighboring piconets and one/third with 2 piconet distance. The same in 802.11 is 2 hops, 4 hops and 6 hop connections.

Figure 5.38 summarizes these experiments by showing averages over all connections in each experiment. Bluetooth is able to deliver the most throughput to its connections in the larger network and larger connection numbers case. 50Kbps are delivered on the worst case average with Bluetooth, while 802.11 is deteriorating its effective throughput as more connections are added. Up to a certain point (14 connections on 16 nodes and 18 connections on 36 nodes) the adaptation works positively in 802.11 as denoted from the decreasing server consumed throughputs. After that adding more connections, creates reverse path problems and increased delays which construed the closed-loop adaptation. In Bluetooth this is significantly less obvious. In both cases, adaptation achieved decreased loss rates but it did not always manage to stabilize the network into low loss rate operation. Note however that the loss rates reported in the figure are averaged over all connections and for the lifetime of the connections (which is 30 secs) and therefore a large portion of them are due to the high loss rates before adaptation could converge to acceptable rates. This initialization time is quite large, as denoted also by the delay graphs. The average delays where high in both MAC cases but much higher and increasing more rapidly in the 802.11 case. This is due to the nature of delay in the asynchronous MAC versus the nature of delay in polling MAC. In the former the more packets thrown in the network, the higher the virtual carrier sense and back-off timers for each MAC re-transmission. In the latter, due to the non-exhaustive polling and the periodic time division multiplexing in gateways the delay maintains a value defined by the link buffers and the superframe. Therefore, adaptation decreased the high delays in the 802.11 case.

Figure 5.36 Adaptive connections in the (top) 4 by 4, (middle) 3 by 3 and (bottom) 2 by 2case. Left column shows (top) reported RTP loss rates (middle) layer from which frame is picked for transmission (bottom) received rate averaged over 1 second. Right column shows jitter and delay

Figure 5.37 The corresponding 802.11/DCF exact topology cases (top, left) 64 node, eq. to 4 by 4 (top,right) 36 node, eq. to 3 by 3 piconets (bottom, left) 16 node, eq. to 2 by 2 and (bottom, right) 4 nodes, eq. to 1 piconet Bluetooth case.

Figure 5.38 Direct Comparison of Bluetooth and 802.11 MAC using an end-to-end adaptation mechanism. (top) Average Recv throughput, Sent Throughput (middle) Loss rates, Delay Jitter (bottom) Delay, Packets Delivered

Comparing to an 802.11, 11Mbps ad-hoc scenario, we noted the big difference in capacity of the two technologies. When only piconets are used, say as Internet connection clients, the network capacity is comparable to the 11Mbps 802.11 case. This is due to the ad-hoc frequency hopping pattern exchange that effectively adds capacity (at Bluetooth data rate increments) as more piconets are introduced. However, in the scatternet case this is not true, because the capacity of the network is now defined by the fraction of the Bluetooth data rate at the gateway nodes. In general, from our experiments it is shown that end-to-end adaptation works very well in scatternets. Generalizing, this indicates that congestion and admission control, measurements and delays may be efficiently handled in the controlled Bluetooth medium. We took our comparison further and looked at how the 802.11/DCF (CSMA/CA with virtual carrier sense, ACK and RTS/CTS) asynchronous MAC compares with the structured poll controlled Bluetooth MAC in terms of effective throughput. We showed that the Bluetooth MAC is offers significantly more effective throughput as well as uses the throughput much more efficiently through a simple adaptation procedure than the asynchronous MAC. Therefore, the trade-off between the two types of MAC should be determined by how efficiently a structured poll controlled MAC can actually maintain and support its structure through mobility without significant overhead. In conclusion, the Bluetooth MAC offers a higher ratio of effective to link throughput while adaptive transports and applications may monitor better and perform their function better providing for low loss rates and efficient QoS adaptation.

3.5.1 Conclusions

We have experimented with issues of congestion control in Bluetooth scatternets. We have used a scatternet architecture based on rendezvous points, and advanced inter-piconet scheduling algorithms to build a state-of-the art Bluetooth ad-hoc network, and used realistic models to experiment with it. We looked at the congestion control as performed by an end-to-end RTP based adaptive video application. Bottleneck points are necessarily created in the scatternet where gateway nodes divide their time to different piconets. They operate at a fraction of the link data rate that determines the effective allowable rate of end-to-end connections. Buffers at these points should be higher as packets are gathered there and waiting for gateway switches. In fact, end-to-end delays are mostly defined by these link queuing delays. We discussed the issues that limit the end-to-end goodputs in such an environment and studied the dynamic adaptation. As was known to happen in piconets, in scatternets too, when using static window and static rendezvous points the control medium lends itself to a very eficient and timely adaptation. The adaptive connections managed to achieve the goodputs that non-adaptive connections achieved but at less than 3% loss rates.

4 Available Bandwidth

4.1 Packet Pair Background

In this section, we qualitatively analyze the source of errors of an end-to-end packet pair based measurement. Our sampling method of available bandwidth is based on packet pairs and trains. The available bandwidth measurement method we introduce suffers similar errors when applied end-to-end. We assume that the reader is already familiar with the basic packet pair analysis (see [Lai99]). In our analysis we focus separately on a path’s three distinct sub-paths: 1) the path consisting of links from the source up to the bottleneck link, 2) the bottleneck link, 3) the path from after the bottleneck link to the destination. When we refer to a link we refer to the outgoing queue and the actual link. Let us assume:

- path of N-1 links, with the bottleneck at link k

- Pb_i, the potential bandwidth exiting link i, determined by the pair dispersion after link i ; Pb₀ is then the pair potential bandwidth at the sender, defined as the payload bytes over the initial time separation.

- B_i, the actual bandwidth of link I

- Pb₀ > B_k

Figure 6.1 Possible events in links before the bottleneck link.

Figure 6.2 Possible events at the bottleneck link

Figure 6.3 Events occurring after bottleneck link

Figure 6.4 ab-probe model

Figure 6.1 shows the different events that may occur at any link before the bottleneck link (more than one event can occur over a single link). This part of the path is interesting because it ultimately defines the potential bandwidth (Pb_k) with which the pair will enter the bottleneck link k. In order to make sure that we can measure the bottleneck link, Pb_k must be greater than the bottleneck link bandwidth. This is ultimately determined by a permutation of possible events a, b and c as depicted in . After each link the pair may decrease its potential bandwidth when B_i<Pb_i-1 (event a). The same can happen when cross traffic arrives in the time between the pair arrivals (event b). (Note that the probability of (b) occurring is inversely proportional to Pb_i-1 for a given network load.) When enough cross traffic is queued already when the packet pair arrives then we may get useful time compression. It is called time compression because the time separation of the packet pair is potentially compressed (it becomes the payload of one packet over the link bandwidth). We call it useful because it increases the pair’s potential bandwidth allowing correct measurement of larger bottleneck link bandwidths.

Figure 6.2 shows the possible events for a packet pair entering the bottleneck link. The packet pair arrives with a potential bandwidth of Pb_k-1 and exits with Pb_k, hopefully equal to B. If Pb_k-1 is less than B then we already have an underestimate of B, which is either sustained (a), or further underestimated from intercepting cross traffic (b). If Pb_k-1 is more than B then the exiting potential bandwidth is either B or some underestimation due to intercepting cross traffic (d).

Figure 6.3 shows events occurring after the bottleneck link. As the pair is traveling after the bottleneck link it should sustain its time separation. However, enough intercepting cross traffic (i.e. traffic queued in the middle of the packet pair) may cause further underestimation over a link (case (b)). Traffic queued in front of it may cause time compression (if service/transmission time for the packets queued in front is more than the packet pair time separation). This is an adverse time compression that causes over-estimation of the bottleneck link bandwidth.

4.2 Extension of Packet Pair/Train For Available Bandwidth Sampling

4.2.1 Why do we need a new method to calculate the samples?

As mentioned previously, the packet pair “bytes over time” sample calculation is as correct as the network follows the weighted fair queuing model. In practice, as we show below, this method results in significant errors in the sampling especially when the network is operating at higher than low loads. Furthermore, the widespread cprobe package that samples available bandwidth using the “bytes over time” method has exhibited unexplained behavior according to its inventor’s experiments [Car97]. The above reasons motivate us to further study the issue of sampling available bandwidth from packet pairs and trains.

Let us attempt to calculate what the available bandwidth over a single link should be and how this is different than the “bytes over time” calculation. Consider two packets, Packet 1 and Packet 2, each of size S, entering the bottleneck link with a potential bandwidth of Pb_k-1 which is close to B, the bottleneck link bandwidth. Packet 1 is used to mark the beginning of time in the measurement as the time it exits from service. The time separation d is defined as the difference between that time and Packet 2 time exiting service. The “bytes over time” method will then calculate the available bandwidth A at S/d.

Now assume that A is indeed the available bandwidth seen by those packets. Then, the consumed bandwidth is obviously B-A. Bandwidth is a rate and therefore we need to determine what is the interval over which the available bandwidth is measured. Let us call this the sampled interval. This interval is defined by the separation of the packets, from the time the Packet 1 enters the queue until Packet 2 enters the queue, which is Pb_k-1/S. Since during that time the consumed bandwidth is assumed to be B-A then the number of bits that enter the link’s queue during the sampled interval is (B-A)* Pb_k-1/ S. The observed separation at the link exit, as defined in the previous paragraph, will be the transmission/service time of the other bits that have entered the queue and the transmission/service of Packet 2 of the packet pair, therefore d=(B-A)* Pb_k-1/ (S*B) + S/B. It is obvious then, by solving for the available bandwidth A, that this is not generally S/d as the “bytes over time” calculation dictates.

Furthermore, following the above logic (we show in more detail later), the “bytes over time” calculation gives a different available bandwidth result for different size trains. This effect has been in fact noticed in real testbed experiments in [Car97] by cprobe researchers. For the reasons above, we conclude that a new model for calculating available bandwidth samples from packet trains is needed.

4.2.2 The ab-probe model

In this paragraph we use a similar to paragraph A. logic to develop a model to calculate the available bandwidth samples from a packet train of N equally sized and spaced in time packets. We later generalize the model, making it more suitable for passive measurement, where the packets used for the measurement are not generated but are part of existing flows.

Assume N (N >= 2) equally spaced packets each of size S bits. Assume the packets reach the bottleneck link with a potential bandwidth of Pb, that the bottleneck link bandwidth is B and the actual available bandwidth (during the train) at the bottleneck link is A, as before. The sampled interval (see par. A.) is

Equation 1

If the available bandwidth during this interval is A then, using Equation 1, the traffic (in bits) the queue should be accepting from other flows during this interval is

Equation 2

Then the observed time dispersion (separation) between Packet 1 exiting service and Packet N exiting this link’s service is the transmission time of the packet train bits and the cross traffic bits through the link.

Equation 3

If this dispersion is maintained (see section B) then this is the T sampled at the observer side. This is the same assumption under which packet pair works. Solving for the available bandwidth A we get:

Equation 4

Last, let us show a generalized form of Equation 4 that is appropriate for passive measurements (when packets in the packet train have different Pb_i s and sizes S_i.)

Equation 5

Note that an available bandwidth sample, averaged over the interval sampled by the packet train, may be negative. This is intuitive and in accordance to available bandwidth definition. It may be negative when, during a small time, more traffic enters the link than its bandwidth. Averaged over longer period of times the available bandwidth with samples given by these equations will be from 0 to B, the bottleneck link bandwidth.

The above model can be used (i) to help understand the error when sampling available bandwidth using the “bytes over time” model in non-WFQ networks. (ii) to develop a network independent measurement. We study these issues further.

4.2.3 The “bytes over time” model

In this paragraph we look at the model used so far in measuring available bandwidth that uses the “bytes over time” sample calculation. Equation 4 gives the correct available bandwidth averaged over a period of time (defined by a train of packets). This equation is to be compared with the “bytes over time” calculation (N-1)*S/T.

The first thing we deal with, is what cprobe researchers noticed in their real testbed experiments when using the “bytes over time” model: that larger size trains would give persistently lower available bandwidth measurement when the network conditions where constant. In Figure 6.5 the available bandwidth that the “bytes over time” method is calculating by the observed separation in time from Equation 3 is used. It shows the exact behavior that real experiment graphs in [Car97] show! Consequently, this is an effect that comes from the “bytes over time” error in non-WFQ networks and can be directly seen using our model, and factored out in a measurement that is based on our model. These cprobe experiments provide then an additional validation that our model is correct.

Figure 6.5 “bytes over time” available bandwidth calculation varies with train size while network conditions remain constant.

Notice that in Figure 6.5 the available bandwidth should be 2Mbps and, even as it varies with train size, it never reaches the correct value. Figure 6.6 and Figure 6.7similarly show the relative error for the “bytes over time” calculation used over a 4Mbps link and trains of 10 packets and 160 bits sent at 4.1Mbps (just above the bottleneck link bandwidth as cprobe indicates). Note that the “bytes over time” error becomes exponentially larger as the network gets congested. Specifically it reaches a 100% relative error when the 4Mbps link has 1.28Mbps available. When the network is in a very congested state, the link having 100Kbps available out of its 4Mbps, it reaches a 2000% relative error.

Figure 6.6 The “bytes over time” relative error for a 4Mbps link with 4Mbps up to 1.280Mpbs available

Figure 6.7 The “bytes over time” relative error for a 4Mbps link with 1.230Mbps to 100Kbps available

4.2.4 Observability and robustness of ab-probe

The ab-probe related model may become a measurement method, if B and Pb can be estimated accurately enough. The measurement is interesting both as an end-to-end measurement and as a network layer measurement in network feedback architectures, in both active and passive forms. In the network layer case, where the measurement takes place link by link, Pb is known and B can be very easily estimated over one link. Next we deal with developing an end-to-end measurement. In an end-to-end measurement, the bottleneck bandwidth can be estimated accurately using the nettimer paradigm [Lai01]. The potential bandwidth entering the bottleneck link (Pb) however is only known to be between the sender bandwidth and the bottleneck link bandwidth. Let us see then how robust to errors in Pb estimation an end-to-end measurement is. Figure 6.8 shows a large range of relative errors in Pb and the corresponding relative error in the available bandwidth measurement based on ab-probe. It shows that an end-to-end measurement that sends the packet trains at near bottleneck link rates is very robust. This is because the Pb is normally limited between the sender bandwidth, which we may have be a little more than the bottleneck bandwidth, and the bottleneck link bandwidth itself by definition. Note that any current measurement requires a sender potential bandwidth higher than the bottleneck link. Note also that the available bandwidth relative error due to Pb relative error does not depend on the available bandwidth magnitude. Also, techniques like packet tailgating can be used to estimate Pb. In our active measurement experiments we use the bottleneck link bandwidth B to estimate Pb and a sending rate of B. This is a good estimation because Pb, the potential bandwidth of a pair entering the bottleneck link, will be by definition between the sending bandwidth ( which we have equal to B) and B. In our passive measurement experiments, we use B or the given sender bandwidth whichever is smaller. If the sender potential bandwidth is below B then we estimate Pb as the sender potential bandwidth (since it is less than B, the pair will ideally reach the bottleneck point with the same separation as it was send). If the sender potential bandwidth is more than B then we estimate Pb as B. Obviously this estimation may have a large error in the cases when both (i) the sender potential bandwidth is much higher than the bottleneck link and (ii) the network topology consists of very different and abruptly changing link bandwidths (see also section ][). The latter may happen for example when a 10Mbps link is followed by a 27.8Kbps link at the end host. However both (i) and (ii) will not happen at all in an adaptive protocol since sending at rates much higher than the bottleneck link bandwidth, when one should be sending at an average less than the real available bandwidth (which is even less than the bottleneck link bandwidth), indicates an unstable state for an adaptive protocol. This fact combined with the measurements resilience in Pb errors shown in Figure 6.8 indicates that the Pb estimation is effective for our passive measurements as well (see passive experiments below).

As the packet train travels towards its destination it undergoes the distortions we described in the packet pair section and appropriate filtering of the resulting samples should be used.

Figure 6.8 How an error in Pb estimation affects the available bandwidth sampled with ab-probe

Figure 6.9 Difference between “bytes over time” and ab-probe measurement.

As mentioned previously a filter for available bandwidth samples has an averaging effect rather than discovering modes and density points. We use the Tustin low pass digital filter, which the filter used in TCPW as well.

δ_κ is the inter-sample time and τ the filter cut-off frequency. In order to focus on the sample calculation we use a very low cut-off frequency for the averaging filter and show and deal with converged values (τ is 10 sec for the presented graphs).

We are now ready to implement our measurement in active and passive forms. As a conclusion to this paragraph we present a graph that shows the diversion between the two models using the equations presented here.

4.3 Experiments

We have run real experiments using long range Internet connections, short range campus Internet connections, and Wavelan wireless links. We also run simulation experiments using NS that validate our theory but here we focus on presenting the real experiments, which provide a hard test and a more reliable setting for validation. The experiments are performed using the active and passive measurement and testing both options of packet pairs or trains. In all cases we have found that practice agrees with theory.

The short range Internet topology is shown in Figure 6.10. The two LANs are connected by a Cisco 2600 router in two 10 Mbps interfaces. On each LAN one host is performing the ‘probing’ and another is used for injecting extra cross traffic. Normal traffic exists in the network as usual, but we inject the extra traffic so that we may observe how the measurements react to it. The long range Internet topology (Figure 6.11) has the two source and destination LANs more than 20 hops apart. The sending hosts (probing and cross traffic) are switched through a Cisco Catalyst switch series 6509, located in California. The two receiving hosts are connected to a Cabletron SmartSwitch 2100, located in Italy. For the wireless experiment we used a single Wavelan ad-hoc IP link.

Figure 6.10 The short range campus Internet experiment topology

Figure 6.11 The long range campus Internet experiment topology

The probing connection is either active or passive and uses the appropriate timestamping in the packet headers as required for the measurement. We use application level timestamps as a measurement application or an adaptive multimedia streaming application would have to. The active measurement uses 128 byte packets changing the inter-packet delay to achieve the required sender bandwidth. In the case of packet pairs, 40 packet pairs (80 packets) are sent at the required bandwidth following a one second idle interval. When packet trains are tested we sent 10 trains of 8 packets following a one second idle interval. We manage to experiment with high potential bandwidth packet pairs by performing busy wait when necessary, dealing effectively with the non-real time Linux (Redhat 6.2) kernel 10msec timer granularity problem. As discussed in the Wavelan case we have used kernel level timestamps (see section V par. C). The passive measurement uses a real source of MPEG video. Traces are pre-captured from a Star Wars trailer clip [13]. It is simply smoothed to 200 byte packets sent uniformly over the frame time. The shape used for a 4Mbps average rate is shown in Figure 6.12 . The cross traffic is simply 512-payload CBR of the reported rate in all cases. As mentioned the bottleneck link bandwidth is determined using the nettimer tool.

Figure 6.12 The sender bandwidth of the MPEG-4 video.

4.3.1 Active measurement

In this paragraph we present the active measurements experiments. We perform one experiment per measurement method using the same probing traffic. All experiments are performed one after the other. These experiments prove that our model is correct in practice and a major improvement from the ”bytes over time” (b.o.t.) case.

The active measurements for the short range experiment are summarized in Figure 6.13. It depicts a graph similar to Figure 6.9 but using the real experiment results instead of drawing the model equations. In this one graph we summarize results for all relevant active measurement experiments, i.e. using pairs and trains and with the “bytes over time” and the ab-probe model sample calculation. The x axis is the rate of the injected cross traffic. Initially we do not inject any cross traffic, measuring the available bandwidth as left from other Internet connections, which we do not control and may share network resources with. In this case, using the b.o.t. sample calculation gives 6.1Mbps available over a 9Mbps link, while the ab-probe calculation gives approximately 4.5Mbps. After getting the initial point for the two methods we draw a lighter ‘expected available bandwidth’ line. It represents the available bandwidth we expect to measure after purposively injecting another 1Mbps to 3Mbps. For example, we would expect to measure 5.1Mbps available bandwidth after injecting a 1Mbps rate cross traffic (6.1Mbps minus 1Mbps). We clearly see however that the b.o.t. sample calculation does not react at all to the fact that another 1Mbps is being consumed in the path. Even 2 or 3 Mbps cross traffic does not have an impact on the b.o.t. measurement. On the other hand, ab-probe measures correctly, following the expected available bandwidth line. Comparing this graph with our model graph at Figure 6.9 we see that the above observations are exactly seen in our model equations as well. Our theoretical model is very closely observed in practice, and that the ab-probe available bandwidth sample calculation significant improvement is visible in practice as well.

In Figure 6.14 we see the same graph regarding the active measurement in the long range Internet scenario. In this case the bottleneck link is measured at 4Mbps. The available bandwidth using the b.o.t. samples in our averaging filter is measured at approximately 2.5Mbps when no other cross-traffic is injected intentionally. In this case too, when we inject cross traffic of 1Mbps and 2Mbps the b.o.t. based measurement fails to drop appropriately, and seems slightly affected by the extra traffic. The ab-probe method initially measures a 1.5Mbps available (when no cross traffic is injected intentionally). When 1Mbps is injected the ab-probe reacts to the available bandwidth drop almost ideally measuring around 200Kbps available. When 2Mbps is injected ab-probe measures –1Mbps. Note that as we mentioned before the available bandwidth may be negative for some time. Looking at the connection loss rates for this case, we see that the probing traffic lost 80% of its packets and the cross traffic lost 33% of its packets. Even in this high congestion unstable condition the ab-probe manages to follow the available bandwidth changes while the b.o.t. is unable to measure or even sense any congestion.

Figure 6.15 and Figure 6.16 show a few more detailed graphs of the experiments performed, the same ones summarized and discussed above. The samples are scattered over a large range of values but we limit the graph to where most points are for legibility. In the short distance experiment (Figure 6.15) we note one point of concentration for the ab-samples at 5Mbps. When 3Mbps of extra cross traffic is injected the concentration point is still there but the density is much lighter. Many packet pair samples have scattered through the lowered bandwidth region of the graph. There are samples with highly negative available bandwidth values (in the figure we show up to –30Mbps but there are some samples that are much lower than that too). As we mentioned before our available bandwidth may be a naturally negative value, as it is sampled over a short interval and it depends on the offered traffic over that interval. During a short interval the offered traffic may be much larger than the link bandwidth and this causes the negative samples. The averaging finally results to an exact measurement as shown before. The ”bytes over time” model (samples shown next to ab-probe samples in the figure) fails to capture this effect. The long distance experiment shows similar behavior. We note the many different distinct concentration points (modes) of the samples in both sample calculation methods. In ab-probe these points may correspond to a negative available bandwidth.

In all, the ab-probe active measurement managed to successfully measure the available bandwidth in all cases, even from long distance (more than 20 hops) end-to-end observation, while the b.o.t. samples have failed.

Figure 6.13 Active Measurement using BOT and AB-probe in the short range Internet topology

Figure 6.14 Active Measurement using BOT and AB-probe in the long range Internet topology

Figure 6.15 . Graphs from the short distance active measurement experiment a. Ab-probe short distance packet pair samples when injected traffic is 0Mbps b. BOT packet pair samples when injected cross traffic is 0Mbps c. Ab-probe short distance packet pair samples w when injected cross traffic is 3Mbps e. Ab-probe packet pair samples filtered f. BOT packet pair samples filtered. The measurement is not affected by the injected traffic in all cases.

Figure 6.16 Graphs from the long distance active measurement experiment. a. Ab-probe short distance packet pair samples when injected traffic is 0Mbps b. BOT packet pair samples when injected cross traffic is 0Mbps c. Ab-probe short distance packet pair samples when injected cross traffic is 3Mbps e. Ab-probe packet pair samples filtered f. BOT packet pair samples filtered. The measurement is not affected by the injected traffic in all cases.

4.3.2 Passive Measurement

In the passive measurement, as mentioned before, we are using a VBR real video source as the probing traffic. This case is especially interesting because it shows that our measurement can be successful even in a rate-based transport. We take advantage of the VBR nature of the video source in order to probe higher than the stream average bandwidth. One advantage is that we may measure possible excess available bandwidth where a ‘trial-and-error’ adaptive protocol would have to possible suffer additional loss to probe the higher bandwidth. At the same ideas similar to potential filtering may be employed to give more significance to high potential bandwidth samples and possibly result in a more accurate measurement. We do not employ this idea here.

The experiments show that ab-probe is less successful than in the active case, as expected due to the lower and uncontrollable potential bandwidth of the probing pairs. Figure 6.17 shows our usual model diversion graph for the short range Internet case where a 4Mbps average video trace is used over a 9Mbps reported bottleneck link bandwidth. Ab-probe is somewhat successful in reporting the extra cross traffic injected in the network whereas the ”bytes over time” remains almost constant in face of the additional traffic. Figure 6.19 shows the samples collected for some cases during the short distance experiment.

We used the 4Mbps average video for the long range case (which reports a 4Mbps bottleneck link, Figure 6.18). This is a very difficult case since our VBR connection will claim the whole capacity of the bottleneck link. We therefore report that the loss rates of the video traffic were 1%, 1.5% and 50% in the case of no additional cross traffic, 1Mbps and 2Mbps cross traffic respectively. In this special case we note: (i) Our probing (video) traffic consumes the bottleneck link and therefore the additional traffic to be measured may be less than reported if it is mostly dropped in the bottleneck link. For example in all cases of 1Mbps cross traffic the cross traffic connection suffered over 40% loss therefore leaving the measurements almost unaffected. (ii) There is significant queuing of other uncontrolled traffic after of the bottleneck link as denoted by the fact that the “bytes over time” pairs average at higher than bottleneck point rates (many time compressed samples). Ab-probe is more effective in dealing with compressed packets since it can average them out with the negative samples. (iii) Both “bytes over time” and ab-probe were able to sense the 2Mbps cross traffic. Note that this is the only satisfactory result regarding the “bytes over time” model and is reported when there is significant loss in probing and competing traffic. The “bytes over time” is able to capture this effect due to the lost packets (the size of the packets is the nominator). (iv) The trains fail to ‘sense’ the 2Mbps injected cross traffic because the lost packets cannot be accounted for correctly in our implementation. Specifically, trains may become smaller when first and last packets are lost leaving large intervals without having samples.

In all, a simple averaging filtering was being used, ab-probe showed measurements close to the expected at all times while the b.o.t. was worse, exhibiting similar deficiencies as in the active measurement case.

Figure 6.17 Model diversion graph for pairs and trains for the short distance case

Figure 6.18 Model diversion graph for pairs and trains for the long distance case

Figure 6.19 Graphs from the short distance active measurement experiment. a. Ab-probe short distance packet pair samples when injected traffic is 0Mbps b. BOT packet pair samples when injected cross traffic is 0Mbps c. Ab-probe short distance packet pair samples when injected cross traffic is 3Mbps d. BOT packet pair samples when injected cross traffic is 3Mbps (note that the samples may look more than the ones in b. but in reality they are just more scattered)

Figure 6.20 Graphs from the wireless measurement experiment. a. Sender timestamps of FTP traffic when injected traffic is 0Mbps b. Sender timestamps of FTP traffic when injected traffic is 3Mbps c. Ab-probe packet pair samples when injected cross traffic is 0Mbps d. BOT packet pair samples when injected cross traffic is 0Mbps e. Ab-probe packet pair samples when injected cross traffic is 3Mbps f. BOT packet pair samples when injected cross traffic is 3Mbps

4.3.3 Wireless Link Measurement

In this case we are experimenting with a wireless link. We setup an ad-hoc IP over Wavelan connection between two hosts and measure an FTP connection. While this is a passive measurement it is a different than the above rate-based experiment. Window based protocols can reach very high potential bandwidths and therefore passive measurements can be quite accurate, as much as active measurements at times. One reason that drove us to look at FTP traffic in this case in particular is that the experiments with the rate-based traffic and application level timestamps could not be performed. We believe this is due to kernel buffering that is necessary at the Wavelan driver. Due to this buffering the timestamps at the application layer are not reliable. By using FTP traffic (TCP) we were able to read kernel timestamps through the nettimer/tcpdump provided packet capture libraries. The bottleneck link reported is 6.85Mbps and the card used is the Avaya Gold World Wi-Fi Card at 11Mbps.

Figure 6.21 Wireless link measurement

In this case too the model diversion graph shows the accuracy with which ab-probe can follow the injected traffic and the failure of the b.o.t. case (Figure 6.21). Some samples from this experiment are shown in Figure 6.20. In part (a) and (b) of the figure we show the potential bandwidths that the TCP connection accomplished during 2 seconds of the connection (the connection was actually run for 6 min). The window and slow start phases can be distinguished there. Note that the “bytes over time” model calculation follows closely the probing traffic pattern, something that we noted in previous passive experiments. This is undesirable effect because it implies being affected by it, while the available bandwidth measurement should be affected mostly by the cross traffic. In the case of 3Mbps cross traffic b.o.t. managed to measure a drop of 1Mbps only (mostly related to the TCP lower potential bandwidths) whereas ab-probe measured a 2Mbps drop. In this wireless link case too the ab-probe followed much better the expected line.

4.4 MMTP

In this paragraph we are developing a Measurement Based Transport protocol, suitable for real-time adaptive multimedia. We have already defined the requirements in 2.3. In summary, we need a transport that works well over these heterogeneous links, converges fast to its stability point, performs smoother, perception-aware flow control and is tcp friendly. Instead of traditional “trial-and-error” techniques it would be desirable to use path available bandwidth measurements. Unfortunately the latter is considered especially difficult for a network that does not well approximate weighted fair queuing models. The fundamental question is then, whether the above requirements can be met while relying in direct measurements. Transports have frequently resorted to packet dispersion techniques, such as packet pairs and trains. However, they have been shown to exhibit errors that cannot generally be overcome with statistical techniques. The first contribution of this paper is to propose a calculation of the available bandwidth samples that is based in packet dispersion but is different than the classic “bytes divided by dispersion” model. We believe our model captures and deducts a portion of the sample calculation error from currently used techniques. We name it path available bandwidth probe or pab, and focus on issues of its deployment in adaptive multimedia. First, we take advantage of the typically bursty video traffic and implement a filtering mechanism suitable for passive measurements. Then, based theoretical results, we develop a fair, stable, and TCP-friendly add/drop mechanism that is based on the available bandwidth, rather than observed loss. Some simulation experiments further support our study. The design of the protocol is motivated by the fundamental question of whether end-to-end stability, fairness, and TCP friendliness can be supported with a pure passive measurement technique, rather than observed loss. We say pure passive as opposed to semi-passive as for example used in [Leg00]. A transport protocol in general may not use active measurements due to the overhead imposed on the network, but it may perform what we call semi-passive measurements. It does so, by using existing packets from the application but shaping them appropriately. Unfortunately, a real time transport has limited shaping flexibility due to delay constraints. PLM ([Leg00]), for example, pairs up all application data to packet pairs in order to increase their potential bandwidth ([Lai99]). Our requirement to handle bursty VBR, delay and delay-jitter sensitive traffic does not allow us to do this. Consider also that bursts usually represent significant frames in video, and pairing increases loss probability as well.

4.4.1 Filtering

Let us now describe the low pass digital filter. We start with the simple low pass filter with cut-off frequency 1/τ. In order to digitize it we use the Tustin approximation. As we mentioned we are using an algorithmic filter before that, to provide the samples and their weights to this filter. In order to accommodate this we use two cut-off frequencies, 1/τ1 and 1/τ2. The weight from the algorithmic filter defines the actual cutoff frequency, which is in the range between the two values. Because we are dealing with passive measurements and inter-sample time may vary, we use the frequency transformation proposed in [Pou00] that minimizes the distortion due to inter-sample varying times. According to all of the above the final low pass filter form used is:

where w is the weight input from previous stage, δκ is the inter-sample time and the inverse of and the cutoff frequencies as provided for the anti-distortion transform .

4.4.2 Confidence based Weighting Algorithm

In our implementation each packet contains a sequence number k and its initial separation time (or application inter-arrival time) named iat. These are marked by the sender (only the latter is considered overhead). The receiver keeps a small window (of 8 packets) to handle re-ordered packets. Each packet is appropriately placed in the window and its sender bandwidth is calculated. An increment number is given to the packet (in) to denote whether the packet has been received in order or not (for the BLB calculation we use only the in-order received packets). We compute all maximums and averages required over a sliding window of size 32.

Next we describe how we use the samples for the available bandwidth digital filter. The packet pair samples that have potential bandwidths more than BLB provide one correct available bandwidth sub-sample. The packet train as it travels towards the receivers suffers similar distortions to the described in 3.1. We calculate the available bandwidth by observing T, the inter-arrival time between reception of Packet 1 and the last packet of the train, and then use

Equation 4

in window of mwnd packets (in our experiments, 8). In reality, we examine the window right before a packet belonging to a new packet train is received and use only the pairs that satisfy Sb>BLB. In order to calculate the available bandwidth we then compute only the ‘good pairs’ contribution to T. We also deal with lost packets in a way that gives us as many possible ‘good’ pairs as possible. Say, for example that Packet 2 and Packet 4 have arrived but 3 is lost. We will then use half the inter-arrival time between 4 and 2 to estimate Packet 4’s potential bandwidth, and count a contribution of zero bytes for packet 3. Note that in order to calculate RTP suggested loss rates, one would use a similar window. If AB_sample is our sample and AB_w our weight we use:

4.4.3 Stability and TCP Friendliness

In this paragraph we study the use of the measurement in a multimedia rate-based transport protocol, and deal with how to be TCP-friendly not based on a flow observed loss, as current models describe, but based on a reliable available bandwidth measurement.

We use our measurement in its passive form with the offered traffic being a layered video stream. The video packets are appropriately timestamped at the server, and acknowledged by the receiver, so that a sender based packet train can be used for calculation of the available bandwidth at the sender. The sender then may choose to send more or less layers depending on the measurement. The RTT updates and the minimum available bandwidth reporting on the round-trip path allow the protocol to be TCP-friendly. We show how a multimedia transport based on the measurement to perform flow control can be TCP friendly, and that such a transport is stable.

As per [Flo99], a flow is TCP friendly if its throughput is conformant with a TCP connection throughput in similar circumstances. This means that its throughput should be less than where B is the segment size, R the roundtrip time and p the loss rate. An equivalent expression is that the throughput of a steady state TCP connection should be .75 * W * B/ R. This is actually obtained before replacing the upper bound of the window as a function of p.

Consider the class of window-based binomial algorithms as those with increase/decrease functions such as

In fact TCP’s AIMD falls into this category when k=0 and l=1. All algorithms with k+l=1 converge to fair allocation and are known to be TCP-friendly [Ban01]. Using the above result we can choose a k and l that results in less abrupt flow control and is therefore suitable for multimedia. F example, the SQRT algorithm that has k=0.5 and l=0.5.

Our problem then is twofold. First, we need to have a rate-based equivalent of the above equation and second, use a TCP friendly adaptation mechanism that does not rely on loss rate, but on our available bandwidth measurement. We develop the following algorithm based on the notion that in TCP steady state the window is the delay-bandwidth product. Our target TCP-friendly throughput then should be where AB is the available bandwidth measured. If our current average sending rate is L then we may use the following equation on each RTT

Now assume our stream is encoded into N layers of Li average throughput each. Then those define N points, used as ‘watermarks’ for the value of w

The above ‘watermark’ points allow us to choose a layer by keeping a ‘fluid’ value of w. A layer is sent if w is above the layer’s corresponding watermark.

5 Comparisons

In this section we are developing a uniform environment under which we can test by simulation all the different options studied in the thesis. We use the exact same network and the exact connections and mobility under different networks and all adaptation strategies. For each environment we report loss rates. As mentioned before loss rates are important because they can indicate whether and how much QoS may be improved. We then show QoS using our evaluation model introduced in Chapter 2. As mentioned, the numbers we use for our model correspond to a high motion, high encoding complexity video stream.

Namely the curves shown in the graphs of this section correspond to:

- Non adaptive high layer transmission

- Non adaptive low layer transmission

- RTP ‘Trial n Error’ Adaptation

- Packet Pair Available Bandwidth based Adaptation

- AB-probe sampling Available Bandwidth based Adaptation

- Network Feedback Adaptation

The topology is setup and described according to the Bluetooth terminology (since we use the same topology for both 802.11 and Bluetooth). It is a normally structured topology that can be recursively defined. In the 802.11 single hop and the Bluetooth experiments the distance between a Master and a Slave is 7 meters when a straight line is used. The same goes for the 802.11 single hop case. In the 802.11 multi-hop case we have scaled the topology to get the multihop effect by having a distance unit that is multiplied by 10.

Figure 7.1 Topology used in the comparison experiments. M denotes a node that would be a Master in the Bluetooth scenario, S a slave. A direct line between a Slave and a Master indicates a Slave’s Home Piconet, a dotted one that the slave acts as a gateway to another piconet

We run a large number of experiments with an increasingly larger number of connections. The connection source and destination points are decided at random but adhering to a hop count distribution. Let us use the Bluetooth terminology to refer to hop and mean a Piconet hop, i.e. any two nodes in adjacent piconets have a hop distance of one. We use connections of 1 to N Piconet hop distance in a network of N by N Piconets (4*N nodes). The probability of a connection being an h-hop connection is inversely proportional to h. The start and end times of the connection are also distributed probabilistically, first around 3 timelines and then uniformly within one second from the timeline chosen. In this way convergence of an adaptation is taken into account in the results. We use 7 pre-encoded layers at 180, 128, 88, 64, 32, 16 and 8 Kbps CBR codecs with paired packets and 256, 128, 80, 64, 48, 32, 16 when mentioned. For each network we run 3 network sizes so that any relation with the scaling of the network is uncovered. When mobility is introduced a random waypoint model is used. Specifically 75% of the nodes (the S nodes) randomly pick a destination every 5 seconds and move to it at the reported speed. We vary speeds from 2-55 km/h.

5.1 802.11 single hop

In this set of graphs we first note that the network feedback adaptation performs very well with respect to loss rates. It has almost the same loss as the low layer transmission. We also see that the AB-probe performs best of all the end-to-end solutions, handling 20 connections at a less than 20% loss. Traditional RTP adaptation works well over a single hop, showing some possibility for QoS improvement, depending on the codec.

Figure 7.2 Loss Rates

5.1.1 QoS

In this paragraph we apply the QoS evaluation model we adopted to the loss rates as recorded every two seconds. A QoS value is calculated and then averaged over all connections and all time spans. This conclusive graph shows that with our high encoding complexity corresponding QoS model only Network Feedback strategies are capable of delivering better QoS across all network operation points. In fact it delivers the same QoS as the low layer does when there are more than 22 connections in the network. Before that the QoS is improved. On the other hand end-to-end strategies quickly deteriorate in terms of the delivered QoS, delivering much lower perceptual quality than the non adaptive low layer scheme. The AB-probe adaptation is again the best among end-to-end techniques, improving QoS when connections are less than 10 and deteriorating after that.

Figure 7.3. QoS, 802.11 single hop

Now let us use the higher rate layers with the highly VBR codecs. The results are very similar:

Figure 7.4 Loss Rates, 802.11 single hop, higher rates

Figure 7.5. QoS, 802.11 single hop, loss rates

5.2 802.11 multi-hop

In this section we show the 802.11 multihop results. The conclusions are very similar to the one hop results. Now however, the network feedback strategy starts to digress from the lower layer non-adaptive transmission level of loss rates. Network Feedback delay, overhead and small accuracy error start to show with a large number of connections (close to the network capacity). When the network can handle 45 low rate connections without loss and 58 with less than 20% loss the network feedback case allows 30 connections without loss and 48 with less than 20% loss.

Ab-probe performs best but starts performing worse than trial and error at the same point that network feedback loss is increased. It starts performing worse than RTP when loss rates are already very high at 45%. The same behavior was seen in the single hop case occurring at a 60% loss rate.

RTP adaptation works well over a single hop, better than direct PP measurement.

Figure 7.6 Loss rates on 802.11 multihop

5.2.1 QoS

Figure 7.7 QoS on 802.11 multi-hop

We show the graphs when using higher rate and variability layers here. An interesting additional note stemming for these graphs is on the situation of much higher traffic than the network can handle, i.e. very high loss rates that were not reached before in the low rate case. We note that the digression of network feedback loss rates from the loss rate loss rates is initially increasing and then decreasing, reaching the same exact loss rates in very high loss rate, low available bandwidth situations. This might be important when codec technology increases compression rates.

Figure 7.8 Loss rates in 802.11 multihop with higher rate video

Figure 7.9 QoS in 802.11 multihop with higher rate video

5.2.2 Mobility

In this section we present some results of our extensive experiments with mobility. We only present the 64 node network case since other cases are similar. We show the 5, 20 and 55 km/h cases. In mobile situations the accuracy of the measurement is not affected for all but the packet pair available bandwidth measurement. The increased loss rates are due to the inevitable mobility related losses and the initialization and convergence times of the filters.

Figure 7.10 5km/h mobility

Figure 7.11 20km/h mobility

Figure 7.12 55km/h mobility

Figure 7.13. QoS in 5 km/h mobility

Figure 7.14 QoS in 20 km/h mobility

Figure 7.15 QoS in 55km/h mobility

5.3 Bluetooth Scatternets

In Bluetooth networks the situation appears different. The network feedback is still fairly accurate but its convergence time now increases due to the gateway’s time division. In order for the correct available bandwidth of a pair involving a gateway to be estimated one full superframe cycle is required. The value during the long initialization phase increases as approaching the next Masters Rendezvous Point and then decreases until the gateway switches back to the destination. On the other hand all end-to-end measurements over the Master-centric MAC work much better than in the 802.11 asynchronous access control. Direct measurement with AB probe does not appear to have and advantage as far as loss rates are concerned in Bluetooth case. In fact it grows to doing slightly worse than other end-to-end methods when the network works in medium to high network loads. The packet pair measurement is more applicable in the Bluetooth case due to the polling scheme that better approximates WFQ. The RTP adaptation works well due to its better synchronization with the time division in gateways. Direct measurement measure high bandwidth when the gateway is aligned to their route and throw a large number of packets in the network, hard to deliver after the gateway switches out. RTP on the other hand performs more conservative stepwise increases. All end-to-end method loss rates appear significantly decreased as compared to the 802.11 case.

Figure 7.16 Bluetooth Scatternets Loss Rates

5.3.1 QoS

By looking at the QoS graphs we note similar situations as before. Besides the small loss rate differences we noted in the previous paragraph, the AB-probe does exhibit a better QoS. It loses some packets when the measurement is affected by the gateway switch but it sends closer to what it should sent resulting in increased QoS. Packet pair is slightly better in this environment. Network feedback delivers the best QoS always, the low rate QoS at worse, but deteriorates to being worse than the low rate when 42 or more connections are thrown in the network. The low rate loss rates are 10% at that point.

Figure 7.17 Bluetooth QoS

5.4 TCP Friendliness

In this section we are presenting results from our experiments with the co-existence of these adaptive mechanisms and TCP. We run all of the above experiments three times, first all connections are TCP – infinite backlog- instead of multimedia and second we replace half of those connections with an adaptive connection of each mechanism, third we use the TCP-friendly mechanism.

In the next figure we are presenting results that show the percentage of the connections that were TCP friendly of the replaced connections, i.e. the ones that sent less than their equivalent TCP in the TCP experiment. We do that for each connection participating on each experiments in the three sizes of networks above. We note that in all applicable situations this percentage is increased when the TCP friendly mechanism is used. In fact it reaches the fraction of the connections that were TCP friendly when the low rate transmission was used. This is the limit of course of TCP friendliness. Initially when a C*AB control was used even though TCP friendliness by the above definition was limited. However, note, that the above definition restricts the throughput of a connection without looking at the efficiency and utilization of the network. For example, assume that bandwidth is available but TCP cannot get it due to its slow converging mechanisms and loss reaction. An AB probing application will utilize this bandwidth without affecting TCP. Therefore a more interesting approach in TCP fairness in multimedia communication is to define a connection fair to TCP (same path to avoid RTT related fairness definitions) if it allows a competing TCP connection to get the same or more bandwidth as it would if that connection was TCP. This is of course an impractical definition since it cannot be implemented in a distributed system.

Figure 7.18 TCP Friendliness Experiment

References

[Agg00] A. Aggarwal, M. Kapoor, L. Ramachandran, A. Sakar “Clustering Algorithms for Wireless Ad Hoc Networks” Fourth International Workshop on Discrete Algorithms and Methods for Mobile Computing and Communications (DIAL M for Mobility) 2000

[Alw96] Alwan, A.;Bagrodia, R.;Bambos, N.;Gerla, M.;Kleinrock, L.;Short, J.; Villasenor, J. Adaptive mobile multimedia networks. IEEE Personal Communications, vol.3, (no.2), IEEE, April 1996. p.34-51

[Ard94] Ardito M., Barbero M., Stroppiana M. and Visca M. Compression and Quality. In Pro-ceedings of the International Workshop on HDTV 94, Brisbane, Australia, October 1994. Springer-Verlag.

[Bag91] R Bagrodia, K.M. Chandy and W-T Liao. A unifying framework for distributed simulations. ACM Trans. On Modeling and Computer Simulation, 1(4):348-385, 1991

[Ban01] D. Bansal, H. Balakrishnan, "Binomial congestion control algorithms," IEEE INFOCOM 2001, Anchorage, Alaska, April 2001. Extended version available as MIT-LCS-TR-806

[Bas01] Basagni, I. Chlamtac, G. V. Záruba, “Bluetrees – Scatternet Formation and Routing in Bluetooth-Based Ad Hoc Networks” Proceedings of IEEE INFOCOM 2001, Anchorage, Alaska, USA, April 22-26, 2001.

[Bia00] Bianchi, G. Performance analysis of the IEEE 802.11 distributed coordination function. IEEE JSAC March 2000. p.535-47

[Bluet] Specification of the Bluetooth System - Core vol.1 v1.1, www.bluetooth.com

[Boc96] V. Bochmann, G.; Kerherve, B.; Hafid, A.; Dini, P.; Pons, A. Architectural design of adaptive distributed multimedia systems. Proceedings International Workshop on Multimedia Software Development, IEEE Comput. Soc. Press, 1996. p.31-40. viii+177 pp.

[Bor98] Borella, M.S.; Swider, D.; Uludag, S.; Brewster, G.B. (Edited by: Das, C.R.) Internet packet loss: measurement and implications for end-to-end QoS. Proceedings of the 1998 ICPP Workshop on Architectural and OS Support for Multimedia Applications Flexible Communication Systems. Wireless Networks and Mobile Computing (Cat. No.98EX206), Los Alamitos, CA, USA: IEEE Comput. Soc, 1998. p.3-12. vii+155 pp.

[Brad99] P.T. Brady: " A model for generating on-o speech patterns in two-way conversation", Bell System Technical Journal, Sept. 1999, pp. 2445-2471.

[Can99] Cansever, D.H.; Michelson, A.M.; Levesque, A.H. Quality of service support in mobile ad-hoc IP networks. MILCOM 1999

[Cap01] Capone, R. Kapoor and M. Gerla: “Efficient Polling Schemes for Bluetooth Picocells” ICC 2001

[Car97] R. L. Carter, M. E. Crovella, "Server selection using dynamic path characterization in wide-area networks," Proceedings of IEEE INFOCOM '97, Kobe, Japan, pp.1014-1021, April 1997.

[Cas00] C. Casetti, M. Gerla, S. S. Lee, S. Mascolo, and M. Sanadidi, "TCP with Faster Recovery," Proceedings of IEEE Military Communications Conference MILCOM 2000, Los Angeles, USA, October 2000.

[Cha97] Chatterjee, S.; Sydir, J.; Sabata, B.; Lawrence, Modeling applications for adaptive QoS-based resource management. Proceedings. 1997 High-Assurance Engineering Workshop, IEEE Comput. Soc, 1997. p.194-201. xii+227 pp.

[Chiu-Jain] Raj Jain, K. K. Ramakrishnan, Dah-Ming Chiu “Congestion Avoidance in Computer Networks With a Connectionless Network Layer” Proceedings of the Computer Networking Symposium; IEEE; Washington, DC 1997

[Cla90] D. Clark and D. Tennenhouse, Architecural considerations for a new generation of protocols - Computer Communication Review vol20, num4,Sep 1990, pages 200-208

[Cla90] D.Clark and D.Tennenhouse, Architecural considerations for a new generation of protocols Computer Communication Review vol20, num4,Sep 1990, pages 200-208

[Cro97] Crow, B.P.; Widjaja, I.; Kim, J.G.; Sakai, P. Investigation of the IEEE 802.11 medium access control (MAC) sublayer functions. Proceedings IEEE INFOCOM '97

[Dal96] Dalgic I. and Tobagi F.A. Glitches as a Measure of Video Quality Degradation Caused by Packet Loss. In 7th International Workshop on Packet Video, pages 201–206, Brisbane, Australia, March 1996

[Das01] Das, A. Ghose, A. Razdan, H. Saran, R. Shorey, “Enhancing Performance of Asynchronous Data Traffic over the Bluetooth Wireless Ad-Hoc Network”, Proceedings of IEEE INFOCOM 2001, Anchorage, Alaska, USA, April 22-26, 2001

[Dov01] Constantinos Dovrolis, Parameswaran Ramanathan, and David Moore. What do packet dispersion techniques measure? In Proceedings of IEEE INFOCOM, April 2001.

[Est99] Deborah Estrin, Ramesh Govindan, John Heidemann and Satish Kumar Next Century Challenges: Scalable Coordination in Sensor Networks ACM MobiCom 99, August 99, Seattle, USA.

[Fab98] Faber, T. ACC: using active networking to enhance feedback congestion control mechanisms. IEEE Network, vol.12, (no.3), IEEE, May-June 1998. p.61-5.

[Flo99] S. Floyd, K. Fall, "Promoting the use of end-to-end congestion control in the Internet," IEEE/ACM Transactions on Networking, vol. 7, no. 4, pp. 458-472, Aug. 1999.

[Fro98] Frossard P. and Verscheure O. MPEG-2 Video over Lossy Packet Networks: Perceptual Syntactic Information Coding. In SPIE’s International Symposium on Voice, Video, and Data Communications, Boston, November 1998.

[Fuk97] Fukuda K., Wakamiya N., Murata M. and Miyahara H. QoS Mapping between User’s Preference and Bandwidth Control for Video Transport. In Proceedings of the Interna-tional Workshop on QoS, pages 291–302, May 1997

[Ger01] Mario Gerla, Rohit Kapoor, Manthos Kazantzidis (UCLA), Per Johansson (Ericsson) “Ad hoc Networking with Bluetooth” WMI at Mobicom 2001

[Gup00] Piyush Gupta and P. R. Kumar, ``The Capacity of Wireless Networks,'' IEEE Transactions on Information Theory, vol. IT-46, no. 2, pp. 388-404, March 2000.

[Haa98] J. Haartsen, M. Naghshineh, J. Inouye, O.J. Joeressen, and W. Allen "Bluetooth: Vision, Goals, and Architecture" ACM SIGMOBILE Mobile Computing and Communications Review, vol. 2, no. 4, Oct. 1998, pp. 38-45.

[Hei99] Heider, G.; Fink, B. Development environment for creating the first Bluetooth products. Electronic Product Design, vol.20, (no.5), IML Techpress, May 1999. p.24-5, 27

[Hwu98] Hsiao-Kuang Wu; Hung, C.-H.; Gerla, M.; Bagrodia, R. Evaluating speech quality in large wireless networks: a case for hybrid simulation. ICC '98. Affiliated with SUPERCOMM'98 p.498-502 vol.1. 3 vol. xxxvii+1838 pp

[Ieee80211] IEEE Std 802.11 - Wireless LAN Medium Access Control and Physical Layer specifications

[Joh00] P. Johansson, R. Kapoor, M. Kazantzidis, M. Gerla - Rendezvous Scheduling in Bluetooth Scatternets – ICC 2002

[Joh01] N. Johansson, U. Körner, L. Tassiulas, “A Distributed Scheduling Algorithm for a Bluetooth Scatternet,” Seventeenth International Teletraffic Congress, ITC’17, Salvador da Bahia, Brazil, September 24-28, 2001.

[Jon96] Johnson, D.B., and Maltz, D.A., "Dynamic Source Routing in Ad-Hoc Wireless Networks," in Mobile Computing, edited by T. Imielinski and H. Korth, chapter 5, pp. 153-181, Kluwer, 1996

[Jum01] N. Johansson, F. Alriksson, U. Jonsson "JUMP Mode- A Dynamic Window-based Scheduling Framework for Bluetooth Scatternets", ACM MobiHoc 2001.

[Kal99] Kalia, M.; Bansal, D.; Shorey, R. “MAC scheduling and SAR policies for Bluetooth: a master driven TDD pico-cellular wireless system.” 1999 IEEE International Workshop on Mobile Multimedia Communications (MoMuC'99)

[Kaz01b] M Kazantzidis, M Gerla, “Permissible Throughput Network Feedback for Adaptive Multimedia in AODV MANETs” ICC 2001

[Kaz01t] M. Kazantzidis, Locally optimal forwarding throughput Bluetooth scatternets– UCLA CS Technical report #010033

[Kaz02] M Kazantzidis, MGerla - On the Impact of Inter-Piconet Scheduling in Bluetooth Scatternets, WWIC 2002

[Kaz99] T.Chen, M. Gerla, M. Kazantzidis, Y. Romanenko, I. Slain Experiments on QoS Adaptation for Improving End User Speech Perception Over Multi-Hop Wireless Networks. In Proceedings of QoS Mini Conference in conjunction with IEEE ICC'99, Vancouver, Canada, Jun. 1999.

[Kaz99b] M. Kazantzidis, L. Wang, and M. Gerla, On Fairness and Efficiency of Adaptive Audio Application Layers for Multihop Wireless Networks IEEE MOMUC'99

[Kaz99c] Manthos Kazantzidis, Tsuwei Chen, Yuri Romanenko, Mario Gerla, “An ultimate encoding layer for real-time packetized voice streaming and experiments over a multi-hop wireless network” - Second Annual UCSD Conference on Wireless Communications, San Diego, California, March 1999

[Kes94] S. Keshav, "Packet-pair flow control" IEEE/ACM Transactions on Networking 1994

[Kok00] Can Emre Koksal, Hisham Kassab, and Hari Balakrishnan “An Analysis of Short-Term Fairness in Wireless Media Access Protocols” Measurement and Modeling of Computer Systems 2000

[Lai01] K. Lai, M. Baker, "Nettimer: A tool for measuring bottleneck link bandwidth", Proceedings of the USENIX Symposium on Internet Technologies and Systems, San Francisco, CA, USA, March 2001.

[Lai99] K. Lai, M. Baker, "Measuring bandwidth", Proceedings of IEEE INFOCOM '99, New York, NY, USA, pp. 235-245, March 1999

[LaiB00] K. Lai, M. Baker, "Measuring link bandwidths using a deterministic model of packet delay", Proceedings of ACM SIGCOMM 2000, Stockholm, Sweden, Aug./Sept. 2000; Computer Communication Review, vol. 30, no. 4, pp. 283-294, Oct. 2000.

[Leg00] A. Legout, E. W. Biersack, "PLM: fast convergence for cumulative layered multicast transmission schemes," Proceedings of ACM SIGMETRICS '2000, Santa Clara, CA, USA, June 2000; Performance Evaluation Review, vol. 28, no. 1, pp. 13-22, June 2000.

[Lin99] C. R. Lin and J.-S. Liu, "QoS Routing in Ad Hoc Wireless Networks", IEEE Journal on Selected Areas in Communications, V. 17, No. 8, pp. 1426-1438, Aug. 1999.

[Liu94] Liu Y, A. Singh and R. Bagrodia. A decompositional approach to the design of efficient parallel programs. IEEE Transactions on Software Engineering, 20(12):914-932, 1994

[Luo00] A New Model for Packet Scheduling in Multihop Wireless Networks Haiyun Luo, Songwu Lu (UCLA, USA) and Vaduvur Bharghavan (University of Illinois at Urbana-Champaign, USA)

[Mas01] S. Mascolo, C. Casetti, M. Gerla, M. Yours truly,. Sanadidi, R. Wang, "TCP Westwood: Bandwidth estimation for enhanced transport over wireless links" Proceedings of Mobicom 2001, Rome, Italy, Jul. 2001.

[McI98] McIlhagga, M.; Light, A.; Wakeman Towards a design methodology for adaptive applications. MobiCom'98. Proceedings of Fourth Annual ACM/IEEE International Conference on Mobile Computing and Networking, ACM, 1998. p.133-44. viii+292 pp.

[MicSR] Microsoft Speech Suite Documentation http://www.microsoft.com/stg - Microsoft Corp.

[Mik00] Miklos G.; Racz, A.; Turanyi, Z.; Valko, A.; Johansson, P. “Performance aspects of Bluetooth scatternet formation” 2000 First Annual Workshop on Mobile and Ad Hoc Networking and Computing. MobiHOC

[Nad00] T. Nandagopal, T. Kim, X. Gao and V. Bharghavan, Achieving MAC Layer Fairness in Wireless Packet Networks. Mobicom 2000

[Nak98] Nakajima, Y.; Yanagihara, H.; Yoneyama, A.; Sugano, M. MPEG audio bit rate scaling on coded data domain. Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181), (vol.6), New York, NY, USA: IEEE, 1998. p.3669-72 vol.6. 6 vol. lxiii+3816 pp.

[Neg98] K.J. Negus, J. Waters, J. Tourrilhes, C. Romans, J. Lansford, and S. Hui "HomeRF and SWAP: Wireless Networking for the Connected Home" ACM SIGMOBILE Mobile Computing and Communications Review, vol. 2, no. 4, Oct. 1998, pp. 28-37.

[NgK97] Ng, K.T.; Chan, S.C.; Ng, T.S. An adaptive multiresolution modification of the H.263 video coding algorithm. Proceedings of ICICS, 1997 International Conference on Information, Communications and Signal Processing. Theme: Trends in Information Systems Engineering and Wireless Multimedia Communications (Cat. No.97TH8237), (vol.1), New York, NY, USA: IEEE, 1997. p.288-91 vol.1. 3 vol. xxxiv+1819 pp.

[NJo01] N. Johansson, U. Körner, L. Tassiulas, “A Distributed Scheduling Algorithm for a Bluetooth Scatternet,” Seventeenth International Teletraffic Congress, ITC’17, Salvador da Bahia, Brazil, September 24-28, 2001

[Nmag01] Per Johansson (Ericsson), Rohit Kapoor, Mario Gerla, Manthos Kazantzidis (UCLA) – “Bluetooth an Enabler of Personal Area Networking” - IEEE Network, Special Issue in Personal Area Networks, Sept-Oct 2001

[Nsm02] The Network Simulator 2 with CMU extensions www-mash.cs.berkeley.edu/ns/

[Ott98] Ott, M.; Michelitsch, G.; Reininger, D.; Welling, G. An architecture for adaptive QoS and its application to multimedia systems design. Computer Communications, vol.21, (no.4), Elsevier, NEC, 10 April 1998. p.334-49.

[Pax97] V. Paxson, Measurements and Analysis of End-to-End Internet Dynamics, Ph.D. Thesis, University of California, Berkeley, 1997.

[Per99] Ad-hoc on-demand distance vector routing, Perkins, C.E.; Royer, E.M. Mobile Computing Systems and Applications, 1999. Proceedings. WMCSA '99. Second IEEE Workshop on , 1999 , Page(s): 90 –100

[Per00] Charles E. Perkins, Elizabeth M. Royer, Samir R. Das “Quality of Service for Ad hoc On-Demand Distance Vector Routing” – IETF Draft

[Pod99] Podolsky, S. McCanne, M. Vettereli Soft ARQ for layered streaming media. Technical Report UC Berkeley 1999

[Pou00] D. Poulton, J. Oksman, "Digital filters for non uniformly sampled signals," NORSIG 2000, Nordic Signal Processing Symposium, Vildmarkshotellet Kolmarden, Sweden, pp. 421-424, June 2000.

[Rac01] A. Racz, G. Miklos, F. Kubinszky, A. Valko "A Pseudo Random Coordination Scheduling Algorithm for Bluetooth Scatternets'', ACM MobiHoc 2001.

[Rej00] Reza Rejaie – An End to End Architecture for Quality Adaptative Streaming Applications in the Internet . USC PhD Disseration

[Rej99] Reza Rejaie, Mark Handley, Haobo Yu, Deborah Estrin. Proxy Caching Mechanism for Multimedia Playback Streams in the Internet. AT&T Center for Internet Research at ICSI, April 16, 1999

[Rom98] Yuri Romanenko, "Qos Adaptation for Wireless Networks" -MS Thesis UCLA CS 1998

[Rtp96] H. Schulzrinne et al., RTP: A Transport Protocol for Real-Time Applications, RFC 1889 Jan 1996

[Sal01] T. Salonidis, P. Bhagwat, L. Tassiulas, R. La Maire “Distributed topology construction of Bluetooth personal area networks.” Infocom 2001

[Ses97] S. Seshan, M. Stemm, R. H. Katz, "SPAND: shared passive network performance discovery," Proceedings of the USENIX Symposium on Internet Technologies and Systems, Monterey, CA, pp. 135-146, Dec. 1997.

[Sis97] D. Sisalem, End-to-End Quality of Service Control using Adaptive Applications IFIP Fifth International Workshop on Quality of Service 1997

[Sis98] Sisalem, D.; Emanuel, F. QoS control using adaptive layered data transmission. Proceedings. IEEE International Conference on Multimedia Computing and Systems :IEEE Comput. Soc, 1998. p.4-12. xiv+373 pp.

[Sla97] Ilya Slain, "A Programming Model for Adaptive QoS-aware Audio Applications" - MS Thesis UCLA CS 1997

[Sta00] Star Wars Trailer Episode IV – http://www.starwars.com

[Tak99] M. Takai L, Bajaj, R, Ahuja, R. Bagrodia and M. Gerla, “GloMoSim: A Scalable Network Simulation environment” Technical report 990027, UCLA, Computer Science Department, 1999.

[Tan99] Fair Sharing of MAC under TCP in Wireless Ad Hoc Networks.Ken Tang and Mario Gerla.Proceedings of IEEE MMT'99, Venice, Italy, October 1999.

[Tsa95] J.T. Tsai and M. Gerla Multicluster, Mobile, Multimedia Radio Network ACM-Baltzer Journal of Wireless Networks, Vol.1, No.3, pp.255-65, 1995

[Van96] Van den Branden Lambrecht C.J. and Verscheure O. Perceptual Quality Measure using a Spatio-Temporal Model of the Human Visual System. In Proceedings of the SPIE, volume 2668, pages 450–461, San Jose, USA, February 1996.

[Ver96] Verscheure O., Basso A., El-Maliki M. and Hubaux J.-P. Perceptual Bit Allocation for MPEG-2 CBR Video Coding. In Proceedings of the International Conference on Image Processing, Lausanne, Switzerland, September 1996

[Yao97] Jey-Hsin Yao, Yao-Min Chen. Experiments of real-time MPEG audio over the Internet. Fujitsu Scientific and Technical Journal, vol.33, (no.2), Fujitsu, 1997. p.138-44

[Zha98] Qian Zhang, Ya-Qin Zhang, and Wenwu Zhu, “Resource Allocation for Audio and Video Streaming over the Internet” - ISCAS 1998