The features and capacities embod ied in the modern 3G mobile net work make
for a complex system with many modes, nodes, elements, interfaces, and
protocols. Consequently, problems, when they arise, can also have multiple
sources of origin–in hardware or software. As mobile Internet connectivity
becomes common, the challenge of maintaining uninterrupted data transactions
will require newer and more powerful monitoring solutions and procedures.
Call Set-up Failure
In order to obtain packet data services, the mobile performs registration
with the serving wireless network on the A1 interface and then with the packet
network on the A10/A11 interface. The mobile sends an origination message to the
base station (BS) that includes the packet data service option. This results in
assignment of the traffic channel, establishment of a A10 connection,
establishment of the link layer (PPP) and in cases where Mobile IP is used by
the terminal, Mobile IP registration with the serving packet network. User data
traffic then passes over the A10 connection encapsulated within generic routing
encapsulation (GRE) frames.
The packet control function (PCF) periodically re-registers with the selected
packet data serving node (PDSN) by sending the A11-Registration Request message
before the A10 connection lifetime expires.
A successful call set-up scenario is illustrated in the diagram above. This
standard message sequence chart outlines a series of steps, summarized in items
1—12 to follow. Note that this explanation bypasses the radio
reception/transmission activities of the base transmission system (BTS),
concentrates on the protocol functions instead which begin with the origination
dialogue between the mobile and the BSC.
- To register for packet data services, the mobile sends an origination
message over the access channel to the BSS.
- The BS acknowledges the receipt of the origination message, returning a
base station Ack order to the mobile.
- The BS constructs a CM service request message and sends the message to
- The mobile switching center (MSC) sends an assignment request message to
the BSS requesting assignment of radio
resources. No terrestrial circuit between the MSC and the BS is assigned to
the packet data call.
- The BS and the mobile perform radio resource set-up procedures. The packet
control function (PCF) recognizes that no A10 connection associated with
this mobile is available and selects a PDSN for this data call.
- The PCF sends an A11 registration request to the selected PDSN.
- The A11 registration request is validated and the PDSN accepts the
connection by returning an A11 registration reply message. Both the PDSN and
the PCF create a binding record for the A10 connection.
- After the radio link and A10 connection are setup, the BS sends an
assignment complete message to the MSC.
- The mobile and the PDSN establish the link-layer (PPP) connection and then
perform the MIP registration procedures over the PPP connection.
- After completing MIP registration, the mobile can send/receive data via
GRE framing over the A10
- The PCF periodically sends an A11 registration request message for
refreshing registration for the A10
- For a validated A11 registration request, the PDSN returns an A11
registration reply message. Both the PDSN and the PCF update the A10
connection binding record.
This complex process can be a source of problems. A rigorous monitoring
scheme involving simultaneous observation of the A1 interface and the
A10/A11 interface is the best way to detect and correct errors early. Here,
a multi-interface call-trace application is especially productive, since it
can trace and group all of the procedures related to the activity of each
single subscriber in a CDMA network, even as the procedures evolve over
Within the call set-up process, an error in any element or procedural step
can inhibit the remaining steps. For example, suppose that the MSC does not
respond to the CM service request message (Step 3 in the figure) sent by the BSC/PCF
over the A1 interface. This is at times caused by internal MSC problems. If this
prevents the completion of the CM service request, the BSC/PCF cannot assign
radio resources to the mobile station, thus preventing establishment of the
connection. The user finds it impossible to make a data call–a service for
which he has paid a premium. Before a specific timer expires, the PCF sends
periodically A11 registration request message (Step 11) to refresh the
registration for the A10 connection. For a validated A11 registration request,
the PDSN returns an A11 registration reply message (Step 12).
Here, internal problems in the PDSN can cause it to respond late or not at
all. As a result the process of establishing or maintaining the connection
cannot continue. The user is once again unable to make a data call. In both
cases, a protocol analyzer connected to the A1 and A10/A11 interfaces can help
track down the problem. The call trace application can make out the origin of
messages and detect failures to respond.
Frequently in a CDMA2000 network the TCP user-plane packets are of small
window size. This is a by-product of the soft-start mechanism of in the TCP
protocol and also implies that end-to-end TCP connections are not stable. The
more TCP packets lost in the network and not acknowledged, the smaller the
window size, resulting in dropping and re-establishing of more TCP connections.
To analyze this problem, it is necessary to capture the TCP/IP user plane
packets flowing on the GRE tunnels on the A10 interface. Protocol filtering
allows the tool to home in on just the data of interest. By applying different
types of filtering with increasing level of details, it is possible to ‘drill
down’ and isolate the root cause of the shrinking TCP packet Window size.
Routing Loop Problems
‘Tunnel router loops’ are another class of CDMA2000 network problems.
The problem is caused by misconfiguration in the PDSN routers, which can be
detected by acquiring and analyzing IP traffic on the P-H interface. To
understand tunnel router loops, imagine a subscriber surfing the Web with a
laptop connected to a CDMA2000 handset. Packets addressed to go to a specific
HTTP proxy are routed from the PDSN/FA (foreign agent) to the home agent (HA)
With certain incorrect internal routing configurations, packets destined for
Port 80 www are not de-tunneled by the HA. Instead, they are sent back
downstream toward the PDSN/FA. As a result, multiple packets travel on the same
network segment with the same packet ID, wasting precious bandwidth–and not
reaching the intended destination. In addition, for each repetitive hop a packet
takes between the PDSN/FA and HA nodes, the IP time to live (TTL) field is
decremented. If the packet is stuck in a router loop, the TTL eventually
decrements to zero and the packet is discarded by the network nodes. ‘Lost’
packets must be retransmitted. This means more packet retransmission and reduced
As in the earlier examples, the solution is to use protocol filtering to
capture IP packets on the PH interface. Browsing through the captured data by
applying increasingly fine levels of filtering, it is possible to see the
repeating packets and resolve the problem.
Duplication of IP Traffic
PDSN configuration problems can give rise to other types of problems in addition
to tunnel loops. One common issue is associating the PDSN’s logical IP
addresses with more than one physical medium access control (MAC) address. When
this occurs, more than one hardware card has the same IP address. All traffic
sent to that IP address goes to two different hardware entities and receives
responses from both. This effectively doubles the amount of IP traffic
associated with that single IP address on that segment. Once again,
protocol-filtering capabilities are required for effective troubleshooting. A
protocol analyzer should capture IP packets traveling to a specific IP
destination address via the PH interface. Browsing through the data and
filtering helps narrow down the inquiry to give the nature of the problem (the
duplicated address) soon.
Sometimes internal problems can cause PDSN routers to go offline and come
back online after some time. This can happen frequently on a CDMA2000 core data
network. When a router first comes online, its routing tables are not optimized.
It takes time for the built-in open shortest path first (OSPF) routing algorithm
to learn the best way to route packets. Until the routing tables are optimized,
there will be degradation in quality of service.
By capturing IP packets on the PH interface with a protocol analyzer and
applying filters on the OSPF routing messages, changes in designated router and
changes in neighbors of a router can easily be identified. Using intelligent and
detailed filtering capability on OSPF messages and information elements within
these messages identifying routing problems on an IP network becomes an easy
Troubleshooting activities now require an understanding of both traditional
‘telecom’ concepts related to the circuit-switched domain and new ‘datacom’
concepts related to the packet switched-domain. Protocol analysis tools can play
a bigger role than ever in keeping a network running efficiently. Features such
as multi-interface call tracing and protocol filtering will become critical for
Excerpted from a white paper by Enrico Zanoio and Steve Urvik (monitoring and
protocol test) Tektronix Inc