FL15A ZoneConfigGuide App
Comments
Description
Nokia NetworksLTE Radio Access, Rel. FDD-LTE15A, Operating Documentation, Issue 02, Documentation Change Delivery 2 Configuring Flexi Zone Controller LTE Transport DN09224449 Issue 01B Approval date 2015-12-08 Disclaimer Configuring Flexi Zone Controller LTE Transport The information in this document is subject to change without notice and describes only the product defined in the introduction of this documentation. This documentation is intended for the use of Nokia Solutions and Networks customers only for the purposes of the agreement under which the document is submitted, and no part of it may be used, reproduced, modified or transmitted in any form or means without the prior written permission of Nokia Solutions and Networks. The documentation has been prepared to be used by professional and properly trained personnel, and the customer assumes full responsibility when using it. Nokia Solutions and Networks welcomes customer comments as part of the process of continuous development and improvement of the documentation. The information or statements given in this documentation concerning the suitability, capacity, or performance of the mentioned hardware or software products are given "as is" and all liability arising in connection with such hardware or software products shall be defined conclusively and finally in a separate agreement between Nokia Solutions and Networks and the customer. However, Nokia Solutions and Networks has made all reasonable efforts to ensure that the instructions contained in the document are adequate and free of material errors and omissions. Nokia Solutions and Networks will, if deemed necessary Nokia Solutions and Networks, explain issues which may not be covered by the document. Nokia Solutions and Networks will correct errors in this documentation as soon as possible. IN NO EVENT WILL NOKIA SOLUTIONS AND NETWORKS BE LIABLE FOR ERRORS IN THIS DOCUMENTATION OR FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA, THAT MAY ARISE FROM THE USE OF THIS DOCUMENT OR THE INFORMATION IN IT. NSN is a trademark of Nokia Solutions and Networks. Nokia is a registered trademark of Nokia Corporation. Other product names mentioned in this document may be trademarks of their respective owners, and they are mentioned for identification purposes only. Copyright © Nokia Solutions and Networks 2015. All rights reserved. Nokia Solutions and Networks are continually striving to reduce the adverse environmental effects of its products and services. We would like to encourage you as our customers and users to join us in working towards a cleaner, safer environment. Please recycle product packaging and follow the recommendations for power use and proper disposal of our products and their components. If you should have questions regarding our Environmental Policy or any of the environmental services we offer, please contact us at Nokia Solutions and Networks for additional information. 2 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Table of Contents Table of Contents This document has 157 pages. 1 Introduction ....................................................................................................... 9 1.1 Purpose ............................................................................................................. 9 1.2 Overview of FDD-LTE15A reference configurations ......................................... 9 1.3 Feature dependency overview ........................................................................ 10 1.4 Network performance ...................................................................................... 13 1.4.1 User plane ....................................................................................................... 13 1.4.2 Control plane ................................................................................................... 14 1.4.3 Management plane ......................................................................................... 14 1.4.4 Synchronization plane ..................................................................................... 14 1.4.5 Summary ......................................................................................................... 15 1.5 Document Icons .............................................................................................. 16 1.6 Network Element Icons ................................................................................... 17 2 General aspects .............................................................................................. 19 2.1 Hardware......................................................................................................... 19 2.1.1 FZAP ............................................................................................................... 19 2.1.2 Synchronization............................................................................................... 23 2.1.3 Operation & maintenance ............................................................................... 23 2.2 Traffic dimensioning ........................................................................................ 23 2.2.1 Dimensioning assumptions ............................................................................. 23 2.2.2 Dimensioning results ....................................................................................... 25 2.2.3 Ethernet OAM dimensioning ........................................................................... 26 2.2.4 IPsec impact on dimensioning ........................................................................ 27 2.2.5 WiFi dimensioning ........................................................................................... 28 2.3 Quality of Service model ................................................................................. 29 2.3.1 FZAP Traffic classification............................................................................... 29 2.3.2 FZC Traffic Classification ................................................................................ 31 2.3.3 FZAP uplink scheduling .................................................................................. 32 2.3.4 FZC Packet Scheduling .................................................................................. 35 2.3.5 MME / SAE-GW downlink scheduling ............................................................. 35 2.3.6 Measurement-Based Transport Admission Control ........................................ 35 2.4 Ethernet switching in FZAP ............................................................................. 36 2.4.1 Egress Packet Scheduling .............................................................................. 36 2.4.2 Egress Rate Shaping ...................................................................................... 37 © 2015 Nokia Networks DN09224449 Issue 01B 3/157 Configuring Flexi Zone Controller LTE Transport 2.4.3 Ingress Rate Limiting ...................................................................................... 37 2.4.4 FZAP Port Traffic Roles .................................................................................. 38 2.5 Ethernet OAM ................................................................................................. 38 2.6 IP addressing in Flexi Zone............................................................................. 38 2.7 IP layer fragmentation and network MTU ........................................................ 39 2.7.1 Jumbo frames supported end-to-end .............................................................. 41 2.7.2 Jumbo frames not supported end-to-end ........................................................ 41 2.7.3 Minimum burst size ......................................................................................... 42 2.8 Synchronization............................................................................................... 43 2.9 Security ........................................................................................................... 44 2.9.1 Flexi Zone Access Point local network security features ................................ 45 2.10 Auto configuration ........................................................................................... 45 2.11 Shared Backhaul ............................................................................................. 46 2.12 Extended Redundancy Mechanisms............................................................... 47 2.13 Transport Network Layer (TNL) IP Fragmentation Avoidance ........................ 48 3 Layer 3 access network with IPsec ................................................................. 50 3.1 Recommended network configuration............................................................. 50 3.1.1 Configuration overview.................................................................................... 51 3.1.2 Feature usage and interdependences ............................................................ 54 3.1.3 Transport network elements............................................................................ 54 3.2 RAN parameter examples for the configuration .............................................. 55 3.2.1 Backhaul network VPN service configuration ................................................. 55 3.2.2 Flexi Zone detailed configurations .................................................................. 56 4 Multiple Enterprise Zone Shared Backhaul ..................................................... 68 5 References ...................................................................................................... 71 6 Glossary .......................................................................................................... 72 7 Appendix A ...................................................................................................... 74 A. Appendix A – FZC North Bound Site Solutions ............................................... 75 A.1 Separate Physical Interfaces with Separation of Application Addresses with IPSec – Separate C/M/U ................................................................................. 76 A.1.1 Overview: ........................................................................................................ 76 A.1.2 Delete base/default factory configurations and IP addresses ......................... 77 A.1.3 Example Network Addresses and Configuration ............................................. 78 A.1.4 Add Network configuration for NB on FZC ...................................................... 78 4 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Table of Contents A.1.5 IPSec configuration ......................................................................................... 79 A.2 Collapsed Physical Interface with Separation of Application Addresses with IPSec – Separate C/M/U ................................................................................. 80 A.2.1 Overview: ........................................................................................................ 80 A.2.2 Delete base/default factory configurations and IP addresses ......................... 81 A.2.3 Example Network Addresses and Configuration ............................................. 82 A.2.4 Add Network configuration for NB on FZC ...................................................... 83 A.2.5 IPSec configuration ......................................................................................... 84 A.3 Collapsed Physical Interface and Separate Application Addresses without IPSec – Separate C/M/U addresses ............................................................... 85 A.3.1 Overview: ........................................................................................................ 85 A.3.2 Delete base/default factory configurations and IP addresses ......................... 86 A.3.3 Example Network Addresses and Configuration ............................................. 87 A.3.4 Add Network configuration for NB on FZC ...................................................... 87 A.4 Collapsed Physical Interface and Separate Application Addresses with IPSec – Separate C/M/U addresses .......................................................................... 89 A.4.1 Overview: ........................................................................................................ 89 A.4.2 Delete base/default factory configurations and IP addresses ......................... 90 A.4.3 Example Network Addresses and Configuration ............................................. 91 A.4.4 Add Network configuration for NB on FZC ...................................................... 91 A.4.5 IPSec configuration using FZC wrappers ........................................................ 92 A.5 Separate Physical Interfaces and Application Addresses without IPSec – Mplane Address behind C-plane ........................................................................ 93 A.5.1 Overview: ........................................................................................................ 93 A.5.2 Delete base/default factory configurations and IP addresses ......................... 94 A.5.3 Network Addresses and Configuration ............................................................ 95 A.5.4 Add Network configuration for NB on FZC ...................................................... 95 A.6 Collapsed Physical Interface with Separate Application Addresses without IPSec – M-plane Address behind C-plane ...................................................... 97 A.6.1 Overview: ........................................................................................................ 97 A.6.2 Delete base/default factory configurations and IP addresses ......................... 98 A.6.3 Network Addresses and Configuration ............................................................ 99 A.6.4 Add Network configuration for NB on FZC .................................................... 100 A.7 Separate Physical Interfaces with shared interface and application addresses without IPSec – Combined C/M and Separate U-plane ................................ 101 A.7.1 Overview: ...................................................................................................... 101 A.7.2 Delete base/default factory configurations and IP addresses ....................... 102 © 2015 Nokia Networks DN09224449 Issue 01B 5/157 Configuring Flexi Zone Controller LTE Transport A.7.3 Network Addresses and Configuration .......................................................... 103 A.7.4 Add Network configuration for NB on FZC .................................................... 103 A.8 Collapsed Physical Interface with shared interface and application addresses without IPSec – Combined C/M and Separate U-plane ................................ 105 A.8.1 Overview: ...................................................................................................... 105 A.8.2 Delete base/default factory configurations and IP addresses ....................... 106 A.8.3 Network Addresses and Configuration.......................................................... 107 A.8.4 Add Network configuration for NB on FZC .................................................... 107 A.9 Separate Physical Interfaces with shared interface and application addresses with IPSec – Combined C/M and Separate U-plane ..................................... 109 A.9.1 Overview: ...................................................................................................... 109 A.9.2 Delete base/default factory configurations and IP addresses ....................... 111 A.9.3 Network Addresses and Configuration .......................................................... 112 A.9.4 Add Network configuration for NB on FZC .................................................... 112 A.10 Collapsed Physical Interface with shared interface and application addresses with IPSec – Combined C/M and Separate U-plane ..................................... 114 A.10.1 Overview: ...................................................................................................... 114 A.10.2 Delete base/default factory configurations and IP addresses ....................... 116 A.10.3 Network Addresses and Configuration .......................................................... 117 A.10.4 Add Network configuration for NB on FZC .................................................... 117 A.11 Separate Physical Interfaces with shared interface and application addresses without IPSec – Separate C/M/U planes ....................................................... 119 A.11.1 Overview: ...................................................................................................... 119 A.11.2 Delete base/default factory configurations and IP addresses ....................... 120 A.11.3 Network Addresses and Configuration .......................................................... 121 A.11.4 Add Network configuration for NB on FZC .................................................... 121 A.12 Collapsed Physical Interface with shared interface and application addresses without IPSec – Separate C/M/U planes ....................................................... 123 A.12.1 Overview: ...................................................................................................... 123 A.12.2 Delete base/default factory configurations and IP addresses ....................... 124 A.12.3 Network Addresses and Configuration .......................................................... 125 A.12.4 Add Network configuration for NB on FZC .................................................... 125 A.13 Separate Physical Interfaces with shared interface and application addresses with IPSec – Separate C/M/U planes ............................................................ 127 A.13.1 Overview: ...................................................................................................... 127 A.13.2 Delete base/default factory configurations and IP addresses ....................... 129 A.13.3 Network Addresses and Configuration .......................................................... 130 6 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Table of Contents A.13.4 Add Network configuration for NB on FZC .................................................... 130 A.14 Collapsed Physical Interfaces with shared interface and application addresses with IPSec – Separate C/M/U planes ............................................................ 132 A.14.1 Overview: ...................................................................................................... 132 A.14.2 Delete base/default factory configurations and IP addresses ....................... 134 A.14.3 Network Addresses and Configuration .......................................................... 135 A.14.4 Add Network configuration for NB on FZC .................................................... 135 8 Appendix B .................................................................................................... 137 B. Appendix B – ToP Connectivity Options in Zone .......................................... 138 B.1 TOP-F Sync using IP Unicast with GMC/BC present in Zone Network in NonDaisy Chain Mode ......................................................................................... 139 B.1.1 Overview ....................................................................................................... 139 B.1.2 Configuration via BTS Site Manager ............................................................. 140 B.2 TOP-P Sync using Ethernet Multicast with GMC/BC present in Zone Network in Non-Daisy Chain Mode ............................................................................. 141 B.2.1 Overview ....................................................................................................... 141 B.2.2 Configuration via BTS Site Manager ............................................................. 143 B.3 TOP-P Sync using IP Unicast with GMC/BC present in Zone Network in NonDaisy Chain Mode ......................................................................................... 146 B.3.1 Overview ....................................................................................................... 146 B.3.2 Configuration via BTS Site Manager ............................................................. 146 B.4 TOP-F Sync using IP Unicast with GMC/BC present in Operator Core Network in Non-Daisy Chain Mode ............................................................................. 150 B.4.1 Overview ....................................................................................................... 150 B.4.2 Configuration via BTS Site Manager ............................................................. 151 B.5 TOP-P Sync using IP Unicast with GMC/BC present in Operator Core Network in Non-Daisy Chain Mode ............................................................................. 154 B.5.1 Overview ....................................................................................................... 154 B.5.2 Configuration via BTS Site Manager ............................................................. 155 © 2015 Nokia Networks DN09224449 Issue 01B 7/157 Summary of changes Changes between issues 01A (2015-11-20, FDD-LTE15A) and 01B (201512-08, FDD-LTE15A) • Chapter 8 (Appendix B) was added Changes between issues 01 (2015-09-17, FDD-LTE15A) and 01A (201511-20, FDD-LTE15A) • Chapter 7 (Appendix A) was added Issues 01 (2015-09-17, FDD-LTE15A) is the first issue for FDD-LTE15A 8 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport 1 Introduction Introduction 1.1 Purpose LTE RAN transport features allow many different configurations, suitable for different transport networks and different strategies related to transport cost savings and provisioning of end-user services. This document is a guideline for network planners, providing the required information to configure the transport network and LTE Flexi Zone Controller (FZC) transport features for a collection of representative cases. This document defines recommended transport network configurations for logical S1, X2, management and synchronization interfaces of LTE FDDLTE15A system releases of NOKIA. Configuring Flexi Zone Controller LTE Transport provides general descriptions of the recommended network configurations. The feature set and related parameters given for the network configurations have been chosen as assumed being representative for configurations available in FDD-LTE15A. As shortcut throughout the document it is referred to FDD-LTE15A reference configurations. 1.2 Overview of FDD-LTE15A reference configurations Section 1 and section 2 represent the introduction and the common feature overview part of the document. Section 3 describes in detail the FDD-LTE15A reference configurations: Layer 3 access network with IPsec : This configuration is mainly based on a backhaul network comprised of IP routers in the access area, IPsec is deployed between security gateways attached to core network edge routers and Zone eNodeB. Three FZC northbound backhaul configurations are discussed and two southbound Z1 backhaul configurations are discussed. © 2015 Nokia Networks DN09224449 Issue 01B 9/157 Section 4 describes the configuration of the shared backhaul capability. This configuration provides support for creating multiple enterprises under the same Flexi Zone Controller. 1.3 Feature dependency overview Table 1-1 and Table 1-2 list all feature dependencies for the configurations described in this document. Please note, optional features are those that can be introduced into the described configurations without requiring significant adjustments of the presented parameters. Some features are also not marked applicable to any of the network cases; these features either do not add any value in the specified cases or they can be applied regardless of network architecture and other features in use. Intro Rel. Basic Feature (BSW) Requires system release FDD-LTE15A Zone Component RL10 RL10 RL10 RL10 Basic software features (no license, no activation needed) RL10 RL30 RL20 RL40 RL70 RL20 RL10 RL30 FDD-LTE15A RL60 RL70 RL60 FDD-LTE15A FDD-FL15A FDD-FL15A 1 10 /157 LTE118 FE/GE electrical interface LTE119 GE optical interface LTE129 Traffic prioritization on Ethernet layer LTE131 Traffic prioritization on IP layer (DiffServ) LTE138 Traffic shaping (UL) LTE144 Transport admission control LTE592 Link supervision with BFD1 LTE593 Security for Ethernet Ports LTE648 SCTP Multihoming (at eNB) LTE775 SCTP Multihoming (at MME) LTE875 Different IP addresses for U/C/M/Splane LTE931 Ethernet Jumbo Frames LTE1244 Source Based Routing in BTS LTE1401 Measurement based TAC LTE1448 VLAN Scan Optimization LTE1460 Local and Remote IP Traffic Capturing LTE1559 SCTP enhancements LTE1996 Flexi Zone Controller Application LTE2156 Site & Agg Solution for Flexi Zone Configuration description in section 3 3 4 FZC FZC FZAP X X X X X X X X X X X X X X X X X X BFD functionality is required as a basis for LTE866 Fast IP Rerouting feature. © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Introduction FDD-FL15A LTE2203 Flexi Zone Controller Application on X X BCN Platform FDD-FL15A LTE2346 FZC Shared Backhaul Support X Phase-I X : mandatory feature for the respective configuration : feature is not applied in the respective configuration : feature is not supported by the respective component/system release Table 1-1: Basic feature dependency matrix © 2015 Nokia Networks DN09224449 Issue 01B 11/157 Intro Rel. Application Feature (ASW) Requires system release FDD-LTE15A Zone Component RL10 RL10 RL20 RL30 RL70 RL20 RL10 RL10 RL10 RL30 RL70 RL70 RL60 RL70 RL70 LTE132 VLAN based traffic differentiation LTE134 Timing over packet LTE140 Ethernet OAM LTE574 IP transport network measurements LTE610 ToP Resilience LTE649 QoS aware Ethernet switching2 LTE664 LTE transport protocol stack LTE689 LTE IPsec Support LTE713 Synchronous Ethernet LTE866 Fast IP rerouting LTE891 Timing over Packet with phase synchronization3 LTE1240 User Layer TCP MSS Clamping LTE1390 IPSec Emergency Bypass LTE1623 Timing over Packet-Frequency Support for FZM LTE1629 Integrated GPS synchronization for Flexi Zone Micro Configuration description in section 3 3 4 FZC FZC FZAP X X X X X X X X X X RL70 LTE1753 Backup IPsec Tunnel FDD-FL15A LTE1771 Dual U-plane IP Addresses RL70 LTE1781 Integrated Multi-GNSS Sync Support 4 RL50FZ LTE1857 Basic Ethernet Switching 5 FDD-FL15A LTE2017 IPSec Support for FZC X X X : mandatory feature for the respective configuration : feature is not applied in the respective configuration : feature is not supported by the respective component/system release Table 1-2: Application feature dependency matrix 2 LTE649 has partial functionality which provides L2 QoS on egress ports The features LTE134/LTE1623 and LTE891 are mutually exclusive. Available on Flexi Zone Micro enhanced BTS and Flexi Zone Pico BTS Hardware 5 Feature LTE1857 is the QoS unaware variant of LTE649 for FZM. 3 4 12 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Introduction 1.4 Network performance Transport network performance has a significant impact on end user experienced network quality and general network stability. Minimum requirements for transport network implementation (also for possible use in SLAs with network service providers) are required in order to avoid excessive investments into network infrastructure while ensuring targeted end-user QoS. It is important to note that there is no “upper threshold” for network performance, i.e. the more stable and efficient a transport network operates, the better the end-user service that can be offered. Contrary to that, there is a “lower threshold”, defining the minimum performance required that needs to be provided in order to keep the radio access network alive and stable. Please also note that network performance requirements for operation of an LTE radio access network significantly differ with respect to the transport plane. While user plane network performance mainly translates into QoS perceived by the end users, the control plane requires some strict characteristics in order to guarantee network stability. These different requirements are summarized in the following sections. In case all traffic for a certain FZAP is using a single transport service only, all the requirements for all the different planes need to be fulfilled. If there are multiple transport services available per FZAP, these services can be tailored more explicitly with respect to the actual traffic being carried. 1.4.1 User plane As mentioned above, network performance required for user plane transport derives from acceptable end-user expectations. 3GPP has defined a set of standardized characteristics that can be used as an indicator for the recommended minimum transport network performance with respect to packet delay and loss rate [3GPP_23203]. With respect to packet delay, no strict performance requirements have to be met in order to ensure network stability. However, transport network should ensure maximum packet delay of 50ms in order to avoid significant impact on end user service quality. In general it is recommended to keep delay limited to at most 20ms to allow provisioning of all defined end user services with superior QoS. Please note that according to 3GPP [3GPP_23203] the value of 20ms is an average value which is derived from combining the delay assumed for users in the local network (10ms) and the one assumed for roaming cases (50ms). In this document only this average value will be further used, real network deployments should of course consider both cases (if applicable). The maximum packet loss to be caused by transport network can be referenced from [3GPP_23203]: it is specifying a limit of 10-6. In addition to packet delay and loss, also packet delay variation plays a role. However, this strongly depends on the end-user application. While real time © 2015 Nokia Networks DN09224449 Issue 01B 13/157 services (e.g. VoIP, audio/video streaming) typically tolerate jitter in the order of +/- 10..20ms by using de-jitter buffers, normal data services are completely tolerant with respect to delay variation. Please note, that in addition to transport network induced jitter, also air interface scheduling contributes to this value. 1.4.2 Control plane In the LTE RAN, control plane related network performance requirements are mainly defined by signalling over X2 interface: for certain handover scenarios, X2AP protocol timeouts have been defined to 500ms. Considering a requestreply message exchange and X2 star (hub and spoke) topology, for the complete handover signalling the one-way delay from FZAP to hub router shared by the involved FZAPs would be experienced four times. Reserving an additional 100ms for processing of the X2AP messages in the FZAPs, the single one-way delay shall not exceed 100ms in transport network between eNB and edge router. SCTP protocol is rather insensitive to delay variation, so no special requirements are established in this respect. Packet loss should be kept below 10-3 in order to avoid massive SCTP retransmissions. Nevertheless, control plane would also operate normally based on a higher loss ratio – with some performance penalty though. 1.4.3 Management plane Much of the management plane traffic is carried via TCP transport protocol which is truly robust against packet delay and delay variation. However, in order to ensure reasonable transport efficiency and O&M responsiveness, a maximum delay of 1000ms is defined. Higher delays are supported, but there may be noticeable impact. TCP is also comparably tolerant to packet loss (similar to SCTP) and thus the same recommendation for a maximum packet loss of 10-3 is stated. The FZAP requires the periodic renewal of its IP address lease from the DHCP server that resides on the FZC. Loss of the renewal requests and the subsequent expiration of the lease will result in the FZAP resetting and performing an autoconnection. In order to reduce the risk of losing the DHCP REQUEST messages associated with renewal, it recommended that a maximum packet loss of 10-6 be provided for this traffic 1.4.4 Synchronization plane In case LTE RAN synchronization is based on IEEE1588-2008, strict network performance requirements have to be met in order to guarantee stable synchronization. NOKIA recommends the following minimum characteristics: - Packet delay < 100 ms - Packet delay variation (jitter) < +/- 5 ms 14 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Introduction - Packet loss rate < 2% Please note, maximum packet delay variation requirement is based on ±50ppb FDD air interface frequency synchronization via Timing-over-Packet (IEEE 1588-2008). As various aspects may impact the frequency synchronization, it is recommended to consider the following additional engineering rules: - IEEE1588-2008 packets should have the highest priority (or at least the same priority as the real-time traffic) and receive expedited forwarding QoS, - High-priority traffic share should not exceed 60% of the link capacity, - The network between IEEE1588-2008 master and slave should not comprise more than 20 hops (packet switches); if the network is based on Microwave Radios, the number of total hops should be less than 10, - Maximum 6 packet delay discontinuities (spontaneous, significant changes of the packet delay) should occur per day. Accurate phase synchronization is also required for certain services in FDD mode (e.g. MBMS, location based services, eICIC). In addition to GNSS phase synchronization in RL15(RL70), the alternative synchronization solution LTE891 - Timing over Packet with phase synchronization (ToP-P) is introduced. Basically it is not wrong to apply for ToP-P the same ToP-F network performance requirements. However ToP-P network performance requirements are different to ToP-F and they are more. For deployment of a ToP-P capable network there is still some more technical know-how necessary. Accordingly for more details see the synchronization section 2.8. 1.4.5 Summary Table 1-3 summarizes the explained performance requirements for the end-toend LTE network. Please note that the delay values for control and synchronization planes are the only strict requirements – other figures simply translate into perceived end-user service quality and will not affect overall system stability. Looking at user plane transmission it is not expected to see significant differences from end-user perspective when experiencing one-way S1 delay beyond 20ms. End-to-end performance requirements Plane Delay Packet Delay Variation User (real time) 50 ms (6) User (non real time) 300 ms (6) Control 100 ms 6 Depends on end-user service © 2015 Nokia Networks DN09224449 Issue 01B +/- 10 ms n/a n/a Packet Loss 10-3 (6) 10-6 10-3 15/157 Management Synchronization 1000 ms 100 ms n/a 10-6 ToP-Frequency = +/- 5ms (7) 2x10-2 ToP-Phase = +/- 2.5ms Table 1-3: Summarized LTE end-to-end performance requirements Based on the strict requirements presented in Table 1-3, the following target values in Table 1-4 are recommended by NOKIA and can be used as network service performance target values (e.g. in SLA) in case different transport services are deployed for the different planes (e.g. multiple VLANs/EVCs per eNB). Network performance recommendations Plane Delay Packet Delay Variation +/10 ms User (real time) 20 ms User (non real time) 20 ms n/a Control 100 ms n/a Management 100 ms n/a Synchronization 20 ms ToP-Frequency = +/- 5ms ToP-Phase = +/- 2.5ms Packet Loss 10-4 10-7 10-6 10-6 10-3 Table 1-4: NSN LTE transport network performance recommendations Assuming a common transport solution for all the planes, the following minimum transport network characteristics are recommended by NOKIA: - Packet delay < 20 ms - Packet delay variation (jitter) < +/- 5 ms when using ToP-Frequency - Packet delay variation (jitter) < +/- 5 ms when using ToP-Frequency - Packet loss rate < 10-7 These recommendations are based on [3GPP_23203], the minimum requirements given in Table 1-3, and a reasonable trade-off between network implementation cost and resulting end user service experience. 1.5 Document Icons This document makes use of various icons to identify information that requires additional attention by the reader. DANGER! Hazard description Potential loss Avoid loss 7 16 /157 In case Timing-over-Packet is used as synchronization solution © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Introduction NOTICE: Hazard description Second paragraph General note or hint Second paragraph Tip Second paragraph 1.6 Network Element Icons This document makes use of various icons depicting network elements. This section explains their generic characteristics in order to avoid ambiguities. Figure 1-1 shows the icons being used in this document. Please note that the icons described below do not depict certain kinds of network elements; instead they will be used to underline the specific use of those. So while some network vendors for example make a differentiation between routers and switches (e.g. Juniper MX vs. EX), this document just focuses on the differences in terms of configuration: - Routers are network devices configured to terminate layer 2 broadcast domains. Forwarding of packets is purely based on layer 3 (IP) routing. - Layer 2 switches are network devices configured to layer 3 (IP) agnostic Ethernet frame forwarding based on VLAN and Ethernet MAC address information. © 2015 Nokia Networks DN09224449 Issue 01B 17/157 Figure 1-1: Network element icons used in this document 18 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport 2 General aspects General aspects This chapter describes features and configuration aspects that are shared by multiple network configurations. 2.1 Hardware 2.1.1 FZAP The recommended network configuration consists of a Flexi Zone Controller BTS and a cluster of Flexi Zone Access Points. The available Flexi Zone Access Points (FZAP) are comprised of the following hardware models: 1. 2. 3. 4. Flexi Zone Micro Access Point Flexi Zone Micro Access Point with integrated Wi-Fi AP Flexi Zone Pico Access Point Flexi Zone Pico Access Point with integrated Wi-Fi AP All configurations described refer to certain Flexi Zone Access Point models. Please note that the hardware models have some differences in their feature sets. These are mentioned throughout this document. The FZC supports both FDD and TDD FZAPs, however, a single FZC does not support a mixed set of FDD and TDD FZAPs. This document addresses an FZC supporting FDD FZAPs. All recommended network configurations are comprised of two Flexi Zone Access Point connection setups. The first connection setup is for a standalone Flexi Zone Access Point (e.g. FZAP #1). The second connection setup provides a group of chained Flexi Zone Access Points (e.g. FZAP #2 – FZAP # ... ) that share the same backhaul capacity of the connection setup. Flexi Zone Micro Access Point and Flexi Zone Pico Access Point are functionally identical equipment. The difference is in the form factor and deployment position. While the Flexi Micro Access Point can be used Indoor or Outdoor, the Flexi Pico Access Point is an indoor only unit. For the configurations in this guide Flexi Zone Micro Access Point and Flexi Zone Pico Access Point are interchangeable and the configurations will reference the Flexi Zone Access Point (FZAP). 2.1.1.1 Flexi Zone Platform (FxP) The FZC supports the following field configurations: Single Control Card (FCPU) – Slot 1 Single Bearer Card (FZBU) – Slot 2 © 2015 Nokia Networks DN09224449 Issue 01B 19/157 The FZC has two logical interfaces, as shown in Figure 2-1. One is the northbound interface which connects the FZC to the EPC. This interface handles the interactions with the MME, Serving GW, and PDN GW like any other eNodeB. The other is the southbound interface (also referred to as the Z1 backhaul) which handles the interaction between the individual FZAPs and the FZC. Figure 2-1: FZC North/Southbound Interfaces The FZC supports three different configurations for the northbound Ethernet Control and Management plane traffic: 10GE for U-plane and 1GE for C/M-plane (default) 10GE for U-plane, 1GE for C-plane, and 1GE for M-plane 10GE for U/C/M-plane The FZC supports two different configurations for the southbound Ethernet control interface. There is no support for using the two configurations together: Default: Single 10G SB Ethernet interface for combined Control/Management/U-plane traffic. Multiple 1G SB Ethernet interfaces for combined Control/Management/U-plane traffic. Ethernet Interfaces: When the FZC is first powered up for field commissioning, the FZC is configured with the standard configurations shown in Table 2-1. Card 20 /157 Interface SFP Interface Label © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects FCPU 1G SFP 13 nbmplane, nbcplane FZBU 10G SFP+12 sbzplane FZBU 10G SFP+11 nbuplane Table 2-1: FZC supported Ethernet interfaces for transport use When this configuration is deployed, no changes are needed to the Ethernet interface configuration. Three other configurations are supported that require reconfiguration during field commissioning. • When M-plane and C-plane interfaces are separated on the FCPU. Card Interface SFP Interface Label FCPU 1G SFP 13 nbcplane FCPU 1G SFP 14 nbmplane Table 2-2: FZC Split M-Plane and C-Plane Ethernet Interface • When M-plane and C-plane interfaces are merged onto the same link as the U-plane traffic. Card Interface FZBU, FCPU SFP 10G Interface Label SFP+ 11 nbuplane, nbcplane, nbmplane Table 2-3: FZC Merged M-Plane and C-Plane Ethernet Interface • When the southbound Z1 sbzplane is comprised of a set of 1G interfaces instead of a single 10G interface. The number of available 1G interfaces is dependent on how many 1G interfaces are in use for the Northbound interfaces. There are 10 1G ports available: if SFP 14 is used for northbound interfacing, only SFP 15 to SFP 22 will be available for southbound. Card Interface FZBU SFP 1G SFP 14/158 - SFP 22 Interface Label sbzplane 8 The number of SFPs available is dependent on the number of SFPs used for the northbound interfaces. © 2015 Nokia Networks DN09224449 Issue 01B 21/157 Table 2-4: FZC Multiple Z1 Ethernet Interfaces 2.1.1.2 Flexi Zone Micro / Flexi Zone Pico Access Point The Flexi Zone Micro and Flexi Zone Pico Access Point are functionally the same; however, there are differences in the packaging to support different deployment models. The recommended network configurations using the Flexi Zone Micro Access Point can be implemented on the Flexi Zone Pico Access Point. Ethernet Interfaces: Flexi Zone Micro and Flexi Zone Pico Access Points provide multiple Ethernet configurations based on the model. All configurations provide two Ethernet ports and some models provide three Ethernet ports. On the two port models one port is the main backhaul port while the other can be used as either a Local Management Port or for connecting other chained devices such as other Access Points or WiFi Access points. On the three port models any of the ports can be used as the main backhaul port. Either of the two remaining ports can be used for Local management, daisy chaining, or packet forwarding for external equipment. Each port can only play one of those roles at a time. Table 2-5 shows the Ethernet configurations available on the different Flexi Zone Micro and Flexi Zone Micro Access Point. The enhanced three port FZAP contains an integrated Wi-Fi AP and the third copper port is a PoE port. # Flexi Zone Access Point Models EIF1 Transport module interfaces EIF2 EIF3 EIF1 (SFP) (RJ-45) (RJ-45) (SFP) EIF3 (RJ-45) 1 FZAP 2 port FZAP 3 port FZAP 3 port enhanced Indoor PICO 2 3 4 Table 2-5: Flexi Micro/Pico BTS supported Ethernet interfaces for transport use 2.1.1.3 Integrated Wi-Fi Access Point Some Flexi Zone Micro and Flexi Zone Pico Access Point variants include an integrated Wi-Fi AP. The integrated Wi-Fi AP is connected to the internal switch and all Wi-Fi traffic is switched to the primary backhaul interface. The Wi-Fi and LTE APs share the common physical backhaul, but connect logically to their own core networks independently of each other. The Flexi Zone Controller does not support the transiting of Wi-Fi traffic. The Wi-Fi traffic is split out at the aggregation switch terminating the Z1 backhaul. 22 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects 2.1.2 Synchronization For synchronization an IEEE1588 (ToP) master clock and Synchronous Ethernet primary reference clock are deployed. Microsemi (formerly Symmetricom) TP5000 IEEE1588-2008 Grand Master Clock is used as master clock or equivalent. With respect to Synchronous Ethernet just any standards compliant reference clock is suitable. In addition to SyncE and ToP, Flexi Zone Micro and Flexi Zone Pico Access Points also provide GNSS (GPS or GLONASS) synchronization. The use of GNSS for indoor deployments may not be possible depending upon the reduced signal strength experienced at the location. Flexi Zone Controller does not currently support IEEE1588-2008 boundary or master clock functionality. As a result the switching/routing devices in the Access Point network need to provide this functionality. Unicast and Multicast modes of operation are supported by the Flexi Zone Access Point. 2.1.3 Operation & maintenance The FZC supports a BTS-OM interface to iOMS for LTE application management and a NE3S interface for FZC platform/transport management. For local management, the SCLI is used for platform & networking configuration, while BTS Site manager for the FZC supports the LTE eNB data management similar to the eNB. The FZC provides the management of the FZAPs for different FCAPS procedures required. FZAPs support BTS Site Manager as well as WebGUI for limited local management capabilities. WebGUI is introduced to support extensions needed for Indoor Enterprise deployments. 2.2 Traffic dimensioning All recommended network configurations presented in this document share the same traffic type to PHB class mapping (according to Table 2-14) and the same input traffic model is applied (according to Table 2-6). The dimensioning example in this document focuses on the network architecture using IPSec on the southbound Z1 interface and no IPSec on the northbound interface. To make it applicable also for architectures that include IPsec on the northbound interface, the additional IPsec header has to be taken into account as described in chapter 2.2.4. Bandwidth demand estimation for S1/X2 interface is based on dimensioning rules presented in LTE Access Dimensioning document [LTE_Access_Dim]. 2.2.1 Dimensioning assumptions In the following, an FZC/FZAP transport dimensioning example is presented for a FZAP 1-cell 20 MHz@ 2x2 MIMO configuration and an FZC supporting 100 APs, assuming the traffic scenario as presented in Table 2-7. Note that © 2015 Nokia Networks DN09224449 Issue 01B 23/157 input values are examples only and cannot be considered as a reference. The dimensioning should be performed for each network individually. When the FZAP network contains fewer numbers than 100 FZAPs, the dimensioning should be scaled accordingly. Value Parameter FZC 100 7525 Number of supported cells Number of RRC connected users per eNB9 Real time services (RT) Conversational Voice Non-real time services (NRT) User plane traffic (for real time services) per user User plane traffic (for non real time services) per user Number of inter-FZAP handovers per call Control plane traffic per user 10 O&M traffic per node (management plane) ToP-Frequency traffic per FZAP (S-Plane)11 ToP-Phase traffic per FZAP (S-Plane)12 FZAP 1 175 Internet Static Application 3.6 kbps / 3.6 kbps (DL/UL) 38.4 kbps / 9.6 kbps (DL/UL) 4.5 0.13 kbps / 0.03 kbps (DL/UL) 10 Mbps average 2 Mbps average 15 kbps/ 0 kbps 0 kbps in DL/UL (DL/UL) 150 kbps/ 50 0 kbps in DL/UL kbps in (DL/UL) Table 2-6: General eNB dimensioning assumptions Table 2-7 summarizes user traffic demand per eNB (user and control plane) for the assumed traffic scenario. 9 The maximum number of RRC connected users supported by the FZC is 10,000. The value shown for the FZC RRC connected users is based on the assumption that the aggregate traffic of the 100 FZAPs is equivalent to the typical loading of 43 FZAPs. 24 /157 10 Based on detailed control plane analysis in [LTE_Access_Dim] 11 FZC does not support S-plane traffic in RL15A. 12 FZC does not support S-plane traffic in RL15A. © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Traffic demand per eNB for the assumed traffic scenario S1 user plane – real time (RT) S1 user plane – non-real time (NRT) X2 user plane – real time (RT) X2 user plane – non-real time (NRT) S1 control plane X2 control plane Management plane Synchronization plane with ToPFrequency Synchronization plane with ToP-Phase Z1 C-plane Total (using ToP-F) General aspects FZC FZAP DL UL DL UL [Mbps] [Mbps] [Mbps] [Mbps] 2.00 User plane real time traffic includes Conversational Voice User plane non-real time traffic includes e.g. web and FTP User plane real time traffic on X2 due to inter-eNB handovers User plane non-real time traffic on X2 due to inter-eNB handovers Signaling (S1AP) Control plane traffic on X2 due to inter-eNB handovers (X2AP) eNB O&M 27.09 27.09 0.63 0.63 289.82 72.24 6.74 1.68 0.0086 0.0086 0.0002 0.086 0.43 0.002 0.01 0.86 0.43 0 0 0.43 0.43 0 0 10.00 10.00 2.00 Comment 0.0002 0 0 0.015 0.10 IEEE1588-2008 0 0 0.15 0.05 IEEE1588-2008 2.58 330.9 1.72 0.06 112.3 9.5 0.04 4.5 Table 2-7: Dimensioning assumptions: eNB traffic load for the assumed traffic scenario 2.2.2 Dimensioning results In network capacity dimensioning some spare capacity should be considered in addition to user payload and protocol header overheads in order to ensure QoS targets. A typical link utilization applied is 80% of the physical link capacity. Table 2-8 summarizes calculated bandwidth for an individual FZAP and for the north bound FZC interface based on the assumed traffic profile. It is recommended that the south bound Z1 interface always be provisioned with a 10 Gbps link. These calculations are performed to assure proper QoS targets (packet delay, loss ratio) for services defined in traffic profile. Based on that necessary throughput could be calculated using Multidimensional ErlangB and extended M/G/R-PS models formulas. Calculations take into account transport protocol overheads defined for each service respectively. Also the impact of other factors like: TCP slow-start, IP fragmentation, queue differentiation and IPSec encapsulation is considered during dimensioning if necessary. Detailed calculations results for this example were finally averaged and presented in Table 2-8 for selected plane and each interface type. © 2015 Nokia Networks DN09224449 Issue 01B 25/157 For detailed dimensioning methodology please refer to LTE Access Dimensioning Guideline [LTE_Access_Dim]. As explained above, user traffic can be split into real time and non-real time services. With respect to traffic engineering this leads to different requirements regarding transport capacity: while for real time services it is recommended to provision the expected bandwidth in a guaranteed way, it is common to use non-guaranteed transport capacity for non-real time U-plane services. In the context of Metro Ethernet Networks these different capacities are typically referred to as CIR (committed capacity) and EIR (non-guaranteed capacity). eNB transport bandwidth per logical interface S1 user plane – real time S1 user plane – non-real time X2 user plane – real time X2 user plane – non-real time S1 control plane X2 control plane Management plane Synchronization plane with ToPFrequency Synchronization plane with ToPFrequency Total (using ToP-F) Guaranteed bit rate traffic Best effort traffic Total Required capacity FZC FZAP Downlink Uplink Downlink Uplink [Mbps] [Mbps] [Mbps] [Mbps] 67.94 67.94 1.58 1.58 605.44 234.35 14.08 5.45 0 0 0 0 0.215 0.129 0.005 0.003 0.86 0.43 0 0 0.215 0.129 0 0 16.7 16.7 3.34 3.34 0 0 0.03 0.0 0 0 0.30 0.10 691.4 148.14 605.66 1445.2 319.7 147.62 234.48 701.8 19.4 3.445 14.085 36.9 10.5 3.433 5.453 19.4 Table 2-8: Calculated eNB transport capacity based on the assumed user traffic profile (incl. Ethernet & IP overhead) Ideally, calculated bandwidth demand, FZAP egress traffic shaping, and Z1 transport network capacity are fully aligned. In general it is recommended to configure FZAP traffic shaping parameters (i.e. shaping rates per FZAP: SIR) according to available transport capacity for the FZAP. The impact of IPSec overhead is not included in these calculations (see chapter 2.2.4). 2.2.3 Ethernet OAM dimensioning Ethernet OAM is not supported on the FZC and FZAP. 26 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects 2.2.4 IPsec impact on dimensioning Some network configurations are designed to use IPsec to protect selected traffic planes while transmitting data through the access and aggregation part of the network. The FZC northbound interface can be IPSec protected based upon operator configuration. The user, control, and management planes can be independently configured for IPSec. The southbound interface always uses IPSec for the control, and management traffic planes. The use of IPSec for the user plane is configurable by the operator. When LTE2346 FZC Shared Backhaul Support Phase-I is enabled, IPSec of the user traffic is required. The synchronization plane is not IPSec protected for any case. This protection is realized using selected encryption and authentication algorithms that are applied on IP datagrams transmitted through the IPsec tunnel. These datagrams include IPsec related information in an additional packet header, resulting in additional overhead bytes that have to be taken into account. For some encryption algorithms (e.g. AES) explicit padding needs to be added to the payload data so that a certain alignment is achieved (for AES 0..15 bytes padding apply). As this padding cannot be exactly estimated (and also does not significantly impact the results), for the dimensioning calculations in this document an average padding of 8 bytes is assumed. ESP protocol header AES initialization vector ESP trailer13 Authentication (HMAC-SHA-1-96) IPsec tunnel mode IP header Total 8 bytes 16 bytes 10 bytes 12 bytes 20 bytes 66 bytes Table 2-9: IPSec overhead – example calculation The IPsec impact on network dimensioning is strongly correlated with average packet size used for selected services. For small messages, the additional 66 bytes introduced by IPsec header may lead to more than 100% overhead. For bigger messages this impact will be much lower. For messages with size very close to MTU value, IPsec may result in fragmentation – thus it can also influence efficiency substantially. To properly include IPsec overhead in dimensioning, each plane has to be analyzed separately, taking into account message size for all services that are transmitted within that plane. It is estimated that user plane traffic overhead will increase from 14% (without IPsec used) to 26.5% with IPsec used. This estimation is calculated assuming traffic consisting of 50% of small-size packet (40 byte UE payload), 25% of medium-size packets (576 byte UE payload), and 25% of big-size packets (1500 byte UE payload,) without any packet fragmentation. 13 Please note that this value includes assumed average of 8 padding bytes. © 2015 Nokia Networks DN09224449 Issue 01B 27/157 For control plane traffic the IPsec impact is much higher (relatively) and it may increase header overheads from 52% (without IPsec) to 99% of the payload (with IPsec). This estimation is calculated assuming traffic consisting of 90% of small-size packet and 10% of medium-size packets. This increase of header overhead has to be taken into consideration when performing calculations as described in chapter 2.2.2. 2.2.5 WiFi dimensioning In the following four tables, a Wi-Fi transport dimensioning example is presented for a Flexi Zone Access Point with integrated Wi-Fi, assuming traffic models as presented in Table 2-10 and Table 2-12, for both cases of 802.11ac and 802.11n. Note that input values are examples only and cannot be considered as a reference. The dimensioning should be performed for each network individually. The Wi-Fi traffic must be removed before it reaches the FZC. As a result Wi-Fi traffic has not impact on FZC dimensioning. Traffic Number of users Frame Size (Bytes) Frame / sec Data Rate / user (kbps) Data Rate (all users, kbps) Voice G723 Coder 20 96 28 21.5 430 Video MPEG-2 Coder DL 5 1200 1500 14400 72000 H.263 Video DL 5 500 3000 12000 60000 MPEG-2 Coder UL 2 1200 800 7680 15360 H.263 Video UL 2 500 2400 9600 19200 FTP Get 3 600 3500 16800 50400 FTP Put 1 600 3000 14400 14400 HTTP Get 4 500 3500 14000 56000 HTTP Get 6 1400 1500 16800 100800 HTTP Put 1 500 4000 16000 16000 HTTP Put 2 1400 1000 11200 22400 FTP HTTP Table 2-10: 802.11ac Wi-Fi Traffic Model Voice Video HTTP FTP Total Traffic Total Traffic (UL+DL) (Mbps) DL Traffic (Mbps) 0.43 132 156.8 50.4 339.63 UL Traffic (Mbps) 0.43 34.56 38.4 14.4 87.79 DL Traffic (pps) 560 22500 23000 10500 56000 UL Traffic (pps) 560 6400 6000 3000 15400 427.42 71400 Table 2-11: Wi-Fi transport bandwidth consumption with 802.11ac Traffic Model 28 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Traffic General aspects Number of users Frame Size (bytes) Frame / sec Data Rate / user (kbps) Data Rate (all users, kbps) Voice G723 Coder 20 96 28 21.5 430 Video MPEG-2 Coder DL 2 1200 1500 14400 28800 H.263 Video DL 2 500 3500 14000 28000 MPEG-2 Coder UL 2 1200 800 7680 15360 H.263 Video UL 2 500 1600 6400 12800 FTP Get 8 600 1300 6240 49920 FTP Put 3 600 900 4320 12960 HTTP Get 3 500 3000 12000 36000 HTTP Get 6 1400 1200 13440 80640 HTTP Put 2 500 1900 7600 15200 HTTP Put 2 1400 900 10080 20160 FTP HTTP Table 2-12: 802.11n Wi-Fi Traffic Model Voice Video HTTP FTP Total Traffic DL Traffic (Mbps) 0.43 56.8 116.64 49.92 223.79 UL Traffic (Mbps) 0.43 28.16 35.36 12.96 76.91 DL Traffic (pps) 560 10000 16200 10400 36600 UL Traffic (pps) 560 4800 5600 2700 13100 Total Traffic (UL+DL) (Mbps) 300.7 49700 Table 2-13: Wi-Fi transport bandwidth consumption with 802.11n Traffic Model 2.3 Quality of Service model 2.3.1 FZAP Traffic classification Traffic classification and marking is provided by features LTE131 Traffic prioritization on IP layer (DiffServ) and LTE129 Traffic prioritization on Ethernet layer. While the first one provides mappings from LTE traffic type and QCI to DSCP, the second feature adds the mapping from DSCP to PCP. Table 2-14 describes the recommended mapping configuration for the FZAP. © 2015 Nokia Networks DN09224449 Issue 01B 29/157 Plane QCI/Protocol DSCP PCP QCI 114 46 EF 6 QCI 215 26 AF32 4 15 46 EF 6 15 28 AF32 4 QCI 3 QCI 4 QCI 5 34 AF41 5 QCI 6 18 AF21 3 QCI 7 20 AF22 3 QCI 8 10 AF11 1 QCI 9 0 BE 0 GTP-u path management19 46 EF 6 46 EF 6 16 User plane PHB 17 Control plane Z1 C-plane Synchronizati on IEEE1588 (ToP)18 46 EF 6 Management plane CM, FM, PM 34 AF41 5 20 AF22 3 ICMP, MLD, 10 AF11 1 BFD20 34 AF41 4 34 AF41 5 36 AF42 5 6 BE 0 Network control plane Trace traffic 19 21 IKE Wi-Fi Control plane 22 User plane Table 2-14: Recommended FZAP QoS mappings 14 Supported only in combination with feature LTE010 EPS bearers for conversational voice. 15 Supported only in combination with features LTE496 Support of QCI 2, 3 and 4, LTE497 Smart Admission Control, and LTE534 ARP-based Admission Control for ERABs. 16 Supported only in combination with feature LTE009 Service differentiation. 17 The Z1 C-plane traffic is based upon FZC terminated S1 and X2 traffic 18 In some environments allocation of even higher priority (e.g. DSCP=56 and PCP=7) to synchronization traffic (IEEE1588) has proven to result in more stable eNB synchronization. 30 /157 19 DSCP mappings for GTP path management and trace traffic cannot be modified. 20 DSCP for BFD packets can be configured per session. 21 Only relevant in case of feature LTE689 LTE IPsec Support. 22 WiFi Control is low bandwidth traffic so is low impact to LTE AF4 Traffic. © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects The QoS mappings in Table 2-14 are configurable, except as noted. Although a default mapping covers the most typical mappings it is recommended to double-check the mappings e.g. in case of MEN EVC profile applications. The operator is allowed to configure the WiFi DSCP values quite differently. If he sees that the WiFi QoS needs are similar to the LTE user-plane QoS he can also use DSCP = 0(dec) for WiFi user traffic. However in this case the WiFi traffic is concurrent to LTE Non-GBR traffic and may have impact on LTE end user experience. DSCP values 2(dec) and 4(dec) are also available in the same IP Precedence class if additional DSCP values are required to provide service differentiation of the WiFi user plane traffic for services such as Voice over WiFi. 2.3.2 FZC Traffic Classification Traffic classification and marking is provided as part of feature LTE2203 Flexi Zone Controller Application on BCN Platform. This provides mappings from LTE traffic type and QCI to DSCP. Table 2-14 describes the recommended mapping configuration for the FZC Z1 backhaul and northbound interfaces, which should be aligned with the FZAP DSCP markings. © 2015 Nokia Networks DN09224449 Issue 01B 31/157 Plane QCI/Protocol User plane DSCP Management plane Network control plane PCP QCI 1 46 EF 6 QCI 2 26 AF32 4 QCI 3 46 EF 6 QCI 4 28 AF32 4 QCI 5 34 AF41 5 QCI 6 18 AF21 3 QCI 7 20 AF22 3 QCI 8 10 AF11 1 QCI 9 0 BE 0 GTP-u path management19 46 EF 6 Z1 C-plane Northbound C-plane 46 EF 6 CM, FM, PM 34 AF41 5 20 AF22 3 ICMP, MLD 10 AF11 1 BFD25 34 AF41 4 IKE 34 AF41 5 23 Control plane PHB Trace traffic 24 Table 2-15: Recommended FZC QoS mappings The QoS mappings in Table 2-14 are configurable, except as noted. Although a default mapping covers the most typical mappings it is recommended to double-check the mappings e.g. in case of MEN EVC profile applications. 2.3.3 FZAP uplink scheduling Nokia’s FZAP offers a very flexible IP uplink scheduling configuration based on a two level packet scheduler as shown in Figure 2-2. For the FZAP there is only one IP address and only one VLAN. As a result only one DSCP scheduler is required. The first level scheduler (“DSCP Scheduler”) provides a combined strict priority / WFQ scheduler: while the first queue is always served with strict priority, the remaining 5 queues are served by a WFQ scheme. This WFQ scheduler can be parameterized by assigning dedicated weights per queue as part of the QoS configuration. 32 /157 23 The Z1 C-plane traffic is based upon FZC terminated S1 and X2 traffic 24 DSCP mappings for GTP path management and trace traffic cannot be modified. 25 DSCP for BFD packets can be configured per session. © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Uplink traffic DSCP marking General aspects Schedulers DSCP scheduler #1 SP FIFO queue #1 U‐plane S1+X2; M‐plane; C‐plane; S‐plane WFQ aggregator Scheduler/shaper FIFO queue #2 FIFO queue #3 FIFO queue #4 FIFO queue #5 FIFO queue #6 SIR1 SBS 1 rate1 WFQ WFQ SIRtotal SBS total Service‐OAM frame processor 1) Figure 2-2: eNB packet scheduler architecture In an All-in-One (AiO) FZM the second level scheduler aggregates this traffic with remaining other planes, e.g. synchronization, management and/or control plane traffic (in case traffic differentiation is configured for all these planes). This scheduler operates in WFQ mode and weights must be configured per network interface/VLAN. Since there is only one network interface configured for the FZAP, the second level scheduler does not provide any impact on the handling of the uplink packets. Total egress (uplink) traffic can be shaped according to a total SIR / SBS parameter unless shaping per VLAN is configured (controlled by SIRx / SBSx parameters). These two shaping functionalities are mutually exclusive. In case feature LTE140 Ethernet OAM is deployed; the FZAP scheduler will support another queue for Service OAM traffic as shown in Figure 2-2. This queue is also assigned a configurable weight at the WFQ aggregate scheduler. © 2015 Nokia Networks DN09224449 Issue 01B 33/157 The FZAP implements a token bucket L2 packet rate shaper which allows provisioning of an egress bandwidth limit and a burst size. The burst size defines how much the egress bandwidth limit can exceed for short periods of time. In general burst sizes are defined by the network implementation (SLA) and should be configured in the FZAP accordingly. Values given in this guide are to be understood as configuration examples. Table 2-16 describes the recommended scheduler weight configuration. PHB/Queue EF AF4 AF3 AF2 AF1 BE Services26 QCI-1, QCI-3, IEEE1588 (ToP), control plane, GTP-u path management QCI-5 (IMS signaling), BFD, IKE, O&M, BFD, Wi-Fi Control QCI-2, QCI-4 (such traffic is not used in the described configurations though) QCI-7 (voice, live streaming, gaming), QCI-6 (buffered streaming, TCP-based, e.g. www, email, etc.) QCI-8 (buffered streaming, TCP-based, e.g. www, email, etc.), ICMP QCI-9 (buffered streaming, TCP-based, e.g. www, email, etc.), Wi-Fi User Scheduler Weight n/a 10000 1000 100 10 1 Table 2-16: Scheduler weights configured to FZAPs These scheduler weights apply to FZAP QoS parameters as configured in perHopBehaviourWeightList parameter in IPNO-QOS object and wfqSchedOamWeight parameter in IPNO object27. The scheduler weight for AF4 queue has been selected comparably high based on the assumption that in practice only a small fraction of traffic will be delivered via this queue. By configuring a significantly higher weight, rather strong prioritization of signaling traffic is achieved without risking starvation of traffic in other queues. Contrary to that, weight for best effort queue is set to the lowest possible value, ensuring that this queue is really served in a best effort manner only. Scheduler weight for AF3 and AF2 classes have been configured to intermediate values in order to balance prioritization need (requires higher weight) and minimization of BE queue starvation risk (requires lower weight). Weight for Ethernet OAM traffic is specified rather small due to low traffic amount caused by Ethernet OAM (applies to configurations with Ethernet OAM only). 34 /157 26 Usage examples according to 3GPP TS23.203. 27 On Flexi Zone Micro eNB, QoS is only applied on traffic originated by the eNB. © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects 2.3.4 FZC Packet Scheduling The FZC packet scheduler has a maximum of six packet scheduling queues. They are configured in a similar to the FZAP packet scheduling queues. The queue structure for the northbound and southbound backhauls is shown in Table 2-17. Queue overflows are handled by tail drop. PHB/Queue Scheduler Weight EF / 5 AF4 / 4 AF3 / 3 AF2 / 2 AF1 / 1 BE / 0 n/a 8 4 1 1 1 Queue Length (bytes) 20,480 409,600 409,600 409,600 409,600 819,200 Table 2-17: Scheduler weights configured to FZC 2.3.5 MME / SAE-GW downlink scheduling Packet scheduling in Core Network elements is implementation specific for the respective network element and thus out of scope of this document. 2.3.6 Measurement-Based Transport Admission Control The transport admission control is performed by the feature LTE1401 Measurement-Based TAC (MB-TAC). This feature is continuously measuring the actual traffic in DL and UL. New GBR connections are accepted or rejected depending on the transport load situation. Transport admission control is supported on the FZAP, but is not supported on the FZC. With Measurement-Based Transport Admission Control, the FZAP will accept only a limited amount of guaranteed bitrate connections, thus preventing exceeding the available backhaul capacity for GBR bearers (which would lead to degradation of QoS for these bearers). The functionality "Transport Admission Control" is done for Guaranteed Bit Rate ("GBR") traffic only. More precisely, the MB-TAC is called whenever a connection with QCI=1,2,3,4 request is to be established. However, the measurement of the traffic in the LTE system supervised by MB-TAC-includes traffic with QCI values 1,2,3,4, plus C-plane traffic, plus traffic with QCI=5, plus synchronization traffic. MB-TAC measures the outgoing and incoming traffic of these types directly at the network interface(s). The decision about acceptance of a new GBR connection is done separately for uplink and downlink. Only if both decisions are "accept", MB-TAC accepts set-up of this new connection. The three threshold level parameters for transport admission control are: tacLimitGbrNormal ≤ tacLimitGbrHandover ≤ tacLimitGbrEmergency © 2015 Nokia Networks DN09224449 Issue 01B 35/157 The thresholds for handover and emergency cases are introduced in order to allow reservation of a certain bandwidth share for these exceptional cases, thus preventing rejection of such connections by admission control. The parameter upperLayerShaping can be used to exclude Ethernet & VLAN overheads in the admission control calculations, for example when transport capacity limits are specified or agreed (SLA) at IP layer (excluding Ethernet layer overheads). 2.4 Ethernet switching in FZAP 2.4.1 Egress Packet Scheduling Flexi Zone Access Point integrated Ethernet switching functionality deployed in the described configurations is based on a portion of the feature LTE649 QoS Aware switching. This feature enables traffic aggregation in the access network without the need for additional investments in Ethernet switching equipment, as well as providing L2 QoS processing. The VLAN aware switching and ingress policing portions of LTE649 will be supported in a future release. The packet scheduling queues and rate shaping capability being provided for each egress port is shown in Figure 2-3. The L2 QoS provides the capability to manage the aggregation of FZAP uplink traffic, daisy chain traffic, and Wi-Fi traffic. The L2 QoS traffic classification can be based upon the DSCP or PCP value of the packets. The structure and behavior of the packet scheduling queues is similar to that of the DSCP scheduler discussed in Section 2.3.2. Figure 2-3: Layer 2 Egress Packet Scheduling The Strict Priority (SP) queue will be serviced first whenever there are packets in the queue. The five Weighted Fair Queues (WFQ) will be serviced with the 36 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects remaining bandwidth after the SP queue is serviced. The bandwidth is divided between the WFQs based upon the weight assigned to each queue. Bandwidth that is not used by a WFQ is available for other WFQs to use based upon their relative weights. Table 2-16 describes the recommended L2 scheduler weight configuration. Services28 PHB/Queue EF AF4 AF3 AF2 AF1 BE QCI-1, QCI-3, IEEE1588 (ToP), control plane, GTP-u path management QCI-5 (IMS signaling), BFD, IKE, O&M, BFD, Wi-Fi Control QCI-2, QCI-4 (such traffic is not used in the described configurations though) QCI-7 (voice, live streaming, gaming), QCI-6 (buffered streaming, TCP-based, e.g. www, email, etc.) QCI-8 (buffered streaming, TCP-based, e.g. www, email, etc.), ICMP QCI-9 (buffered streaming, TCP-based, e.g. www, email, etc.), Wi-Fi User Scheduler Weight n/a 8 4 1 1 1 Table 2-18: L2 Scheduler weights configured to FZAPs These scheduler weights apply to FZAP L2 switching parameters as configured in the l2PriorityQueueWeightx parameters in the APL2SW object. 2.4.2 Egress Rate Shaping The FZAP implements a token bucket L2 packet rate shaper which allows provisioning of an egress bandwidth limit and a burst size. The burst size defines how much the egress bandwidth limit can be exceeded for short periods of time. In general burst sizes are defined by the network implementation (SLA) and should be configured in the FZAP accordingly. Values given in this guide are to be understood as configuration examples. The FZAP will provide a minimum burst size of 25,000 bytes and a maximum burst size of approximately 1,040,000 bytes. When the configured burst size results in values outside of that range, the actual burst size will be limited to the range supported by the FZAP. Care should be taken to minimize the amount of IP fragmentation that occurs when egress port rate shaping is applied. The use of egress port rate shaping will increase the performance degradation that results from IP fragmentation. 2.4.3 Ingress Rate Limiting The FZAP does not provide ingress rate limiting. 28 Usage examples according to 3GPP TS23.203. © 2015 Nokia Networks DN09224449 Issue 01B 37/157 The FZC does provide ingress rate limiting capability for 10 Gbps links on both the north and south bound Ethernet interfaces. The FZC defaults to an ingress rate limit of 4 Gbps on any 10 GE port. There is no ingress rate limiting applied to 1 GE ports. 2.4.4 FZAP Port Traffic Roles For the FZAP when layer 2 switching is enabled, each of the L2 Ethernet ports can takes on a role related to the type of traffic being handled by the port. A port can only take on one role at a time. The roles are: Primary Backhaul. One port must always be assigned the primary backhaul role. The port assigned is the one that provides connectivity towards the FZC. LMT. Optionally a port can be configured for the connection of the LMT to the FZAP. Daisy Chain Backhaul. Optionally a port can be configured to operate as a daisy chain port when multiple FZAPs are connected in a chain. Packet Forwarding. The packet forwarding role allows traffic for that port to flow to/from the primary port. This will typically be used for Wi-Fi AP or other site traffic. The port used to connect an integrated Wi-Fi AP defaults to the packet forwarding role. External ports default to the packet forwarding role when they are not explicitly configured for one of the other roles, with the following exceptions: o Only one port can be defined for the packet forwarding role. If there are two ports that are not assigned one of the other roles, then neither is assigned the packet forwarding role. o For three-port hardware without an integrated Wi-Fi AP, when one port is assigned the daisy chain role, then the other port cannot be in the packet forwarding role. 2.5 Ethernet OAM Ethernet OAM is not supported by the FZAP or the FZC. 2.6 IP addressing in Flexi Zone A Flexi Zone Controller requires the creation of at least two northbound IP interfaces: 38 /157 IP interface dedicated for U-plane traffic Either/or: o One IP interface that supports combined M/C-plane traffic. o Two IP interfaces that support M-plane and C-plane traffic separately © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects The U-plane interface connects the FZC to the Serving Gateway. It is a mandatory interface on the Controller and functions as the only interface that transmits and receives GTP-U bearer traffic on the backhaul (S1-U). Management and C-plane services are not provided on this interface. The M-plane interface connects the FZC to iOMS/NetAct. It is a mandatory interface on the Controller and functions as the only interface that transmits and receives management traffic on the backhaul. U-plane (bearer) services are not provided on this interface. If no distinct C-plane interface is created, Mplane (nbmplane) and C-plane traffic are combined together onto the M-plane interface. The M-plane interface address also serves as the interface for FZAP management. The C-plane interface connects the FZC to an MME. It is an optional interface on the Controller when not combined with the M-plane interface and functions as the only interface that transmits and receives call control traffic on the backhaul with the MME. Uplane (bearer) services are not provided on this interface. M-plane (management) services are not provided on this interface. The S-plane supports synchronization traffic. There is no support for S-plane traffic in the FZC. The names of the U-plane, M-plane, and C-plane interfaces must be nbuplane, nbmplane, and nbcplane, respectively. The Z1, southbound, interface connects the FZC to the network of FZAPs. It is a mandatory interface on the Controller and functions as the default route for all FZAPs in a flat layer 2 network for the Zone eNB. In a non flat layer 2 network, such as one with multiple enterprises behind a router, the Z1 interface will function as the default route from the edge router for the enterprises. The name of the Z1 interface must be sbzplane. FZAPs request their IP address assignment from a DHCP server located on the FZC. The DHCP server will allocate an IP address from the configured IP subnet that applies to the FZAP. The FZAP will periodically renew the lease on the granted IP address. The IP address for the FZAP can be predetermined by configuring a static MAC/IP address mapping in the DHCP server. 2.7 IP layer fragmentation and network MTU When planning the mobile backhaul network, dimensioning of the MTU in all the involved nodes should be carefully considered, due to potentially significant impact on overall system performance. Flexi Zone Controller and Flexi Zone Acces Points support an MTU size range of 576 to 1644 octets. Compared to normal data networks based on IP/Ethernet, mobile backhaul incorporates two characteristics that require further consideration: a) End user packets (e.g. data downloaded from internet) are typically already aligned to standard IP/Ethernet parameters (e.g. IP payload is © 2015 Nokia Networks DN09224449 Issue 01B 39/157 often 1500 octets) and due to additional headers in transport (e.g. GTP-u) final packet encapsulation will quite often lead to packets larger than 1500 octets. b) Application of IPsec adds another overhead to each transport packet, thus further increasing the portion of packets to be fragmented. Total packet sizes resulting from these encapsulations are summarized in Table 2-19. Some applications in the management plane (e.g. CA server) apply the DF29 bit, thus preventing fragmentation of the data. However, in case a packet transmitted with DF bit on is exceeding the path MTU (e.g. due to added IPsec overhead) it might be dropped in the network. These issues can typically be solved by reducing the MTU in the involved endpoints. Some configurations in this document do not make use of Jumbo frames in order to show configuration examples for networks that cannot support Jumbo frames (either due to network/service provider limitations or due to missing availability of the Jumbo frame feature, e.g. in RL20 based networks). GTP with max. extension headers30 IPv4 End-user IP packet GTP GTP GTP (typical for S1) (typical for X2) 1500 1500 1500 32 8 16 UDP 8 8 8 IPv4 20 20 20 1560 1536 1544 24 24 24 12 12 12 2 2 2 Total ESP ESP Auth. 31 PL/NH 32 Padding Tunnel IPv4 Total with IPsec 6 14 6 20 20 20 1624 1608 1608 Table 2-19: Maximum packet size (IPv4) 29 Don’t Fragment 30 Please note that this column shows maximum values according to standards. NOKIA LTE systems do not use such packets currently, for applicable values please refer to the two columns on the right side. 31 Pad Length / Next Header (as defined in RFC4303) 32 Padding of up to 15 byte is required to align the overall ESP payload size to the block size of the chosen block cipher (AES-CBC with 128 bits = 16 byte). 40 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects 2.7.1 Jumbo frames supported end-to-end In case all nodes in the transport network support Jumbo frames, the Jumbo frame support provided by feature LTE931 Ethernet Jumbo Frames for the FZAP and the Jumbo frame support provided by feature LTE2203 Flexi Zone Controller Application on BCN Platform for the FZC should be used to completely avoid IP layer fragmentation. According to Table 2-19, for IPv4 an MTU of at least 1608 bytes should be supported in order to avoid fragmentation of 1500 byte payload IP packets with IP/UDP/GTP and IPsec headers, assuming that no GTP extension headers are used. When introducing jumbo frames in the network, it may be reasonable to dimension the transport network for a maximum packet size of 1644 bytes, allowing for potential IPv6-in-IPsec/IPv6 encapsulation in the future. 2.7.2 Jumbo frames not supported end-to-end If path MTU is limited to the typical value of 1500 octets (e.g. due to restrictions in intermediate nodes), the configurations described below are recommended. In case an even lower MTU applies, the parameter settings explained need to be decreased accordingly. In scenarios featuring IPsec, the security gateway at the core network edge demands a special configuration with respect to IP layer fragmentation/ reassembly. So it is recommended to enforce fragmentation of downlink clear text IPv4 packets in security gateway to 1400 byte before encryption (also called prefragmentation). This means that also after ESP encapsulation packets do not exceed default network MTU of 1500 byte. Please note, in general ESP encapsulation may add up to 93 bytes of overhead but 1400 byte was chosen here as it is a common setting in many security gateways. 1500 FZC Max pkt. size 1500 B 1560 1560 pre-fragmentation=> 1400 1560 1560 Max pkt. size 1560 B SAE‐GW Legend: Network I/F MTU Encryption MTU © 2015 Nokia Networks DN09224449 Issue 01B 41/157 Figure 2-4: MTU configuration applicable for downlink data (IPv4) 1560 1500 1560 FZC SAE‐GW 1560 Decryption Engine Figure 2-5: MTU configuration applicable for uplink data (IPv4) By assuming IPv4 traffic Figure 2-4 and Figure 2-5 explain the recommended MTU configuration when Jumbo frames are supported between SAE-GW, edge router and SEG (avoiding fragmentation as much as possible). SEG shall internally fragment IP packets to a 1400 byte boundary just before encryption. This avoids fragmentation of the ESP encapsulation packets at edge router – backhaul network interface (MTU=1500). In the uplink path, usage of Jumbo frames as recommended above (between edge router and SEG) does not cause any problems, as packets from eNB will be smaller or equal to 1500 byte anyway. In case that Jumbo frames are not supported at all, SAE-GW network interface MTU should be reduced according to backhaul network MTU, GTP and IPsec overheads in order to avoid double fragmentation. Based on an SEG MTU of 1400 byte, the recommended settings for the SAE-GW UE-IP layer MTU are: For IPv4: 1400 byte – 60 byte = 1340 byte. It is recommended to keep SEG internal fragmentation setting according to 1400 byte limit in order to ensure proper fragmentation also for traffic not coming from SAE-GW, such as control or management plane traffic. 2.7.3 Minimum burst size When configuring burst sizes in the system, the applicable MTU has to be taken into account as a lower limit for the burst size, as it needs to accommodate for at least one packet of maximum length. Consequently the minimum burst size would be defined as: 42 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport . General aspects 22 33 However, based on the NOKIA FZAP implementation it is recommended to configure the shaper burst size (parameters IEIF-sbs and IVIF-sbs, respectively) to a value of at least 4000 bytes. In case of smaller burst sizes, the shaper may show inaccurate results. Please note that this rule does not apply to the burst size configuration of the shaper(s) provided at the Ethernet switch ports (parameter ETHLNKl2BurstSize). These are based on a different implementation and application of the normal minimum burst size rule (as explained in the beginning of this section) is sufficient. The FZC does not support an egress rate shaping functionality. It is possible for the operator to configure ACL rules which provide egress rate limiting. 2.8 Synchronization For FDD-operation the synchronization options for FZAP and FZC hardware models are: - LTE1629 Integrated GPS synchronization for Flexi Zone Micro (FZAP only) - LTE713 Synchronous Ethernet (FZAP only) - LTE1623 Timing over Packet-Frequency Support for FZM (FZAP only) - LTE891 Timing over Packet with phase synchronization (FZAP only)34 - LTE1781 Integrated Multi-GNSS Sync Support (FZAP only) Note: The synchronization configurations described in this document are limited to use of IEEE1588-2008 (Timing-over-Packet) and Synchronous Ethernet, as these are seen as the most generic solutions. GPS and ToP with phase synchronization can be used just as well. The use of Synchronous Ethernet is limited to Access Points connected directly to an aggregation switch in the Zone network that is acting as a boundary clock. FZAPs do not provide the reclocking of signals necessary to utilize Synchronous Ethernet in a chained configuration; therefore, Synchronous Ethernet can only be used when the Flexi Zone Access Point is directly connected to the network. GPS synchronization can be used when the Access Point is in a chained configuration and not directly connected to the Backhaul network or when the network does not provide Synchronous Ethernet. 33 The additional 22 octets are required for the Ethernet header (14 octets), CRC (4 octets), and the optional VLAN tag (4 octets). If no VLANs are used, the minimum burst size allowed can be decreased by 4 octets accordingly. 34 The features LTE1623 and LTE891 are mutually exclusive. © 2015 Nokia Networks DN09224449 Issue 01B 43/157 The use of ToP phase synchronization to provide timing to FZAPs in a daisy chain, other than the head-of-chain FZAP, is not supported. Support of ToP frequency synchronization is supported. The FZC does not provide timing to the FZAPs. Timing is provided by a Grand Master Clock (GMC) that must be connected to the aggregation switch(es) that the FZAPs are connected to. The aggregation switch must act as a boundary clock. The aggregation switch needs to support IEEE1588-2008 (Timing-overPacket) or Synchronous Ethernet in order to provide the timing to the FZAPs. The FZC does not require frequency/phase synchronization. The FZC receives its wall clock time via NTP. Microsemi (formerly Symmetricom) TP5000 is the recommended solution for IEEE1588-2008 master functionality, thus it is used in the described cases. With respect to Synchronous Ethernet no specific Primary Reference Clock is recommended, any standards compliant device can be deployed. Note: In all cases with using ToP variants it is recommended to terminate ToP on Flexi Zone Access Point with a network interface IP address. 2.9 Security NOKIA eNB provides state-of-the-art transport layer security by means of the following IPsec features: Base station Flexi Zone Access Point Flexi Zone Controller Applicable IPsec feature LTE689 LTE IPsec Support LTE2017 IPSec Support for Flexi Zone Controller Table 2-20: IPsec feature overview Based on these features, traffic on all logical interfaces can be protected. The solution is based on a standards compliant IPsec architecture using ESP protocol (RFC4303). ESP services integrity protection, data origin authentication, confidentiality (encryption) and anti-replay protection are supported. IPsec for IEEE 1588-2008 (Timing-over-Packet) NOKIA LTE systems provide a complete IPsec solution, covering all logical interfaces and management plane. Hence, traffic received and transmitted by the FZAP can be protected by IPsec. However, when considering IPsec for Timing-over-Packet (IEEE 1588-2008) traffic, the following aspects should be considered: - As a matter of fact, the only potential target for synchronization plane attacks is to drive eNBs out of sync. However, basic design of IEEE 1588-2008 protocol prevents dramatic impacts, as it does not accept quickly changing conditions at all. An attack based on faked timing information will lead to ToP clients discarding this information in most of the cases. Any information accepted by the ToP client would lead to very 44 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects minor timing adjustments only, thus such an attack would require long term monitoring of the connection, followed by long period of injection of very carefully manipulated synchronization data. While such an attack is theoretically possible, it has to be noted that the overall risk of this extremely sophisticated approach is rather minor: only a single eNB can be slowly driven out of sync. - Full encryption of ToP traffic does not add any value compared to just authentication of the traffic, as no confidential data is exchanged. No significant difference in terms of processing delay is seen between full encryption and just authentication of ToP traffic in security gateways. The application of IPSec to synchronization traffic is not supported. 2.9.1 Flexi Zone Access Point local network security features Flexi Zone Access Point provides one optional feature to protect eNB and network from unauthorized access: LTE593 Security for Ethernet Ports protects the Ethernet ports on the FZAP against known Layer 2 vulnerabilities. 2.10 Auto configuration For the Flexi Zone there are two stages of autoconnection. The first stage is between the FZC and NetAct. This stage is based upon manual input of the autoconnection data separately for the BTS level functionality in the FZC and the FxP level functionality. The second stage of autoconnection is between the FZAPs and the FZC. After the FZC has completed its autoconnection/autoconfiguration, it acts as the Zone Manager for the FZAPs underneath it. The autoconnection/autoconfiguration of the FZAP comes about by connecting to the Zone Manager in the FZC. The autoconnection is automated, with the FZAP obtaining the necessary connection information from a DHCP server that is part of the Zone Manager on the FZC. Once the FZAP has obtained an IP address and associated information from the FZC DHCP server, the FZAP will continue to renew the lease on the address throughout the time that it is up. The loss of the lease or changes to the leased data will result in the FZAP resetting and performing a new autoconnection procedure. For the proper support of DHCP functionality with LTE2346 Shared Backhaul, the first hop routers (from FZAP perspective) need to provide DHCP relay towards the dedicated DHCP Server in the FZC. This is the same interface as the default router. In the case of northbound IPSec deployment on the FZC, then additional IPSec policies need to be manually configured on the FZC and SEG. The FxP needs to be manually configured to point to the CMP server. © 2015 Nokia Networks DN09224449 Issue 01B 45/157 2.11 Shared Backhaul The LTE2346 FZC Shared Backhaul Support Phase-I shared backhaul feature provides for the grouping of Flexi Zone Access Points (FZAP) being managed under a Flexi Zone Controller (FZC). Figure 2-6 illustrates the Enterprise networks provided by LTE2203 and LTE2346. Figure 2-6: Shared Backhaul Enterprises Shared backhaul will typically be used when FZAPs are being added within a building that has existing equipment and cable runs for an existing set of private networks. The shared backhaul capability allows the FZC to support FZAPs using the existing backhaul equipment within those private enterprises. An example is illustrated in Figure 2-7 . 46 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects Figure 2-7: Shared Backhaul Enterprise Deployment Shared backhaul allows for the addition of up to nine Enterprises. These Enterprises are created in addition to Enterprise 0 which always exists. However, Enterprise 0 may, at the operator’s choice, not have any FZAPs if there are other Enterprises defined. The L3 router must have an IP address from the Enterprise 0 subnet. The Enterprises must have separate IP subnets that are reachable through the L3 router. The firewalls cannot be configured to perform NAT. Shared backhaul is transparent from the FZAP point of view. The FZAP continues to acquire and maintain its IP address with the DHCP server. In the FZC the DHCP server provides IP address assignment to the FZAPs based upon the Enterprise that they are in. 2.12 Extended Redundancy Mechanisms Two extended redundancy mechanisms are supported on the FZC. There are no extended redundancy mechanisms supported on the FZAP: © 2015 Nokia Networks DN09224449 Issue 01B 47/157 1. “LTE775 - SCTP Multihoming” (at the MME; eNB MME end-to-end C-plane application specific; two C-plane IP addresses (primary and secondary) at MME side only. 2. “LTE866 – Fast IP Rerouting”; IP traffic re-routing between eNB next hop routers with selected IP traffic via dedicated routes and impacted by route bound BFD path supervision; 2.13 Transport Network Layer (TNL) IP Fragmentation Avoidance Feature LTE1240 – User layer TCP MSS Clamping reduces FZAP’s TNL IP layer fragmentation. Feature LTE1240 is not applicable for the FZC. The reduction of the Transport Network Layer (TNL) IP layer fragmentation in the RAN is achieved by shortening the packet length of TCP/IP user sessions encapsulated on top of GTP-U layer of the TNL. This results in the TNL IP layer packet lengths of the encapsulated user TCP/IP traffic being equal to or below the configured eNB MTU size. Normally the UE is unaware of the path MTU size in the RAN. So the negotiated user TCP MSS values between UE and the server in the internet represent only both peers’ internet configurations. Accordingly the MSS values are therefore internet adapted. In the FZAP the UE IP packets are encapsulated in the RAN’s TNL with the final result that the overall max packet length may be beyond FZAP’s MTU size. (In detail: TNL overhead GTP-U/UDP/IP, optionally IPsec- is added in the FZAP to the user TCP/IP traffic.). The packets beyond the FZAP’s MTU size require performance robbing IP fragmentation. In order to avoid user traffic fragmentation in the RAN the TCP MSS Clamping function is introduced. The FZAP observes the user’s TCP session establishment and manipulates the UEs negotiated MSS during TCP SYN phase. So finally the user’s TCP SYN MSS value considers FZAP’s TNL overhead inclusive of the FZAP’s MTU size towards the backhaul link. Note: Due to the limitation of the feature on user TCP/IP traffic this feature has no impact on any other non-TCP user traffic and also no impact on any other non user plane traffic, like M-plane traffic. So there may be still a few IP fragments observed in RAN. 48 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport General aspects Figure 2-8: User TCP/IP session MSS modification with TCP MSS clamping in FZAP; © 2015 Nokia Networks DN09224449 Issue 01B 49/157 3 Layer 3 access network with IPsec This section covers several different physical configurations for both the northbound and Z1 transport. For the northbound interface: 1. 10 GE link for user plane and 1 GE link for control and management plane traffic 2. 10 GE link for user plane and separate 1 GE links for control and management plane traffic 3. 10 GE link for user plane, control plane, and management plane traffic For the southbound Z1 transport: 1. Single 10 Gbps Ethernet link 2. Multiple 1 Gbps links Note: The Z1 transport configurations are mutually exclusive. None of the configurations cover FZAPs with integrated Wi-Fi APs. 3.1 Recommended network configuration This reference configuration describes a typical LTE Access Network configuration using a routed IP backbone. The radio access network is attached to core network via an IP router. A security gateway device is attached to (or integrated into) this edge router for termination of backhaul IPsec tunnels. Further details of this configuration: - Northbound Backhaul network: backhaul network provides IP transport between core edge router and FZC sites. A typical carrier implementation of this configuration is provisioning of L3 VPNs35 over an MPLS network or a flat IP network based on static or dynamic routing. - Northbound Backhaul network sharing: the network is shared by several services, but there is a guaranteed capacity allocated to the mobile network, e.g. by deploying a dedicated VPN for the backhaul. Dimensioning of the guaranteed capacity should be according to the mobile network dimensioning. The access network part (“last mile”) is not shared with other services. - Z1 Backhaul network: backhaul network provides IP transport between FZC and FZAP sites. - Synchronization: in this configuration, synchronization is provided by IEEE 1588-2008 (Timing-over-Packet). 35 Please note L3 VPNs are not to be mistaken for IPsec VPNs. A definition of generic L3 VPNs is provided by RFC4026; a typical implementation of such L3 VPNs is based on BGP/MPLS IP VPNs as specified in RFC4364. 50 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec - Network redundancy: It is assumed that the transport network features inherent redundancy, transparently protecting the mobile traffic against failures inside the network. - Northbound and Z1 Backhaul network QoS: backhaul network is able to prioritize traffic based on DSCP as received at the edges of the network. - O&M connectivity: from FZC perspective management services are reached via the core network edge router. - Logical X2 topology: all X2 connections are established via the core network edge router (“X2 star” configuration). - Network security: traffic on all logical interfaces is secured. The solution is based on a standards compliant IPsec architecture using ESP protocol (RFC4303). ESP services integrity protection, data origin authentication, confidentiality (encryption) and anti-replay protection are used. 3.1.1 Configuration overview Figure 3-1 shows the recommended implementation of an LTE radio access network featuring L3 VPNs in the backhaul network and IPsec for transport security. Logically this configuration implements X2 star topology, i.e. all IPsec tunnels are terminated at the core network edge, so there are no end-to-end X2 tunnels between two eNodeBs. Please note, although security gateway (SEG) is shown attached to the router, router is configured in a way that all protected traffic needs to pass SEG, thus preventing bypassing of SEG. Figure 3-1: Recommended configuration for layer 3 based access network NOTE: eNB#1 is shown for reference purpose to show X2 flows. Configuration of eNB#1 will not be covered in this document. In the shared transport network, a dedicated L3 VPN is provisioned for the mobile network traffic, effectively separating it from other services using the network. In practice, using multiple backhaul VPNs is recommended (e.g. in order to limit fault domains. Separation of the network into several VPNs also © 2015 Nokia Networks DN09224449 Issue 01B 51/157 increases security by completely preventing packet transmission between different L3 VPNs – thus it is impossible to sniff packets from a different L3 VPN. In case of traffic engineering capable backhaul networks, dimensioning of L3 VPN capacity is supported in principle. However, it is not used in this example. Transport QoS is provided by means of DiffServ architecture: all edge nodes in the transport network are required to apply egress traffic classification based on DSCP (traffic prioritization within the transport network itself may be implemented differently, though, for example based on MPLS EXP marking). eNB provides DSCP marking based on traffic class. In downlink direction it is assumed that proper DSCP marking is provided by core network nodes. Thus proper configuration of traffic classification and scheduling in eNBs and core network is essential in order to guarantee end user QoS. Synchronization of the radio access network is provided via IEEE1588-2008 (Timing-over-Packet) as this technology is agnostic to physical network implementation. The IEEE1588 master or boundary clock needs to be positioned within the Z1 backhaul to support FZAP synchronization. Aggregation network resilience is expected to be provided by the transport network in a transparent way. This means, average availability and maximum downtime of the used VPN(s) need to offer carrier level quality. However, deployment of SCTP (MME) multi-homing can improve network resilience at transport layer for the control plane. Transport security is provided by means of IPsec for user, control and management plane. All traffic is integrity protected and encrypted. ToP (synchronization) traffic is not protected as described in section 0. Security architecture is based on a Public Key Infrastructure offered by Nokia Security Services. In this configuration, dedicated IPsec tunnels between the central security gateway and the FZC are deployed, serving both S1 and X2 transport (“X2 star” topology). IPsec key management is provided by IKEv2 protocol in this reference configuration, in general IKEv1 is supported by NOKIA eNodeB, too. The Flexi Zone Controller (FZC) and Flexi Zone Access Points (FZAP) shown in Figure 3-1 comprise a logical eNB. To the backhaul network and the Enhanced Packet Core the combination looks like a single high capacity eNB. The Z1 provides connectivity between the Flexi Zone Controller and the Flexi Zone Access Points. Figure 3-2 shows the recommended high capacity implementation based on the 10 Gbps Ethernet link for the Z1 southbound backhaul. In the Z1 one or more Fan out switches are used to provide the connectivity. An IEEE1588v2 master source is required to provide synchronization. This master source can be a dedicated master or another switch with Boundary clock functionality that has connection to a ToP master clock. Alternatively the built in GNSS (GPS or GLONASS) capabilities of the FZAP can be used for synchronization. 52 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec Figure 3-2: Recommended configuration for Z1 with Single 10 GE Link Figure 3-3 shows the recommended implementation of the Z1 based upon the use of multiple 1 Gbps links for the Z1 backhaul. In the Z1Fan out switches are used to provide the connectivity. On the controller side there are up to 7 individual 1 Gbps Ethernet interfaces that can be used. Connected to each of the interfaces there is a Fan out switch that is used to connect multiple FZAP under the one link. The number of Flexi Zone Access points that can be supported by one link and the aggregation switch is based on the traffic model for the individual cells (see Section 2.2). An IEEE1588v2 master source is required to provide synchronization. This master source can be a dedicated master or another switch with Boundary clock functionality that has connection to a ToP master clock. Since spanning tree is not supported by the FZC, the switches cannot be cross connected which is the reason each switch has a connection to the master clock. Alternatively the built-in GNSS (GPS or GLONASS) capabilities of the FZAP can be used for synchronization. Figure 3-3: Recommended configuration for Z1 with Multiple 1 GE Links © 2015 Nokia Networks DN09224449 Issue 01B 53/157 IPSec is implemented on the interface between the FZAP and the FZC with the FZC acting as a Security Gateway to the Z1. In this configuration the same Public Key Infrastructure used on the Backhaul network is leveraged by the Z1. The FZC proxies PKI traffic from the FZAP as necessary to provide a single IP interface for the complete zone for all Public Key Infrastructure traffic. 3.1.2 Feature usage and interdependences The features supported for the FZC and FZAPs are provided in Table 1-1 and Table 1-2. 3.1.3 Transport network elements 3.1.3.1 Core network edge router In this configuration, core and backhaul networks are interconnected via an edge router. This additional gateway provides various benefits: - Full flexibility with respect to IP subnetting and addressing, full IP subnet separation between core, backhaul, and O&M, - QoS (e.g. alignment of traffic marking in core and backhaul, downlink shaping for backhaul traffic, etc), - Protection of core against various layer 2 network disturbances (e.g. broadcast storms originating somewhere in the backhaul network), - Detailed performance monitoring can be performed. Due to the central location (both physically and logically) high availability of this edge router is of utmost importance. It is recommended to consider full redundancy; including supervisor boards/routing engines, line cards, fan trays, power supplies, etc. Software redundancy features (such as in-service software update capability) significantly simplify maintenance of the equipment. In addition, this configuration features a security gateway which is attached to (or integrated into) this router. 3.1.3.2 Security gateway (IPsec) In this configuration, IPsec tunnels originating in FZCs are terminated in a security gateway attached to backbone edge router. A high availability solution needs to be deployed, providing the required resilience for IPsec termination. 3.1.3.3 Aggregation network PE routers As this configuration is meant to be generic also in the sense that the whole aggregation network could be provided by a third party network service provider, no particular requirements are established for aggregation PE routers, except the capability of providing IP interfaces towards FZC sites and maintenance of proper IP routes towards core network. Depending on the exact backhaul network implementation this could mean dynamic routing and MPLS support. 54 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec 3.1.3.4 Fan Out Switches In addition to the normal Ethernet related requirements, such as MTU, VLAN separation and QoS support, switches deployed in the Z1 need to support IEEE1588v2 Boundary Clock functionality. 3.2 RAN parameter examples for the configuration 3.2.1 Backhaul network VPN service configuration Figure 3-4 shows the detailed backhaul network configuration. No specific recommendations with respect to the implementation of the aggregation part (backbone) are given as simply a generic, isolated L3 VPN service is expected, making this configuration applicable to a multitude of different networks. A typical implementation of this scenario is using BGP/MPLS IP VPNs (RFC4364) based on a multi-service IP/MPLS backbone (potentially operated by a third party service provider). Figure 3-4: Detailed backhaul network configuration This configuration example comprises just a single backhaul VPN, which is certainly feasible for the eNBs deployed in this configuration. For large scale deployments it is obviously more reasonable to use multiple VPNs. Although in general IP/MPLS backbones are often capable of trafficengineering, this configuration will not rely on it. Instead, sufficient transport capacity is assumed to be provisioned to the backhaul VPN. Section 2.2 explains a basic dimensioning approach that can be used to estimate backhaul VPN bandwidth demand. The configured L3 backhaul VPN service provides an IP layer MTU of 1500 octets. Backhaul network edges are represented by provider edge (PE) routers: - PER at the core network edge - PEM1 connecting to Flexi Zone Controller © 2015 Nokia Networks DN09224449 Issue 01B 55/157 - PEM2 connecting to eNB #1 (shown for reference only) Router interfaces connecting to Flexi Zone Controller will be configured as VLAN interfaces using VLAN ID 48 for M-Plane and C-Plane on a 1 Gigabit Ethernet link. U-Plane will use VLAN ID 20 on a 10 Gigabit Ethernet link. NOTE: the VLAN numbers are for example only. While not strictly required in this configuration using VLANs is recommended to allow easier addition of features in later releases 3.2.2 Flexi Zone detailed configurations All FZAPs in this scenario are connected to the transport network via 1 Gbps Ethernet. This is done using any of the possible physical Ethernet connections supported for the FZAP, such as 1000BASE-SX, 1000BASE-LX, or 1000BASE-T. Two configurations are presented, one which is based on a single 10 Gbps Z1 backhaul using a 10GBase-SR to an aggregation switch and the other is based on the use of multiple 1 Gbps Z1 backhauls using multiple 1000Base-SX to multiple aggregation switches. The two configurations are mutually exclusive and cannot be combined on the same FZC. For both configurations, the FZC is configured with two physical connections to the northbound backhaul network. There is a 1000Base-SX connection for the M/C-plane traffic and a 10GBase-SR connection for the U-Plane traffic. NOTE: if additional physical separation is required between the switches or routers and the FZAP or FZC 1000Base-LX and 10GBase-LR can be used with no impact to the configuration. For traffic models that are heavily weighted towards small packets, utilizing the full 10 Gbps bandwidth of the Z1 and northbound U-plane interfaces may result in CPU overload on the FZC. In such cases, the ingress policing should limit the bandwidth on those interfaces to 4 Gbps. IPsec is deployed for all logical interfaces based on LTE689 LTE IPsec Support 3.2.2.1 Ethernet layer configuration Figure 3-5 shows the basic L1/L2 connectivity used for these reference configurations for Flexi Zone Controller. There are two physical connections to the PE router. The 1Gbps link is used for C-Plane and M-Plane traffic while the 10Gbps link is used for U-Plane. Towards the Backhaul network from the aggregation switch a single 10Gpbs link is used. QoS classification is performed based on DSCP as no PCP support is expected from backhaul network. This also means a DSCP map needs to be configured, providing the proper mapping from DSCP to transmission queue. Details of this mapping are provided in Table 2-14. 56 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec Figure 3-5: Flexi Zone Controller Backhaul connections. Figure 3-6 shows an alternative L1/L2 connectivity for Flexi Zone Controller. There is one physical 10Gbps connection to the PE router. Towards the Backhaul network from the aggregation switch a single 10Gpbs link is used. Figure 3-6: Flexi Zone Controller Backhaul connections. Figure 3-7 shows the basic L1/L2 connectivity used in single 10 Gbps link Z1 backhaul configuration. This configuration uses VLANs on all interfaces to allow for future migration to configurations that require VLAN differentiation with needing to reconfigure the existing connections. For the Z1 based upon a 10 Gbps link for the Z1 backhaul, as shown in Figure 3-2, the interface between the Flexi Zone Controller and Fan Out Switch #1 is a 10 Gbps connection to provide enough bandwidth when the maximum of 100 FZAP are configured into the Zone. Likewise the links between switches in a 10Gbps link; however depending on the actual bandwidth require smaller links could be used. The traffic model in Section 2.2.2 implies a 100 Mbps link could be used between the switch and the FZAP; however, when the overhead of IPSec is added the traffic at full load could overload a 100Mbps connection so a 1Gbps link is used. Fanout Switch #1 provides connection to the IEEE1588 master clock. For the Z1 based upon multiple 1 Gbps links for the Z1 backhaul, as shown in Figure 3-3, the interface between the Flexi Zone Controller and each Fan Out Switch is a 1 Gbps connection is shown in Figure 3-8. The traffic model in Section 2.2.2 implies a 100 Mbps link could be used between the switch and the FZAP; however, when the overhead of IPSec is added the traffic at full load could overload a 100Mbps connection so a 1Gbps link is used. © 2015 Nokia Networks DN09224449 Issue 01B 57/157 Each Fanout Switch provides connection to the IEEE1588 master clock. If a master or boundary clock is not supported at the PE, an actual IEEE1588 Master clock can be provided within the Z1 transport network if the backhaul network cannot pass the IEEE1588 traffic from the Master clock. Each of the Fanout Switches should also be a boundary clock to provide the best timing accuracy. The FZAP expect sync packets to be sent at the rate of 16 packets per second; therefore, when using Ethernet Multicast mode the clock providing the timing into the network needs to be configured for 16 packets per second. NOTE: An alternative to using IEEE1588 multicast mode is to use the unicast mode. With Unicast mode the PE device would require an IP from the Z1 so that the FZAP could contact it. This creates a potential for traffic bypass so the PE router should have ACL control list configured to prevent direct access for anything but IEEE1588 traffic into the Z1 which is why multicast mode is preferred. The configuration only shows 3 FZAP per switch. In the typical configuration where the Ethernet switch is a 48 port device a more typical install would have 40-45 FZAP per switch. Figure 3-7: Flexi Zone Access Point connections for 10 Gbps Z1 Backhaul. 58 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec Figure 3-8: Flexi Zone Access Point connections for Multiple 1 Gbps Z1 Backhaul. 3.2.2.2 FZC Northbound IP layer configuration based on collapsed IP addressing Figure 3-9 shows the recommended IP layer configuration for collapsed IP addressing, where the transport and application IP address are the same, in the Flexi Zone controller for the interfaces towards the Backhaul network. In this configuration the IP address assigned to the interface is also used as the application IP and the IPSec local tunnel endpoint. Network Interface 1 FZC IPsec Tunnel‐C/M Network Interface 2 M/C IPsec Tunnel–U U Physical Interface Network Interface IP address Figure 3-9: Flexi Zone Controller Backhaul IP Layer configuration IP layer configuration details for Flexi Zone Controller for dual 10 GE and 1 GE northbound links are shown in Table 3-1. NOTE: IP addresses and VLAN number are for example only. © 2015 Nokia Networks DN09224449 Issue 01B 59/157 FZC /f VLAN IP address SFP13 48 100.64.48.72 /25 SFP12 20 100.64.68.72 /25 Configured applications 1. Network interface IP address (IPsec termination) 2. Control plane 3. Management plane 1. Network interface IP address (IPsec termination) 2. User plane Table 3-1: FZC IP address configuration 3.2.2.3 FZC Northbound IP layer configuration based on virtual application IP addressing Figure 3-10 shows the recommended IP layer configuration in the Flexi Zone controller for the interfaces towards the Backhaul network. In this configuration the IP address assigned to the interface is also used as the application IP and the IPSec local tunnel endpoint. FZC Network Interface 1 C Network Interface 2 M U IPsec Tunnel‐C/M Physical Interface IPsec Tunnel–U Network Interface IP address Virtual IP address Figure 3-10: Flexi Zone Controller Backhaul IP Layer configuration IP layer configuration details for Flexi Zone Controller for a single 10 GE northbound link are shown in Table 3-2. NOTE: IP addresses and VLAN number are for example only. FZC /f VLAN 10 SFP12 10 Virtual Virtual Virtual n/a n/a n/a IP address 100.64.17.8 /28 100.64.17.7 /28 100.64.80.72 100.64.112.72 100.64.48.72 Configured applications Network interface IP address (M/C-plane IPsec termination) Network interface IP address (U-plane IPsec termination) Control plane Management plane User plane Table 3-2: FZC IP address configuration 60 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec 3.2.2.4 FZAP IP layer configuration based on application IP addressing Figure 3-11 shows the recommended IP layer configuration in the Flexi Zone Access Point IP configuration. In this configuration the IP address assigned to the interface is also used as the application IP and the IPSec local tunnel endpoint. The assignment of the IP address is performed by a DHCP server located on the FZC. The FZC automatically allocates addresses out of the subnet defined on its interface to the Z1. Network Interface 1 FZAP IPsec Tunnel‐ M/C/U C/U M/S Physical Interface Network Interface IP address Figure 3-11: Flexi Zone Access Point IP Layer configuration IP layer configuration details for Flexi Zone Controller Z1 backhaul are shown in Table 3-3. NOTE: IP addresses and VLAN number are for example only. FZC /f SFP11 (10GE based Z1) SFP15 SFP22 (Multip le 1GE based Z1) VLAN 192 192 IP address Configured applications 192.168.192.1 /24 1. Network interface IP address (IPsec termination) 2. User plane 3. Control plane 4. Management plane 192.168.192.1 /24 1. Network interface IP address (IPsec termination) 2. User plane 3. Control plane 4. Management plane Table 3-3: FZC IP address configuration IP layer configuration details for Flexi Zone AP of this case are shown in Table 3-4. © 2015 Nokia Networks DN09224449 Issue 01B 61/157 FZAP IF EIF1 VLAN IP address 192 DHCP Assigned from 192.168.192.0 /24 Configured applications 1. Network interface IP address (IPsec termination) 2. User plane 3. Control plane 4. Management plane 5. Synchronization Plane Table 3-4: Flexi Zone Access Point IP address configuration Due to application of IPsec in this case, MTU size has to be carefully configured, i.e. MTU of the Flexi Zone Controller’s Ethernet interfaces shall be set to Path MTU (PMTU) supported by the whole network (in this case 1500 byte as specified above). The Flexi Zone Controllers MTU on the Z1 should be set the same as the backhaul network MTU. The DHCP server in the Flexi Zone Controller will then configure the Flexi Zone Access Point for the same MTU. This prevents fragmentation of encrypted IP packets during whole transport end-to-end, instead eNB will derive IPsec header size from configuration and fragment clear text packet accordingly before applying encryption/authentication. 3.2.2.5 Uplink Traffic QoS Configuration of quality of service in the FZAP comprises the following core components: - DSCP mapping profile - Uplink packet scheduler configuration DiffServ markings are configured according to section 2.3.1. Uplink packet scheduler parameters in FZAPs are configured according to Table 2-16. Uplink traffic shaping is configured at line rate since sufficient backhaul network capacity is assumed. Table 3-5 shows the parameterization of the aggregate traffic shaper (total egress traffic per FZAP) which is common to all FZAPs in this configuration. Object IEIF APELNK Parameter qosEnabled trafficPathShapingEnable upperLayerShaping sirTotal sbsTotal sir / sbs wfqSchedQueueWeight localIpAddr l2BurstSize l2ShaperRate Value true 0 (TPS_OFF) n/a n/a n/a n/a n/a 0.0.0.0 0 RT_LINE_RATE (0) Table 3-5: Common eNodeB aggregate traffic shaping configuration When the DHCP Sever is on a VLAN, the IVIF object defines the settings for the VLAN related QoS. If this object has not been provided in the site 62 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec configuration file, the FZAP will create the IVIF object. The VLAN ID is set to the value of the VLAN that the DHCP server is on and the local IP address is set to the value provided by the DHCP server. The remaining attribute values of the IVIF are copied from the IEIF. An example of the created IVIF object is provided by Table 3-6. For each FZAP just a single VLAN is used, WFQ weights do not have a real practical meaning and are left with the default value of 1000. This configuration is applied to all FZAPs. eNB 1 VLAN 192 Parameter vlanId36 localIpAddr37 qosEnabled sir sbs wfqSchedQueueWeight Value 192 192.168.192.10 true (copied from IEIF)38 n/a (copied from IEIF) n/a (copied from IEIF) 1000 (copied from IEIF) Table 3-6: Configuration of VLAN related QoS parameters 3.2.2.6 FZAP QoS Aware Ethernet Switching The FZAP embedded Ethernet switch provides parameters for configuration of the following aspects: - Traffic classification (DSCP to priority mapping) - Queuing / scheduling (based on priority) DiffServ marking to PCP (priority) mappings are configured according to section 2.3.1. The configuration of the L2 packet scheduling for the egress ports is defined in Table 3-7. This configuration would be applied to all FZAPs. 36 VLAN ID is determined by setting the value to the same VLAN that the DHCP server is on. 37 The value for the localIpAddr is provided by the DHCP server. 38 Must be set to “true” in the IEIF. © 2015 Nokia Networks DN09224449 Issue 01B 63/157 Object Parameter dscpMap (dscp-> priorityQueue) APL2SW enableLayer2Switching l2PriorityQueueWeight2 l2PriorityQueueWeight3 l2PriorityQueueWeight4 l2PriorityQueueWeight5 l2PriorityQueueWeight6 portDefaultVlanId portDefaultPriority priorityQueueNonIP priorityQueuePcp0 priorityQueuePcp1 priorityQueuePcp2 priorityQueuePcp3 priorityQueuePcp4 priorityQueuePcp5 priorityQueuePcp6 priorityQueuePcp7 priorityQueueUntagged qosClassification vlanAwareSwitch Value 46, 48, 56 -> 1 (EF) 34, 36, 38 -> 2 (AF4) 26, 28, 30 -> 3 (AF3) 18, 20, 22 -> 4 (AF2) 10, 12, 14 -> 5 (AF1) all others -> 6 (BE) true 8 4 1 1 1 not used not used 1 6 5 4 4 3 2 1 1 1 0 (based on DSCP) always be set to false Table 3-7: QoS Aware Ethernet switching configuration When supporting Wi-Fi traffic an evaluation should be done that considers the relative priority between LTE and Wi-Fi traffic desired by the operator. Adjustments may be necessary to the dscp to queue mapping, priority queue weights, and priority to queue mappings. 3.2.2.7 IPsec This configuration provides state-of-the-art transport network security based on IPsec architecture. This section describes configuration of the Zone eNB IPsec Security Policy Database (SPD). The database defines how the different traffic flows are handled by IPsec, i.e. if and how certain traffic is protected. Note: this document focuses on describing the configuration applied in the FZC. Obviously a suitable configuration of IPsec gateways is required in addition. However, due to high dependency on actual gateway device selection and high complexity of a high-availability IPsec gateway installation, description of the network side IPsec configuration cannot be covered in this document. Further assistance is offered by NOKIA Security Services division39. Security Policy Database configured in Flexi Zone Controller in this configuration is described below. Parameters provided by Table 3-8 apply to 39 64 /157 http://www.nokiasiemensnetworks.com/portfolio/services/security © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec all security policies. Certificate based authentication is applied based on a Public Key Infrastructure (PKI), but more detailed description of it is out of scope of this document. Integrity protection is always applied (not configurable) and HMAC-SHA1-96 algorithm is used for all protected traffic flows (regardless of configured encryption). The encryption method setting ENC_AES_128_CBC_OR_3DES_192_CBC means that either AES-128 (preferred) or 3DES-192 will be negotiated with the peer. Parameter Value Anti-Replay (window size) IKE protocol version IKE encryption method ESP encryption method40 Diffie-Hellman group Active (256) IKEv2 ENC_AES_128_CBC_OR_3DES_192_CBC ENC_AES_128_CBC_OR_3DES_192_CBC DH2 Dead peer detection delay Dead peer detection timeout Security Association lifetime 10 seconds 120 seconds41 24 hours Table 3-8: Common IPsec parameters shared by all security policies Table 3-9 shows the security policy database. For all the listed policies, encryption method shall be configured to try AES-128 CBC first, followed by 3DES-192 CBC (this is achieved by using the settings shown in Table 3-8). Policy order number needs to be different for each policy, however it only applies in case multiple policies match (based on local/remote IP, local/remote port and protocol information) for the same packet. In that particular case, policy with lowest order number will be applied. Tunnel Endpoint IP Address 1000 any any / any 100.64.48.72 /32 <CA Server IP> n/a BYPASS 1200 1 (ICMP) any / any 100.64.48.72 /32 any n/a PROTECT 1300 any any / any 100.64.68.72 /32 any UP Local IP Address / Subnet Remote IP Address / Subnet Encrypt BYPASS ICMP Port Number (local/remote) Order Number CA Policy Name Protocol IPsec Action (ipSecStatus) L4 yes Local Remote 100.64.68.72 SEG address CP PROTECT 1400 any any / any 100.64.48.72 /32 any yes 100.64.48.72 SEG address MP PROTECT 1500 any any / any 100.64.48.72 /32 any yes 100.64.48.72 SEG address Table 3-9: IPsec Security Policy Database (SPD) for Flexi Zone Controller Northbound 40 Applies only for policies marked for encryption. 41 Actually not relevant here as it applies to IKEv1 configurations only. © 2015 Nokia Networks DN09224449 Issue 01B 65/157 Further remarks with respect to Table 3-9: - For the majority of policies, remote IP address parameters are intentionally not configured for maintenance simplification. Otherwise for example a change of a single IP address in SAE-GW would require adjustment of IPsec policies in all eNBs in the network. - Also specific L4 protocol information is not configured for all policies except the ICMP bypass policy, in order to allow sending/receiving of protected ICMP traffic (e.g. ping requests/responses) through IPsec tunnels. If this is not intended, protocol configuration can be set to specific values instead (e.g. 17 for UDP in the UP policy). The ICMP bypass policy covers ICMP traffic, which is sent/received on the eNB interface address (tunnel address), i.e. ICMP traffic outside IPsec tunnels. - In addition, concrete L4 port numbers are not configured, although well defined numbers exist (e.g. 2152 for GTP-U). However, in order to further harden the configuration, operator selected ports can obviously be used in the policy configuration instead. - As X2 is configured in a logical star topology with all traffic sent to the edge router connected IPsec gateway, common S1/X2 policies are configured for UP and CP traffic, respectively. - The policy for O&M traffic („MP“) is intentionally configured based on eNB local IP address only. This avoids the need to explicitly specify all the NetAct/O&M related addresses. In addition this policy will also apply for the LDAP connections, namely those needed for remote user authentication (RUIM) and certificate management. It will also cover the certificate management (CMP) traffic in general. - IEEE1588-2008 traffic is normally bypassed from IPsec, see section 3.1.1. In this configuration the raw Ethernet multicast mode, where there is no IP header, is used so no specific IPsec bypass is required. - Packets not matching any of the policies will be discarded by default. Table 3-11 shows SPDs for the FZC’s Z1 backhaul. 66 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Layer 3 access network with IPsec Tunnel Endpoint IP Address Encrypt Port Number (local/remote) Action Protocol Policy Name Order Number L4 Remote IP Local IP Address Address / / Subnet Subnet Local Remote ICMP BYPASS 1100 1 (ICMP) any / any 192.168.192.1/24 any n/a DHCP BYPASS 1100 UDP 192.168.192.1/24 any n/a UP-CP-MP PROTECT 1200 any any / any 192.168.192.1/24 any 68/67 yes 192.168.1 192.168.192.0/ 92.1 24 Table 3-10: IPsec Security Policy Database (SPD) for FZC Z1 Backhaul Table 3-11 shows SPDs for the FZAP. The policies in this table are automatically created and pushed to the FZAP by the FZC and are shown for reference only. Tunnel Endpoint IP Address ICMP BYPASS 1100 1 (ICMP) any / any DHCP BYPASS 1100 UP-CP-MP PROTECT 1200 UDP any 68/67 Encrypt Port Number (local/remote) Action Protocol Policy Name Order Number L4 Local IP Address / Subnet Remote IP Address / Subnet DHCP Assigned/32 any Any any n/a any DHCP 192.168.192.1 yes Assigned DHCP any / any Assigned/32 Local Remote n/a Table 3-11: IPsec Security Policy Database (SPD) for Flexi Zone Access Points © 2015 Nokia Networks DN09224449 Issue 01B 67/157 4 Multiple Enterprise Zone Shared Backhaul The shared backhaul configuration is based upon the 10 Gbps Z1 backhaul presented in Sections 3 and 4. As shown in Figure 4-1, shared backhaul adds an L3 router to handle the routing of the traffic between the FZC and the private Enterprises. Figure 4-1: Recommended configuration for Shared Backhaul Z1 Figure 4-2 shows the basic connectivity used for the Shared Backhaul Z Network. This configuration is an extension of the 10 Gbps link for the Z1 backhaul. Configuration of the FZAPs is unchanged. Configuration on the FZC is unchanged, with the exception that the IP subnet for each Enterprise is different and the MTU size may be configured independently for each Enterprise. The firewalls are not shown specifically in the figure at the Enterprise fan out switches. These firewalls cannot perform NAT functions. Enabling of the shared backhaul feature is done through the FZC’s SCLI user interface. When enabling the feature, the operator defines the default gateway for the controller’s southbound Z1 interface. This address must be allocated from the Enterprise 0 subnet. An Enterprise is based on a separate IP address subnet that applies to FZAPs that are physically connected to the routing/switching hardware that defines the shared backhaul Enterprise. Configuration of the Enterprises is done through the FZC’s SCLI user interface. When adding an Enterprise, the operator will provide the following information: 68 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Multiple Enterprise Zone Shared Backhaul 1. Enterprise Name (Mandatory): The name can be up to 15 characters in length and must be unique from the other Enterprises. Enterprise 0 has the fixed name of “Base”. 2. Enterprise IP Address (Mandatory): Enterprises must be defined with unique IP subnets that do not overlap with other IP address assignments within the FZC or for other Enterprises supported by the FZC. This is the default gateway that each FZAP will use. This is also the IP address for the DHCP relay. 3. IP Subnet Mask (Mandatory) 4. Starting IP Address of Allocation Range (Optional; defaults to first valid host IP in the IP subnet) 5. Ending IP Address of Allocation Range (Optional; defaults to last valid host IP in the IP subnet) 6. MTU Size (Optional; range 576-1644 octets, defaults to 1500 bytes) The L3 router is the default route for the FZAPs in the Enterprise networks. The L3 router acts as a DHCP relay agent in support of FZAP connectivity to the DHCP Server on the FZC. The configuration of the DHCP server is handled automatically as the Enterprises are added and removed. In addition, IPSec of the user, control, and management plane traffic on the Z1 is required before an Enterprise can be added. These IPSec policies are created automatically as the Enterprises are created. The operator shall not configure IPSec bypass rules for Enterprise traffic. Figure 4-2: Flexi Zone Access Point connections Shared Z1 Backhaul. The L3 Router will need to have QoS configured to handle potential congestion on its links. The QoS configuration should be based upon the same traffic class to queue mappings as shown in Table 2-14. If the L3 Router only supports four packet scheduling queues then it is recommended that: © 2015 Nokia Networks DN09224449 Issue 01B 69/157 EF traffic is mapped to Queue 0; strict priority AF4 traffic is mapped to Queue 1; WFQ with weight >= 8x Queue 3 and >= 4x Queue 2 AF1, AF2, AF3 traffic is mapped to Queue 2; WFQ BE traffic is mapped to Queue 3; WFQ If the L3 Router is handling Wi-Fi AP traffic the prioritization of the traffic with respect to the LTE traffic is at the operator’s discretion. 70 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport 5 References References [LTE_Access_Dim] “LTE Access Dimensioning Guideline”, DN0951772 [LTE_Transport] “LTE Transport”, DN0943983 [LTE_TM] “LTE Traffic Model”, DN0951784 [IP_Monitor] “IP Transport Monitoring Guideline”, DN09117727 [MEF11] “MEF 11: User Network Interface (UNI) Requirements and Framework”, Nov. 2004, http://www.metroethernetforum.org [3GPP_23203] “Policy and charging control architecture”, 3GPP TS23.203, version 8.3.1 [Backhaul Sharing QoS Guideline] Backhaul Sharing QoS Guideline, DN09123927 [LTE_RL40_Featur LTE RL40, Feature Descriptions and Instructions, e_Description] DN09108269 [Config_Guide_RL 60] Configuring Flexi Zone Controller LTE Transport Document for RL60 © 2015 Nokia Networks DN09224449 Issue 01B 71/157 6 Glossary AIS ANR ARP BGP BH BTS CA CBS CCM CE CF CFM CIR CM CMP CR CoS DSCP DoS EBS EIR eNB E-OAM ESP EVC FSME FSMF FZM GBR GW HMAC HSRP IKE IP IPsec ISP 72 /157 Alarm Indication Signal Automatic Neighbor Relation (detection) Address Resolution Protocol Border Gateway Protocol Backhaul Base Transceiver Station, base station Certification Authority Committed Burst Size Continuity Check Message (IEEE802.1ag) Customer Edge Coupling Flag Connectivity Fault Management (IEEE802.1ag) Committed Information Rate Color Mode Certificate Management Protocol Certificate Repository Class of Service Differentiated Services Code Point Denial of Service Excess Burst Size Excess Information Rate Evolved Node B (eNodeB) Ethernet OAM Encapsulating Security Payload Ethernet Virtual Connection Flexi base station system module release “E” Flexi Multiradio 10 base station, system module rel. “F” Flexi Zone Micro base station Guaranteed Bitrate Gateway Hashed Message Authentication Code Hot Standby Router Protocol Internet Key Exchange (RFC4306) Internet Protocol IP Security (RFC4301) Internet Service Provider © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport L2CP LDAP MA MB-TAC MAN MD MDL MEF MEN MEP MH MLD MME MPLS MR MSS ND PCP PE PHB PKI POP PRC PSK QCI QoS RA RAN RRM RUIM S1AP SAD SAE-GW SCTP SEG SLA SPD SyncE TAC TNL MB-TAC ToP UNI VPN VRRP WFQ X2AP Layer 2 Control Protocol Lightweight Directory Access Protocol Maintenance Association Measurement-based Transport admission control Metro Area Network Maintenance Domain Maintenance Domain Level Metro Ethernet Forum Metro Ethernet Network Maintenance End Point Multihoming Multicast Listener Discovery Mobility Management Entity Multi-Protocol Label Switching Multi Radio Maximum Segment Size Neighbor Discovery Priority Code Point (IEEE802.1p VLAN priority bits) Provider Edge Per Hop Behavior Public Key Infrastructure (ITU-T X.509) Point Of Presence Primary Reference Clock Pre-Shared Key QoS Class Indicator Quality of Service Registration Authority Radio Access Network Radio Resource Management Remote User Information Management S1 Application Protocol (3GPP TS36.413) Security Association Database System Architecture Evolution Gateway Stream Control Transmission Protocol Security Gateway Service Level Agreement Security Policy Database Synchronous Ethernet (ITU-T G.8262) Transport admission control Transport Network Layer Measurement-based TAC Timing-over-Packet (IEEE1588-2008) User Network Interface Virtual Private Network Virtual Router Redundancy Protocol Weighted Fair Queuing (scheduler) X2 Application Protocol (3GPP TS36.423) © 2015 Nokia Networks DN09224449 Issue 01B Glossary 73/157 7 74 /157 Appendix A © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A. Appendix A – FZC North Bound Site Solutions This appendix contains FZC example configurations on the NB towards the operator core to support various site solution options with the Controller. This section will cover the cases with and without IPSec and catalog the configurations. As new configurations are supported they will be added to this section. © 2015 Nokia Networks DN09224449 Issue 01B 75/157 A.1 Separate Physical Interfaces with Separation of Application Addresses with IPSec – Separate C/M/U A.1.1 Overview: This configuration describes the option supported on the FZC where the interface addr and application IP address are split. The Interface addr are exposed towards the operator core terminating at the operator SEG and the C/M/U application level addresses are sent within the IPSec tunnel. User Plane uses a 10G interface (SFP-12). Control and Management Planes share the same IP address and physical 1G interface (SFP-13) Two IPSEC Tunnels User Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 100.64.17.7 Control and Management SeGw endpoint: 10.64.0.18 FZC endpoint: 100.64.17.8 Security GW To SGW To MME To iOMS 100.64.0.18 Operator Transport Network Vlan10 100.64.17.1 Vlan10 100.64.17.7 SFP‐12 SFP‐13 User Plane 100.64.48.72 Vlan10 100.64.17.8 Control Plane 100.64.80.72 Management Plane 100.64.112.72 Flexi Zone Controller 76 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.1.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.1.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.1.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.1.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.1.2.4 Check to make sure that the interfaces and addresses are correctly deleted show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B 77/157 A.1.3 Example Network Addresses and Configuration A.1.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 95.36.65.192 95.36.67.192 SGW IP 95.36.65.193 95.36.67.193 PKI/CMP Server 100.64.0.78 -- Sec Gateway 100.64.0.18 -- A.1.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp13 FCPU fcpuvlan 10 100.64.17.8 24 100.64.17.1 sfp12 FZBU fzbuvlan 10 100.64.17.7 24 100.64.17.1 Sfp11 FZBU sbzplane ANY 192.168.55.1 24 -- A.1.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw -- FCPU nbmplane 4001 100.64.112.72 28 -- -- FCPU nbcplane 4002 100.64.80.72 28 -- -- FZBU nbuplane 4003 100.64.48.72 28 -- A.1.4 Add Network configuration for NB on FZC A.1.4.1 Create the internal application level interfaces – nbmplane, nbcplane, nbuplane towards the north bound operator core and assign IP addresses add networking vlan /FCPU-0 realiface xaui0 vid 4001 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface xaui0 vid 4002 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface xaui0 vid 4003 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 100.64.112.72/28 add networking address /FCPU-0 iface nbcplane ip-address 100.64.80.72/28 add networking address /FZBU-0 iface nbuplane ip-address 100.64.48.72/28 78 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport A.1.4.2 Appendix A – FZC North Bound Site Solutions Create the external interface facing the secure gateway that terminate the VPN tunnel(s) on the FZC and assign (VLANs and) IP addresses add networking ether /FCPU-0 iface fcpuipsectun port sfp13 admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault add networking ether /FZBU-0 iface fzbuipsectun port sfp12 admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface fcpuipsectun vid 10 vlaniface fcpuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface fzbuipsectun vid 10 vlaniface fzbuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface fcpuvlan ip-address 100.64.17.8/24 add networking address /FZBU-0 iface fzbuvlan ip-address 100.64.17.7/24 A.1.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 static-route 100.64.0.18/32 nexthop gateway address 100.64.17.1 priority 1 on set routing instance default node FZBU-0 static-route 100.64.0.18/32 nexthop gateway address 100.64.17.1 priority 1 on A.1.4.4 Add routes to reach the various peers in the network set routing instance default node address 100.64.17.1 priority 1 set routing instance default node address 100.64.17.1 priority 1 FCPU-0 static-route default nexthop gateway on FZBU-0 static-route default nexthop gateway on A.1.5 IPSec configuration A.1.5.1 Create the IPSec routed policies with the Interface IP configured as the local tunnel endpoint and SEG as the remote tunnel endpoint add fzc-ipsec nb protect mplane remote-tep 100.64.0.18 local-tep 100.64.17.8 policy-type ip add fzc-ipsec nb protect cplane remote-tep 100.64.0.18 local-tep 100.64.17.8 policy-type ip add fzc-ipsec nb protect uplane remote-tep 100.64.0.18 local-tep 100.64.17.7 policy-type ip © 2015 Nokia Networks DN09224449 Issue 01B 79/157 A.2 Collapsed Physical Interface with Separation of Application Addresses with IPSec – Separate C/M/U A.2.1 Overview: This configuration describes the option supported on the FZC where the there’s a single external physical SFP interface exposed towards the operator core while the interface and application IP address are split. The Interface addresses are exposed towards the operator core terminating at the operator SEG and the C/M/U application level addresses are sent within the IPSec tunnel. This configuration allows a single Ethernet cable to connect to the controller NB that carries different planes of traffic (in different VLANs) but eliminates the need of an external aggregation switch. NB U/M/C Plane, share the same a 10G interface (SFP-12). Two IPSEC Tunnels 80 /157 User Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 100.64.17.7 Control and Management SeGw endpoint: 10.64.0.18 FZC endpoint: 100.64.17.8 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions Security GW To MME To SGW To iOMS 100.64.0.18 Operator Transport Network Vlan10 100.64.17.1 100.64.17.7 100.64.17.8 SFP‐12 User Plane 100.64.48.72 Control Plane 100.64.80.72 Management Plane 100.64.112.72 Flexi Zone Controller A.2.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.2.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.2.2.2 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 © 2015 Nokia Networks DN09224449 Issue 01B 81/157 A.2.2.3 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.2.2.4 Check to make sure that the interfaces and addresses are correctly deleted show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.2.3 Example Network Addresses and Configuration A.2.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 95.36.65.192 95.36.67.192 SGW IP 95.36.65.193 95.36.67.193 PKI/CMP Server 100.64.0.78 -- Sec Gateway 100.64.0.18 -- A.2.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FCPU fcpuvlan 10 100.64.17.8 24 100.64.17.1 sfp12 FZBU fzbuvlan 10 100.64.17.7 24 100.64.17.1 Sfp11 FZBU sbzplane ANY 192.168.55.1 24 -- A.2.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw -- FCPU nbmplane 4001 100.64.112.72 28 -- -- FCPU nbcplane 4002 100.64.80.72 28 -- -- FZBU nbuplane 4003 100.64.48.72 28 -- 82 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.2.4 Add Network configuration for NB on FZC A.2.4.1 Create the internal application level interfaces – nbmplane, nbcplane, nbuplane towards the north bound operator core and assign IP addresses add networking vlan /FCPU-0 realiface xaui0 vid 4001 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface xaui0 vid 4002 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface xaui0 vid 4003 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 100.64.112.72/28 add networking address /FCPU-0 iface nbcplane ip-address 100.64.80.72/28 add networking address /FZBU-0 iface nbuplane ip-address 100.64.48.72/28 A.2.4.2 Create the external interface facing the secure gateway that terminate the VPN tunnel(s) on the FZC and assign (VLANs and) IP addresses add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 add networking ether /FCPU-0 iface fcpuipsectun port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault add networking ether /FZBU-0 iface fzbuipsectun port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface fcpuipsectun vid 10 vlaniface fcpuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface fzbuipsectun vid 10 vlaniface fzbuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface fcpuvlan ip-address 100.64.17.8/24 add networking address /FZBU-0 iface fzbuvlan ip-address 100.64.17.7/24 A.2.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 static-route 100.64.0.18/32 nexthop gateway address 100.64.17.1 priority 1 on set routing instance default node FZBU-0 static-route 100.64.0.18/32 nexthop gateway address 100.64.17.1 priority 1 on A.2.4.4 Add routes to reach the various peers in the network set routing instance default node address 100.64.17.1 priority 1 set routing instance default node address 100.64.17.1 priority 1 FCPU-0 static-route default nexthop gateway on FZBU-0 static-route default nexthop gateway on © 2015 Nokia Networks DN09224449 Issue 01B 83/157 A.2.5 IPSec configuration A.2.5.1 Create the IPSec routed policies with the Interface IP configured as the local tunnel endpoint and SEG as the remote tunnel endpoint add fzc-ipsec nb protect mplane remote-tep 100.64.0.18 local-tep 100.64.17.8 policy-type ip add fzc-ipsec nb protect cplane remote-tep 100.64.0.18 local-tep 100.64.17.8 policy-type ip add fzc-ipsec nb protect uplane remote-tep 100.64.0.18 local-tep 100.64.17.7 policy-type ip 84 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.3 Collapsed Physical Interface and Separate Application Addresses without IPSec – Separate C/M/U addresses A.3.1 Overview: This configuration describes the option supported on the FZC where a single external physical interface is exposed and the interface addr and application IP address are split. 2 Interface addreses are exposed towards the operator core terminating at the operator L3 router/gateway and the C/M/U application level addresses are routed via the interface addresses. Single external physical 10G interface (SFP-12) towards EPC (c/m/u). All application addresses are unique All traffic is routed via the external interface IPs User plane traffic forwarded via next hop router – 10.94.0.3 Control/Management traffic forwarded via next hop router 10.94.0.2 To SGW To MME To iOMS Operator Transport Network Router 10.94.0.1/28 Vlan101 10.94.0.3/28 Vlan101 10.94.0.2/28 SFP‐12 fzbuvlan nbuplane 10.94.0.255 fcpuvlan nbcplane 10.94.0.233 nbmplane 10.94.0.241 Flexi Zone Controller © 2015 Nokia Networks DN09224449 Issue 01B 85/157 A.3.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.3.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.3.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.3.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.3.2.4 Check to make sure that the interfaces and addresses are correctly deleted 86 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.3.3 Example Network Addresses and Configuration A.3.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 100.100.100.100 100.100.200.100 SGW IP 100.100.100.101 100.100.200.101 A.3.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FCPU fcpuvlan 101 10.94.0.2 28 10.94.0.1 sfp12 FZBU fzbuvlan 101 10.94.0.3 28 10.94.0.1 VID IP addr Subnet Router/Gw A.3.3.3 Application Interfaces SFP# Node Iface Name -- FCPU nbmplane 4001 10.94.0.241 32 -- -- FCPU nbcplane 4002 10.94.0.233 32 -- -- FZBU nbuplane 4003 10.94.0.255 32 -- A.3.4 Add Network configuration for NB on FZC A.3.4.1 Create the internal application level interfaces – nbmplane, nbcplane, nbuplane towards the north bound operator core and assign IP addresses add networking vlan /FCPU-0 realiface xaui0 vid 4001 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.94.0.241/32 add networking aclrule /FCPU-0 index 1234 chain input addr-family ipv4 dstaddr 10.94.0.241 accept** add networking vlan /FCPU-0 realiface xaui0 vid 4002 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbcplane ip-address 10.94.0.233/32 add networking aclrule /FCPU-0 index 1235 chain input addr-family ipv4 dstaddr 10.94.0.233 accept** add networking vlan /FZBU-0 realiface xaui0 vid 4003 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FZBU-0 iface nbuplane ip-address 10.94.0.255/32 add networking aclrule /FCPU-0 index 1236 chain input addr-family ipv4 dstaddr 10.94.0.255 accept** © 2015 Nokia Networks DN09224449 Issue 01B 87/157 A.3.4.2 Create single SFP interface for all planes of traffic: add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 A.3.4.3 Create the external interface facing the secure gateway that terminate the VPN tunnel(s) on the FZC and assign (VLANs and) IP addresses add networking ether /FCPU-0 iface fcpuiface port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault rate-limit 1G add networking ether /FZBU-0 iface fzbuiface port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface fcpuiface vid 101 vlaniface fcpuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface fzbuiface vid 101 vlaniface fzbuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface fcpuvlan ip-address 10.94.0.2/28 add networking address /FZBU-0 iface fzbuvlan ip-address 10.94.0.3/28 A.3.4.4 Add the routes on the box for egress packets so that they go via the default external router set routing instance address 10.94.0.1 set routing instance address 10.94.0.1 88 /157 default node FCPU-0 static-route default nexthop gateway priority 1 on default node FZBU-0 static-route default nexthop gateway priority 1 on © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.4 Collapsed Physical Interface and Separate Application Addresses with IPSec – Separate C/M/U addresses A.4.1 Overview: This configuration describes the option supported on the FZC where a single external physical interface is exposed and the interface and application IP address are split. 2 Interface addreses are exposed towards the operator core (in the same VLAN/subnet) acting as the VPN local tunnel endpoints with the IPSec tunnel terminating at the operator SEG. The C/M/U application level addresses are tunneled inside IPSec tunnel and then terminated behind the operator SEG. Single external physical 10G interface (SFP-12) towards EPC (c/m/u). All application addresses are unique Two IPSEC Tunnels User Plane local Tunnel Endpoint SeGw endpoint: 10.94.0.100 FZC FZBU endpoint: 10.94.0.3 Control/Management local Tunnel Endpoint SeGw endpoint: 10.94.0.100 FZC FCPU endpoint: 10.94.0.2 © 2015 Nokia Networks DN09224449 Issue 01B 89/157 A.4.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.4.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.4.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.4.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.4.2.4 Check to make sure that the interfaces and addresses are correctly deleted 90 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.4.3 Example Network Addresses and Configuration A.4.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 100.100.100.100 100.100.200.100 SGW IP 100.100.100.101 100.100.200.101 PKI/CMP Server 10.94.0.50 -- Sec Gateway 10.94.0.100 -- A.4.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FCPU fcpuvlan 101 10.94.0.2 28 10.94.0.1 sfp12 FZBU fzbuvlan 101 10.94.0.3 28 10.94.0.1 A.4.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw -- FCPU nbmplane 4001 10.94.0.241 32 -- -- FCPU nbcplane 4002 10.94.0.233 32 -- -- FZBU nbuplane 4003 10.94.0.255 32 -- A.4.4 Add Network configuration for NB on FZC A.4.4.1 Create the internal application level interfaces – nbmplane, nbcplane, nbuplane towards the north bound operator core and assign IP addresses add networking vlan /FCPU-0 realiface xaui0 vid 4001 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface xaui0 vid 4002 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface xaui0 vid 4003 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.94.0.241/32 add networking address /FCPU-0 iface nbcplane ip-address 10.94.0.233/32 add networking address /FZBU-0 iface nbuplane ip-address 10.94.0.255/32 A.4.4.2 Create the external interface facing the secure gateway that terminate the VPN tunnel(s) on the FZC and assign (VLANs and) IP addresses add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 © 2015 Nokia Networks DN09224449 Issue 01B 91/157 add networking ether /FCPU-0 iface fcpuipsectun port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault rate-limit 1G add networking ether /FZBU-0 iface fzbuipsectun port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface fcpuipsectun vid 101 vlaniface fcpuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface fzbuipsectun vid 101 vlaniface fzbuvlan admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface fcpuvlan ip-address 10.94.0.2/28 add networking address /FZBU-0 iface fzbuvlan ip-address 10.94.0.3/28 A.4.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default gateway address 10.94.0.1 set routing instance default gateway address 10.94.0.1 A.4.4.4 node FCPU-0 static-route 10.94.0.100/32 nexthop priority 1 on node FZBU-0 static-route 10.94.0.100/32 nexthop priority 1 on Add routes to reach the various peers in the network set routing instance address 10.94.0.1 set routing instance address 10.94.0.1 default node FCPU-0 static-route default nexthop gateway priority 1 on default node FZBU-0 static-route default nexthop gateway priority 1 on A.4.5 IPSec configuration using FZC wrappers A.4.5.1 Create the IPSec routed policies with the Interface IP configured as the local tunnel endpoint and SEG as the remote tunnel endpoint add fzc-ipsec nb protect mplane remote-tep 10.94.0.100 local-tep 10.94.0.2 policy-type ip add fzc-ipsec nb protect cplane remote-tep 10.94.0.100 local-tep 10.94.0.2 policy-type ip add fzc-ipsec nb protect uplane remote-tep 10.94.0.100 local-tep 10.94.0.3 policy-type ip 92 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.5 Separate Physical Interfaces and Application Addresses without IPSec – M-plane Address behind Cplane A.5.1 Overview: This configuration describes the option supported on the FZC where a 2 external physical interfaces are exposed towards the operator core network. The interface and application IP address are split such that the M-plane address is ‘behind’ the C-plane address (which is also the external interface address). All M-plane traffic is routed via the C-plane external interface address (including the L2 VLAN – if tagging is enabled). U-plane and C-plane interface addresses are in the same VLAN/L3 subnet talking to the operator L3 router/gateway. User, Control, and Sync Plane traffic over a single VLAN interface Separate SFP physical interface for C and M plane Application Addresses User Plane 42.255.131.21/29 Control Plane 42.255.131.20/29 Management Plane 11.255.131.20/32 Synchronization Plane Not Applicable Operator Core Network To SGW 10.160.73.1 10.169.56.100 To 10.160.71.44 MME To 5.232.101.133 iOMS 5.232.101.135 Router 42.255.131.17 SFP‐12 VLAN 3049 User Plane 42.255.131.21 SFP-14 VLAN 3049 Control Plane 42.255.131.20 Mgmt Plane 11.255.131.20 Flexi Zone Controller FZAP1 FZAP2 FZAP3 © 2015 Nokia Networks DN09224449 Issue 01B FZAP4 FZAP100 93/157 A.5.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.5.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.5.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.5.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.5.2.4 Check to make sure that the interfaces and addresses are correctly deleted 94 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.5.3 Network Addresses and Configuration A.5.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.169.56.100 10.160.71.44 SGW IP 10.160.73.1 -- iOMS IP 5.232.101.133 5.232.101.135 PKI/CMP Server -- -- Sec Gateway -- -- A.5.3.2 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw SFP13 FCPU nbmplane -- 11.255.131.20 32 -- -- FCPU nbcplane 3049 42.255.131.20 29 -- SFP12 FZBU nbuplane 3049 42.255.131.21 29 -- A.5.4 Add Network configuration for NB on FZC A.5.4.1 Add NB M-plane interface and IP address: add networking vlan /FCPU-0 realiface xaui0 vid 4001 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 11.255.131.20/32 add networking aclrule /FCPU-0 index 1234 chain input addr-family ipv4 dstaddr 11.255.131.20 accept A.5.4.2 Add NB C-plane interface. VLAN, and IP address: add networking ether /FCPU-0 iface nbcplane_novlan port sfp14 admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault add networking vlan /FCPU-0 realiface nbcplane_novlan vid 3049 vlaniface nbcplane add networking address /FCPU-0 iface nbcplane ip-address 42.255.131.20/29 © 2015 Nokia Networks DN09224449 Issue 01B 95/157 A.5.4.3 Add NB U-plane interface. VLAN, and IP address: add networking ether /FZBU-0 iface nbuplane_novlan port sfp12 admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault add networking vlan /FZBU-0 realiface nbuplane_novlan vid 3049 vlaniface nbuplane add networking address /FZBU-0 iface nbuplane ip-address 42.255.131.21/29 A.5.4.4 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node address 42.255.131.17 priority set routing instance default node address 42.255.131.17 priority 96 /157 FCPU-0 static-route default nexthop gateway 1 on FZBU-0 static-route default nexthop gateway 1 on © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.6 Collapsed Physical Interface with Separate Application Addresses without IPSec – M-plane Address behind Cplane A.6.1 Overview: This configuration describes the option supported on the FZC where a single external physical interface is exposed towards the operator core network. The interface and application IP address are split such that the M-plane address is ‘behind’ the C-plane address (which is also the external interface address). All M-plane traffic is routed via the Cplane external interface address (including the L2 VLAN – if tagging is enabled). U-plane and C-plane interface addresses are in the same VLAN/L3 subnet talking to the operator L3 router/gateway. This configuration allows a single Ethernet cable to connect to the controller NB that carries different planes of traffic (in different VLANs) but eliminates the need of an external aggregation switch. User, Control, and Sync Plane traffic over a single VLAN interface Single SFP physical interface for C and M plane Application Addresses User Plane 42.255.131.21/29 Control Plane 42.255.131.20/29 Management Plane 11.255.131.20/32 Synchronization Plane Not Applicable © 2015 Nokia Networks DN09224449 Issue 01B 97/157 Operator Core Network To SGW 10.160.73.1 10.169.56.100 To 10.160.71.44 MME To 5.232.101.133 iOMS 5.232.101.135 Router 42.255.131.17 SFP‐12 VLAN 3049 User Plane 42.255.131.21 Control Plane 42.255.131.20 Mgmt Plane Flexi Zone Controller11.255.131.20 FZAP1 FZAP2 FZAP3 FZAP4 FZAP100 A.6.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.6.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.6.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces 98 /157 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport A.6.2.3 Appendix A – FZC North Bound Site Solutions delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.6.2.4 Check to make sure that the interfaces and addresses are correctly deleted show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.6.3 Network Addresses and Configuration A.6.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.169.56.100 10.160.71.44 SGW IP 10.160.73.1 -- iOMS IP 5.232.101.133 5.232.101.135 PKI/CMP Server -- -- Sec Gateway -- -- A.6.3.2 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw SFP13 FCPU nbmplane -- 11.255.131.20 32 -- -- FCPU nbcplane 3049 42.255.131.20 29 -- SFP12 FZBU nbuplane 3049 42.255.131.21 29 -- © 2015 Nokia Networks DN09224449 Issue 01B 99/157 A.6.4 Add Network configuration for NB on FZC A.6.4.1 Add NB M-plane interface and IP address: add networking vlan /FCPU-0 realiface xaui0 vid 4001 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 11.255.131.20/32 add networking aclrule /FCPU-0 index 1234 chain input addr-family ipv4 dstaddr 11.255.131.20 accept A.6.4.2 Create single SFP interface for all planes of traffic: add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 A.6.4.3 Add NB C-plane interface. VLAN, and IP address: add networking ether /FCPU-0 iface nbcplane_novlan port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault add networking vlan /FCPU-0 realiface nbcplane_novlan vid 3049 vlaniface nbcplane add networking address /FCPU-0 iface nbcplane ip-address 42.255.131.20/29 A.6.4.4 Add NB U-plane interface. VLAN, and IP address: add networking ether /FZBU-0 iface nbuplane_novlan port fzcnb admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no outqset fzcdefault add networking vlan /FZBU-0 realiface nbuplane_novlan vid 3049 vlaniface nbuplane add networking address /FZBU-0 iface nbuplane ip-address 42.255.131.21/29 A.6.4.5 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node address 42.255.131.17 priority set routing instance default node address 42.255.131.17 priority 100 /157 FCPU-0 static-route default nexthop gateway 1 on FZBU-0 static-route default nexthop gateway 1 on © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.7 Separate Physical Interfaces with shared interface and application addresses without IPSec – Combined C/M and Separate U-plane A.7.1 Overview: This configuration describes the option supported on the FZC where separate external physical interfaces are exposed towards the operator core network. The interface and host addresses are the same. C/M plane IPs are the same while the U-plane is separate. VLANs are optional. C/M is on physical interface SFP13 U plane is on physical interface SFP12 C/M planes share same VLAN and IP addr U plane has a unique VLAN and IP The use of vlans (101 and 201 in this example) is optional © 2015 Nokia Networks DN09224449 Issue 01B 101/157 A.7.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.7.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.7.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.7.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.7.2.4 Check to make sure that the interfaces and addresses are correctly deleted 102 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.7.3 Network Addresses and Configuration A.7.3.1 External Core/EPC NEs Core Component IP-1 MME IP 10.10.40.20 SGW IP 10.10.30.2 iOMS IP 10.10.40.10 A.7.3.2 IP-2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.7.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.7.4 Add Network configuration for NB on FZC A.7.4.1 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port sfp13 iface nbmplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G add networking instance default ether /FZBU-0 port sfp12 iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no © 2015 Nokia Networks DN09224449 Issue 01B 103/157 add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.7.4.2 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 static-route default nexthop gateway address 10.10.40.1 priority 1 on set routing instance default node FZBU-0 static-route default nexthop gateway address 10.10.30.1 priority 1 on 104 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.8 Collapsed Physical Interface with shared interface and application addresses without IPSec – Combined C/M and Separate U-plane A.8.1 Overview: This configuration describes the option supported on the FZC where a collased external physical interface is exposed towards the operator core network. The interface and host addresses are the same. C/M plane IPs are the same while the U-plane is separate. VLANs are optional. All planes C/M/U share the same physical interface (SFP-12). C/M planes share same VLAN and IP addr U plane has a unique VLAN and IP The use of vlans (101 and 201 in this example) is optional © 2015 Nokia Networks DN09224449 Issue 01B 105/157 A.8.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.8.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.8.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.8.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.8.2.4 Check to make sure that the interfaces and addresses are correctly deleted 106 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.8.3 Network Addresses and Configuration A.8.3.1 External Core/EPC NEs Core Component IP-1 MME IP 10.10.40.20 SGW IP 10.10.30.2 iOMS IP 10.10.40.10 A.8.3.2 IP-2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.8.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.8.4 Add Network configuration for NB on FZC A.8.4.1 Create single SFP interface for all planes of traffic: add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 A.8.4.2 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port fzcnb iface nbmplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G add networking instance default ether /FZBU-0 port fzcnb iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no © 2015 Nokia Networks DN09224449 Issue 01B 107/157 add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.8.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 static-route default nexthop gateway address 10.10.40.1 priority 1 on set routing instance default node FZBU-0 static-route default nexthop gateway address 10.10.30.1 priority 1 on 108 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.9 Separate Physical Interfaces with shared interface and application addresses with IPSec – Combined C/M and Separate U-plane A.9.1 Overview: This configuration describes the option supported on the FZC where separate external physical interfaces are exposed towards the operator core network. The interface and host addresses are the same. C/M plane IPs are the same while the U-plane is separate. VLANs are optional. IPSec tunnels are terminated between the FZC and the SEG. C/M is on physical interface SFP13 U plane is on physical interface SFP12 C/M planes share same VLAN and IP addr U plane has a unique VLAN and IP The use of vlans (101 and 201 in this example) is optional IPSEC tunnels use FZC interface/application addresses as endpoints Two IPSEC Tunnels User Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.30.2 Router/Gw 10.10.30.1 Control and Management SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.40.2 Router/Gw 10.10.40.1 © 2015 Nokia Networks DN09224449 Issue 01B 109/157 110 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.9.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.9.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) A.9.2.2 networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane Delete the IP addresses associated with the interfaces A.9.2.3 show show show show delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.9.2.4 Check to make sure that the interfaces and addresses are correctly deleted show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B 111/157 A.9.3 Network Addresses and Configuration A.9.3.1 External Core/EPC NEs Core Component IP-1 MME IP 10.10.40.20 SGW IP 10.10.30.20 iOMS IP 10.10.40.10 Sec Gateway 10.64.0.18 A.9.3.2 IP-2 -- Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.9.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.9.4 Add Network configuration for NB on FZC A.9.4.1 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port sfp13 iface nbmplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G add networking instance default ether /FZBU-0 port sfp12 iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 112 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.9.4.2 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 static-route default nexthop gateway address 10.10.40.1 priority 1 on set routing instance default node FZBU-0 static-route default nexthop gateway address 10.10.30.1 priority 1 on A.9.4.3 Create the IPSec policies to protect the different planes of traffic add fzc-ipsec nb protect mplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect cplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect uplane remote-tep 10.64.0.18 policy-type ip © 2015 Nokia Networks DN09224449 Issue 01B 113/157 A.10 Collapsed Physical Interface with shared interface and application addresses with IPSec – Combined C/M and Separate U-plane A.10.1 Overview: This configuration describes the option supported on the FZC where a collapsed/single external physical interface is exposed towards the operator core network. The interface and host addresses are the same. C/M plane IPs are the same while the U-plane is separate. VLANs are optional. IPSec tunnels are terminated between the FZC and the SEG. C/M is on physical interface SFP13 U plane is on physical interface SFP12 C/M planes share same VLAN and IP addr U plane has a unique VLAN and IP The use of vlans (101 and 201 in this example) is optional IPSEC tunnels use FZC interface/application addresses as endpoints Two IPSEC Tunnels 114 /157 User Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.30.2 Router/Gw 10.10.30.1 Control and Management SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.40.2 Router/Gw 10.10.40.1 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport © 2015 Nokia Networks DN09224449 Issue 01B Appendix A – FZC North Bound Site Solutions 115/157 A.10.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.10.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.10.2.2 Delete the IP addresses associated with the interfaces delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 A.10.2.3 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.10.2.4 Check to make sure that the interfaces and addresses are correctly deleted 116 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.10.3 Network Addresses and Configuration A.10.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.10.40.20 SGW IP 10.10.30.20 iOMS IP 10.10.40.10 Sec Gateway 10.64.0.18 -- A.10.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.10.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 A.10.4 Add Network configuration for NB on FZC A.10.4.1 Create single SFP interface for all planes of traffic: add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 A.10.4.2 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port fzcnb iface nbmplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G © 2015 Nokia Networks DN09224449 Issue 01B 117/157 add networking instance default ether /FZBU-0 port fzcnb iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.10.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 static-route default nexthop gateway address 10.10.40.1 priority 1 on set routing instance default node FZBU-0 static-route default nexthop gateway address 10.10.30.1 priority 1 on A.10.4.4 Create the IPSec policies to protect the different planes of traffic add fzc-ipsec nb protect mplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect cplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect uplane remote-tep 10.64.0.18 policy-type ip 118 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.11 Separate Physical Interfaces with shared interface and application addresses without IPSec – Separate C/M/U planes A.11.1 Overview: This configuration describes the option supported on the FZC where separate external physical interfaces are exposed towards the operator core network. The interface and host addresses are the same. All application addresses are separate C/M/U. VLANs are optional. C and M are on separate physical interfaces SFP13 & SFP14 U plane is on physical interface SFP12 C-plane has a unique VLAN and IP M-plane has a unique VLAN and IP U-plane has a unique VLAN and IP The use of vlans (101, 201, and 301 in this example) is optional © 2015 Nokia Networks DN09224449 Issue 01B 119/157 A.11.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.11.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.11.2.2 Delete the IP addresses associated with the interfaces delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 A.11.2.3 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.11.2.4 Check to make sure that the interfaces and addresses are correctly deleted 120 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.11.3 Network Addresses and Configuration A.11.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.10.40.20 SGW IP 10.10.30.2 iOMS IP 10.10.20.10 A.11.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp14 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.11.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp14 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.11.4 Add Network configuration for NB on FZC A.11.4.1 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port sfp13 admin up ipv4forwarding yes ipv4rpfilter no proxy-arp fzcdefault rate-limit 1G add networking instance default ether /FCPU-0 port sfp14 admin up ipv4forwarding yes ipv4rpfilter no proxy-arp fzcdefault rate-limit 1G © 2015 Nokia Networks DN09224449 Issue 01B iface nbmplane_novlan no mtu 1500 outqset iface nbcplane_novlan no mtu 1500 outqset 121/157 add networking instance default ether /FZBU-0 port sfp12 iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface nbcplane_novlan vid 301 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FCPU-0 iface nbmplane ip-address 10.10.20.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.11.4.2 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 gateway address 10.10.20.1 priority 1 set routing instance default node FZBU-0 gateway address 10.10.30.1 priority 1 set routing instance default node FCPU-0 gateway address 10.10.40.1 priority 1 122 /157 static-route 10.10.20.0/24 nexthop on static-route 10.10.30.0/24 nexthop on static-route 10.10.40.0/24 nexthop on © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.12 Collapsed Physical Interface with shared interface and application addresses without IPSec – Separate C/M/U planes A.12.1 Overview: This configuration describes the option supported on the FZC where a collapsed external physical interface is exposed towards the operator core network. The interface and host addresses are the same. All application addresses are separate C/M/U. VLANs are optional. C/M/U are on same physical interface SFP12 C-plane has a unique VLAN and IP M-plane has a unique VLAN and IP U-plane has a unique VLAN and IP The use of vlans (101, 201, and 301 in this example) is optional © 2015 Nokia Networks DN09224449 Issue 01B 123/157 A.12.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.12.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.12.2.2 Delete the IP addresses associated with the interfaces delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 A.12.2.3 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.12.2.4 Check to make sure that the interfaces and addresses are correctly deleted 124 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.12.3 Network Addresses and Configuration A.12.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.10.40.20 SGW IP 10.10.30.2 iOMS IP 10.10.20.10 A.12.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp12 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.12.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp12 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.12.4 Add Network configuration for NB on FZC A.12.4.1 Create single SFP interface for all planes of traffic: add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 A.12.4.2 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port fzcnb iface nbmplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G © 2015 Nokia Networks DN09224449 Issue 01B 125/157 add networking instance default ether /FCPU-0 port fzcnb iface nbcplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G add networking instance default ether /FZBU-0 port fzcnb iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface nbcplane_novlan vid 301 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FCPU-0 iface nbmplane ip-address 10.10.20.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.12.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 gateway address 10.10.20.1 priority 1 set routing instance default node FZBU-0 gateway address 10.10.30.1 priority 1 set routing instance default node FCPU-0 gateway address 10.10.40.1 priority 1 126 /157 static-route 10.10.20.0/24 nexthop on static-route 10.10.30.0/24 nexthop on static-route 10.10.40.0/24 nexthop on © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.13 Separate Physical Interfaces with shared interface and application addresses with IPSec – Separate C/M/U planes A.13.1 Overview: This configuration describes the option supported on the FZC where separate external physical interfaces are exposed towards the operator core network. The interface and host addresses are the same. All application addresses are separate C/M/U. VLANs are optional. IPSec tunnels are terminated between the FZC and the SEG. M plane is on physical interface SFP13 U plane is on physical interface SFP12 C plane is on physical interface SFP14 All planes (C/M/U) have a unique VLAN and IP The use of vlans (101, 201, and 301 in this example) is optional IPSEC tunnels use FZC interface/application addresses as endpoints Three IPSEC Tunnels User Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.30.2 Router/Gw 10.10.30.1 Management Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.40.2 Router/Gw 10.10.40.1 Control Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.20.2 Router/Gw 10.10.20.1 © 2015 Nokia Networks DN09224449 Issue 01B 127/157 128 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.13.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.13.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.13.2.2 Delete the IP addresses associated with the interfaces delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 A.13.2.3 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.13.2.4 Check to make sure that the interfaces and addresses are correctly deleted show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B 129/157 A.13.3 Network Addresses and Configuration A.13.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.10.20.20 SGW IP 10.10.30.20 iOMS IP 10.10.40.10 Sec Gateway 10.64.0.18 -- A.13.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp14 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.13.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp13 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp14 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.13.4 Add Network configuration for NB on FZC A.13.4.1 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port sfp13 admin up ipv4forwarding yes ipv4rpfilter no proxy-arp fzcdefault rate-limit 1G add networking instance default ether /FCPU-0 port sfp14 admin up ipv4forwarding yes ipv4rpfilter no proxy-arp fzcdefault rate-limit 1G add networking instance default ether /FZBU-0 port sfp12 admin up ipv4forwarding yes ipv4rpfilter no proxy-arp fzcdefault rate-limit 4G 130 /157 iface nbmplane_novlan no mtu 1500 outqset iface nbcplane_novlan no mtu 1500 outqset iface nbuplane_novlan no mtu 1500 outqset © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface nbcplane_novlan vid 301 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FCPU-0 iface nbcplane ip-address 10.10.20.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.13.4.2 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 gateway address 10.10.20.1 priority 1 set routing instance default node FZBU-0 gateway address 10.10.30.1 priority 1 set routing instance default node FCPU-0 gateway address 10.10.40.1 priority 1 static-route 10.10.20.0/24 nexthop on static-route 10.10.30.0/24 nexthop on static-route 10.10.40.0/24 nexthop on A.13.4.3 Create the IPSec policies to protect the different planes of traffic add fzc-ipsec nb protect mplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect cplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect uplane remote-tep 10.64.0.18 policy-type ip © 2015 Nokia Networks DN09224449 Issue 01B 131/157 A.14 Collapsed Physical Interfaces with shared interface and application addresses with IPSec – Separate C/M/U planes A.14.1 Overview: This configuration describes the option supported on the FZC where a collapsed/single external physical interface is exposed towards the operator core network. The interface and host addresses are the same. All application addresses are separate C/M/U. VLANs are optional. IPSec tunnels are terminated between the FZC and the SEG. M/C/U planes are on the same physical interface SFP12 All planes (C/M/U) have a unique VLAN and IP The use of vlans (101, 201, and 301 in this example) is optional IPSEC tunnels use FZC interface/application addresses as endpoints Three IPSEC Tunnels 132 /157 User Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.30.2 Router/Gw 10.10.30.1 Management Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.40.2 Router/Gw 10.10.40.1 Control Plane SeGw endpoint: 10.64.0.18 FZC endpoint: 10.10.20.2 Router/Gw 10.10.20.1 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport © 2015 Nokia Networks DN09224449 Issue 01B Appendix A – FZC North Bound Site Solutions 133/157 A.14.2 Delete base/default factory configurations and IP addresses • ASSUMPTION: The flexizone controller (FZC) box is configured with the default interfaces including VLANs and IP addresses on the Nothbound NB C/M plane interface is combined ‘nbmplane’ configured on FCPU node – IP addr = 10.10.40.1/24 (and no VLAN) NB U plane interface ‘nbuplane’ configured on FZBUnode – IP addr = 10.10.30.1/24 (and no VLAN) The below commands are to be executed on the SCLI of the FZC: A.14.2.1 Show the existing interfaces to make sure they’re configured with above addresses (if not then the below steps to delete will need to be altered) show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane A.14.2.2 Delete the IP addresses associated with the interfaces delete networking address /FCPU-0 iface nbmplane ip-address 10.10.40.1 delete networking address /FZBU-0 iface nbuplane ip-address 10.10.30.1 A.14.2.3 Delete the interfaces delete networking ether /FCPU-0 iface nbmplane delete networking ether /FZBU-0 iface nbuplane A.14.2.4 Check to make sure that the interfaces and addresses are correctly deleted 134 /157 show show show show networking networking networking networking interface iface nbmplane address iface nbmplane interface iface nbuplane address iface nbuplane © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix A – FZC North Bound Site Solutions A.14.3 Network Addresses and Configuration A.14.3.1 External Core/EPC NEs Core Component IP-1 IP-2 MME IP 10.10.20.20 SGW IP 10.10.30.20 iOMS IP 10.10.40.10 Sec Gateway 10.64.0.18 -- A.14.3.2 Physical Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp12 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.14.3.3 Application Interfaces SFP# Node Iface Name VID IP addr Subnet Router/Gw sfp12 FZBU nbuplane 101 10.10.30.2 24 10.10.30.1 sfp12 FCPU nbmplane 201 10.10.40.2 24 10.10.40.1 sfp12 FCPU nbcplane 301 10.10.20.2 24 10.10.20.1 A.14.4 Add Network configuration for NB on FZC A.14.4.1 Create single SFP interface for all planes of traffic: add networking switch port-group name fzcnb port LMP-1-1-1:sfp12 © 2015 Nokia Networks DN09224449 Issue 01B 135/157 A.14.4.2 Create the external interface on the FZC and assign (VLANs and) IP addresses add networking instance default ether /FCPU-0 port fzcnb iface nbmplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G add networking instance default ether /FCPU-0 port fzcnb iface nbcplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 1G add networking instance default ether /FZBU-0 port fzcnb iface nbuplane_novlan admin up ipv4forwarding yes ipv4rpfilter no proxy-arp no mtu 1500 outqset fzcdefault rate-limit 4G add networking vlan /FCPU-0 realiface nbmplane_novlan vid 101 vlaniface nbmplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FCPU-0 realiface nbcplane_novlan vid 301 vlaniface nbcplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking vlan /FZBU-0 realiface nbuplane_novlan vid 201 vlaniface nbuplane admin up ipv4forwarding yes mtu 1500 ipv4rpfilter no proxy-arp no add networking address /FCPU-0 iface nbmplane ip-address 10.10.40.2/24 add networking address /FCPU-0 iface nbcplane ip-address 10.10.20.2/24 add networking address /FZBU-0 iface nbuplane ip-address 10.10.30.2/24 A.14.4.3 Add the routes on the box for egress packets so that they go via the default external router set routing instance default node FCPU-0 gateway address 10.10.20.1 priority 1 set routing instance default node FZBU-0 gateway address 10.10.30.1 priority 1 set routing instance default node FCPU-0 gateway address 10.10.40.1 priority 1 static-route 10.10.20.0/24 nexthop on static-route 10.10.30.0/24 nexthop on static-route 10.10.40.0/24 nexthop on A.14.4.4 Create the IPSec policies to protect the different planes of traffic add fzc-ipsec nb protect mplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect cplane remote-tep 10.64.0.18 policy-type ip add fzc-ipsec nb protect uplane remote-tep 10.64.0.18 policy-type ip 136 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport 8 Appendix B © 2015 Nokia Networks DN09224449 Issue 01B Appendix B 137/157 B. Appendix B – ToP Connectivity Options in Zone This appendix contains zone ToP/1588v2 connectivity and configuration options in Zone for RL15A. In this release the 1588v2/ToP packets bypass the controller and routed to/from the FZAPs directly from/to GMC/BC. Some of the options supported for ToP/1588v2 specific to zone are listed below and then detailed in the subsections of this Appendix. GMC/B C Locatio n Non-Daisy Chain Zone Network Sync Type (Frequency/Phas e) ToP-F NO IP Unicast YES B. 1 ToP-P Core Network Ethernet Multicast ToP-F YES YES B. 2 B. 3 NO YES B. 4 ToP-P NO YES B. 5 138 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix B – ToP Connectivity Options in Zone B.1 TOP-F Sync using IP Unicast with GMC/BC present in Zone Network in Non-Daisy Chain Mode B.1.1 Overview The below configuration captures the network configuration and related topology where the FZAPs in the zone require frequency synchronization using 1588v2/PtP. The Boundary Clock (BC) or GMC (Grand Master Clock) is located in the zone network and configured for ToP using IP unicast/L3 mode. The FZAPs in the zone network are also configured to point to the GMC/BC and configured for ToP-F sync in IP unicast mode. MMEs MME IP Prim MME IP Sec FZAP#1 S1‐MME ToP Master 1GE Router R1 1GE FZAP #2 Flexi Zone Controller L2 1GE Core Network O&M S1‐U SAE‐GW FZAP #3 Access Network Zone Network B.1.1.1 Edge Router Primary & Secondary Core Network O&M Network Network Element Addresses Element IP-1 IP-2 TOP Master 192.168.55.100 -- Zone Network 192.168.55.0/24 N/A © 2015 Nokia Networks DN09224449 Issue 01B 139/157 B.1.2 Configuration via BTS Site Manager B.1.2.1 TOPF-1 Feature Activation Flag ToP with freq synchronization “actTopFreqSynch” => True B.1.2.2 Timing over Packet masters properties table IP Address of the ToP master “masterIpAddr” => <ToP master IP@> B.1.2.3 APMOD-1 External GPS as reference source => false 140 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix B – ToP Connectivity Options in Zone Transmission synchronization (TDM/SyncE/ToP) in use => true 1PPS source line delay => blank/empty B.1.2.4 APMOD-1/APSTPG-1 Protocol of the clock source “clockProtocol” => clkToP B.2 TOP-P Sync using Ethernet Multicast with GMC/BC present in Zone Network in Non-Daisy Chain Mode B.2.1 Overview The below configuration captures the network configuration and related topology where the FZAPs in the zone require phase synchronization using 1588v2/PtP. The Boundary Clock (BC) or GMC (Grand Master Clock) is located in the zone network and configured for ToP © 2015 Nokia Networks DN09224449 Issue 01B 141/157 using Ethernet multicast mode. The FZAPs in the zone network are also configured for ToPP sync in Ethernet multicast mode. MMEs MME IP Prim MME IP Sec FZAP#1 S1‐MME ToP Master 1GE Router R1 1GE 1GE FZAP #2 Flexi Zone Controller L2 Core Network O&M S1‐U SAE‐GW FZAP #3 Access Network Zone Network B.2.1.1 Edge Router Primary & Secondary Core Network Network Element Addresses Element IP-1 IP-2 TOP Master 01-1B-19-00-00-00 (L2 multicast MAC) -- Zone Network 192.168.55.0/24 N/A 142 /157 O&M Network © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix B – ToP Connectivity Options in Zone B.2.2 Configuration via BTS Site Manager B.2.2.1 TOPP-1 Feature Activation Flag ToP with phase synchronization also TOP-P mode is set to Ethernet Multicast and multicast MAC address. B.2.2.2 Accepted Clock Quality Tables 3 tables with different clock quality table © 2015 Nokia Networks DN09224449 Issue 01B 143/157 B.2.2.3 Properties Configured Timing over Packet Master(s) IP address of the ToP Master (GMC/BC) is set to 0.0.0.0 – due to the use of Ethernet Multicast B.2.2.4 APMOD-1 External GPS as reference source => false Transmission synchronization (TDM/SyncE/ToP) in use => true 1PPS source line delay => blank/empty B.2.2.5 APMOD-1/APSTPG-1 Protocol of the clock source “clockProtocol” => clkToP 144 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport © 2015 Nokia Networks DN09224449 Issue 01B Appendix B – ToP Connectivity Options in Zone 145/157 B.3 TOP-P Sync using IP Unicast with GMC/BC present in Zone Network in Non-Daisy Chain Mode B.3.1 Overview The below configuration captures the network configuration and related topology where the FZAPs in the zone require phase synchronization using 1588v2/PtP. The Boundary Clock (BC) or GMC (Grand Master Clock) is located in the zone network and configured for ToP-P using IP unicast/L3 mode. The FZAPs in the zone network are also configured to point to the GMC/BC and configured for ToP-P sync in IP unicast mode. MMEs MME IP Prim MME IP Sec FZAP#1 S1‐MME ToP Master 1GE Router R1 1GE 1GE Controller L2 Edge Router Primary & Secondary Core Network O&M S1‐U FZAP #2 SAE‐GW FZAP #3 Access Network Zone Network B.3.1.1 Core Network O&M Network Network Element Addresses Element IP-1 IP-2 TOP Master 192.168.55.100 -- Zone Network 192.168.55.0/24 N/A B.3.2 Configuration via BTS Site Manager B.3.2.1 TOPP-1 Feature Activation Flag ToP with phase synchronization “actTopPhaseSynch” => True 146 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport B.3.2.2 Acceptable Clock Quality Table B.3.2.3 ToP Masters Property table Appendix B – ToP Connectivity Options in Zone Here you provide the IP address of the TOP-P master GMC/BC to achieve sync from. © 2015 Nokia Networks DN09224449 Issue 01B 147/157 B.3.2.4 APMOD-1/APSTPG-1 Protocol of the clock source “clockProtocol” => clkToP B.3.2.5 APMOD-1 External GPS as reference source => false Transmission synchronization (TDM/SyncE/ToP) in use => true 1PPS source line delay => blank/empty 148 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport © 2015 Nokia Networks DN09224449 Issue 01B Appendix B – ToP Connectivity Options in Zone 149/157 B.4 TOP-F Sync using IP Unicast with GMC/BC present in Operator Core Network in Non-Daisy Chain Mode B.4.1 Overview The below configuration captures the network configuration and related topology where the FZAPs in the zone require frequency synchronization using 1588v2/PtP. The Boundary Clock (BC) or GMC (Grand Master Clock) is located in the operator core network and configured for ToP using IP unicast/L3 mode. The FZAPs in the zone network are configured to point to the GMC/BC and configured for ToP-F sync in IP unicast mode. An additional configuration to have a static host/network based route to reach the ToP master is added via the FZC BTSSM to the FZAPs so that the 1588v2 packets can bypass the controller but instead route via the ToP router shown in the figure below. MMEs MME IP Prim MME IP Sec ToP Master FZAP#1 S1‐MME Primary & Secondary Router R1 1GE 1GE Controller L2 Edge Router ToP Router 1GE Core Network O&M S1‐U FZAP #2 SAE‐GW FZAP #3 Access Network Zone Network B.4.1.1 Core Network Network Element Addresses Element IP-1 IP-2 TOP Master 10.83.224.201 -- ToP Router/Gateway 10.254.248.6 N/A Zone Network 10.254.248.0/29 N/A 150 /157 O&M Network © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix B – ToP Connectivity Options in Zone B.4.2 Configuration via BTS Site Manager B.4.2.1 TOPF-1 Feature Activation Flag ToP with freq synchronization “actTopFreqSynch” => True B.4.2.2 Timing over Packet masters properties table IP Address of the ToP master “masterIpAddr” => <ToP master IP@> B.4.2.3 APMOD-1 External GPS as reference source => false Transmission synchronization (TDM/SyncE/ToP) in use => true © 2015 Nokia Networks DN09224449 Issue 01B 151/157 1PPS source line delay => blank/empty B.4.2.4 APMOD-1/APSTPG-1 Protocol of the clock source “clockProtocol” => clkToP B.4.2.5 IPRT-2 Static IP routes set-1 Destination: for the GMC/BC (ToP master) IP address Gateway: this is the L3 router/gateway that is used to route the ToP packets to/from the ToP master bypassing the FZC. Netmask: 255.255.255.255 152 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport © 2015 Nokia Networks DN09224449 Issue 01B Appendix B – ToP Connectivity Options in Zone 153/157 B.5 TOP-P Sync using IP Unicast with GMC/BC present in Operator Core Network in Non-Daisy Chain Mode B.5.1 Overview The below configuration captures the network configuration and related topology where the FZAPs in the zone require phase synchronization using 1588v2/PtP. The Boundary Clock (BC) or GMC (Grand Master Clock) is located in the operator core network and configured for ToP using IP unicast/L3 mode. The FZAPs in the zone network are configured to point to the GMC/BC and configured for ToP-P sync in IP unicast mode. An additional configuration to have a static host/network based route to reach the ToP master is added via the FZC BTSSM to the FZAPs so that the 1588v2 packets can bypass the controller but instead route via the ToP router shown in the figure below. MMEs MME IP Prim MME IP Sec ToP Master FZAP#1 S1‐MME Primary & Secondary Router R1 1GE 1GE Controller L2 Edge Router ToP Router 1GE Core Network O&M S1‐U FZAP #2 SAE‐GW FZAP #3 Access Network Zone Network B.5.1.1 Core Network Network Element Addresses Element IP-1 IP-2 TOP Master 27.128.128.45 -- ToP Router/Gateway 192.168.55.254 N/A Zone Network 192.168.55.0/24 N/A 154 /157 O&M Network © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport Appendix B – ToP Connectivity Options in Zone B.5.2 Configuration via BTS Site Manager B.5.2.1 TOPP-1 Feature Activation Flag ToP with phase synchronization “actTopPhaseSynch” => True B.5.2.2 Acceptable Clock Quality Table B.5.2.3 ToP Masters Property table Here you provide the IP address of the TOP-P master GMC/BC to achieve sync from. © 2015 Nokia Networks DN09224449 Issue 01B 155/157 B.5.2.4 APMOD-1 External GPS as reference source => false Transmission synchronization (TDM/SyncE/ToP) in use => true 1PPS source line delay => blank/empty B.5.2.5 APMOD-1/APSTPG-1 Protocol of the clock source “clockProtocol” => clkToP 156 /157 © 2015 Nokia Networks DN09224449 Issue 01B Configuring Flexi Zone Controller LTE Transport B.5.2.6 Appendix B – ToP Connectivity Options in Zone IPRT-2 Static IP routes set-1 Destination: for the GMC/BC (ToP master) IP address Gateway: this is the L3 router/gateway that is used to route the ToP packets to/from the ToP master bypassing the FZC. Netmask: 255.255.255.255 © 2015 Nokia Networks DN09224449 Issue 01B 157/157
Copyright © 2024 DOKUMEN.SITE Inc.