ALCATELLUCENT_Levrau_MPLSforTeleprotection

Teleprotection over an IP/MPLS NetworkTechnical validation ir. Lieven LEVRAU Alcatel-Lucent IP Division April 4th, 2011 • • • • IP/MPLS-based Utility Networks Challenges for IP/MPLS in teleprotection Other design considerations Conclusion © Alcatel-Lucent 2011 All Rights Reserved 1 IP/MPLS-based Utility Networks All Rights Reserved © Alcatel-Lucent © 2011 . IP/MPLS-Based Utility Network Power Generation 7705 SAR Corporate HQ / NOC OS6855 LAN IED Internet Omni PCX 5620 SAM SCADA 7750 SR 7750 SR IP/MPLS Network 7750 SR 7705 SAR Servers Collaboration Tool NOC Substation Substation Transmission 7750 SR 7705 SAR 7705 SAR IED OS6855 7705 SAR RTU LAN TPR Video Surveillance TPR LAN TDM Video Surveillance 4 | MPLS Network for Teleprotection | Sep 2009 All Rights Reserved © Alcatel-Lucent © 2011 . PWE FR. ATM. end-to-end. synchronous. … Traffic isolation. VLL. analog voice.… IP/MPLS based . DoS End-to-end secure management of network and services Guaranteed bandwidth.L2-VPNs. Serial. L3-VPNs. L2/L3 Encryption + dynamic secure Key distribution. TDM. Rtng protocols authentication. Fast Reroute.How IP/MPLS Address the challenges of the Utility Networks? Reliability Traffic Isolation Multiple Legacy technologies Security Unified management Quality of Service High Availability. low latency and jitter ` Enhanced QoS on different network layers All Rights Reserved © Alcatel-Lucent © 2011 . 2 Challenges for IP/MPLS in teleprotection All Rights Reserved © Alcatel-Lucent © 2011 . 703 MUX E1/T1 E&M RS-232 Ethernet 7710 SR 7750 SR 7705 SAR E1/T1 7705 SAR Teleprotection relay (TPR) signals must be transferred reliably and fast with low latency End to end delay = telecom network latency + teleprotection equipment delay TDM over MPLS for legacy support (requires integration of legacy interface in MPLS node to limit and control end to end delay) VPLS for IEC 61850 (requires high reliable Layer 2 transport) 7 | MPLS Network for Teleprotection | Sep 2009 All Rights Reserved © Alcatel-Lucent © 2011 .Using MPLS Network for Teleprotection Substation Substation kV G.703 MUX TPR ∆t E&M RS-232 Ethernet IP/MPLS TPR G. low latency and jitter Enhanced QoS on different network layers All Rights Reserved © Alcatel-Lucent © 2011 .Main Challenges for Teleprotection End to End Delay Jitter Asymmetry Resiliency Denial of Services Synchronization Quality of Service Depending on vendor equipment May include packetisation/depacketisation Variation of delay in certain circumstances such as traffic burst on network Delay variations between transmit and receive The impact of a failure in the network shouldn’t be noticed by application How DoS/DDoS can affect applications TDM applications need end to end synchronization Guaranteed bandwidth. End-to-End TDM Transport Model (only left-to-right direction shown) TDM Packets moving in this direction DS1 Access Circuit DS1 LIU Data Packetization Si g GigE Packet Switched Network (PSN) GigE Jitter Data Buffer Si g DS1 LIU DS1 Access Circuit Packetization • As TDM traffic from the Access Circuit (AC) is received. it is packetized and transmitted into the PSN • Two modes of operation: • CESoPSN (RFC5086) for structured nxDS0/64k channels • SAToP (RFC4553) for unstructured T1 Network • Fixed delay • Packet transfer delay based on link speeds and distances from end to end Playout • TDM PW packets are received from the PSN and stored into its associated configurable jitter buffer • Play-out of the TDM data back into the AC when it’s at least 50% full • Variable delay • the number of and type of switches • queuing point in the switches • QoS is key to ensure effective service delivery 9 | MPLS Network for Teleprotection | Sep 2009 All Rights Reserved © Alcatel-Lucent © 2011 . PD of 2 ms (16 T1 frames/packet).703 MUX E1/T1 E&M RS-232 Ethernet 7750 SR 7750 SR 7705 SAR E1/T1 7705 SAR TPR relay signals must be transferred in < ∆t ∆T = 1 cycle at 50Hz or 60Hz = 20ms or 16ms End to end delay = telecom network latency + teleprotection equipment time latency = packetization delay + network delay + jitter buffer delay The total end-to-end latency is calculated by summing the packetization delay (PD).Latency Using MPLS Network for Teleprotection kV Substation Substation G.March2011 All Rights Reserved © Alcatel-Lucent © 2011 .703 MUX TPR ∆t E&M RS-232 Ethernet IP/MPLS TPR G. ND of 3 ms.Validation .g. JBD of 4 ms – Total Latency = 2 + 3 + 4 = 9 ms 10 | Teleprotection over IP/MPLS . network delay (ND) and jitter buffer delay (JBD) as shown here: Total Latency = PD + ND + JBD – e. 703 MUX E1/T1 E&M RS-232 Ethernet 7750 SR 7750 SR 7705 SAR E1/T1 7705 SAR TPR relay signals must be transferred in < ∆t ∆T = 1 cycle at 50Hz or 60Hz = 20ms or 16ms End to end delay = telecom network latency + teleprotection equipment time Telecom network latency = packetization delay + network delay + jitter buffer delay 11 | Teleprotection over IP/MPLS .March2011 All Rights Reserved © Alcatel-Lucent © 2011 .703 MUX TPR ∆t E&M RS-232 Ethernet IP/MPLS TPR G.Using MPLS Network for Teleprotection Latency kV Substation Substation G.Validation . Validation .Calculation of Latency for Teleprotection Latency Latency is mainly at the edge where low speeds are present (Serial / E1 / 100 FX) Latency in the core depends on number of nodes but mainly transmission delays Each node adds a maximum of : 150 s (eqpt latency) 10 s (transmission of 1500 Bytes over a Gigabit link) 3 s / km : speed of light transmission over fiber Example of a connexion between 2 TPRs : Over 1000km – traversing 10 nodes : 4ms 12 | Teleprotection over IP/MPLS .March2011 All Rights Reserved © Alcatel-Lucent © 2011 . Validation . The implementation should allow total control of the bandwidth required per application. This can be solved by applying the correct QoS parameters to the node.` Rate Limit TPR to 100Kbps Rate Limit CCTV to 6Mbps TPR TPR gets to EF class CCTV to BE Class Per SAP policing Per interface queuing Ingress Make Sure that no application can go higher than expected bandwidth. Via Rate limit per SAP Egress Ensure that TPR application always gets the priority H-QoS 13 | Teleprotection over IP/MPLS .Jitter Jitter in normal operations may come from the equipments themselves This is minimal (less than 1% of router latency) and compensated by jitter buffer. Jitter happens in non normal conditions such as congestion.March2011 All Rights Reserved © Alcatel-Lucent © 2011 . minimizing jitter. The impact of a failure (node or link) can have huge impact in case the failure exceeds the 50ms.March2011 All Rights Reserved © Alcatel-Lucent © 2011 . These 50ms apply only in the core of MPLS network.Validation . MPLS FRR technology allows 50ms failover time in any failure scenario. 14 | Teleprotection over IP/MPLS . but not in the case of the Access switch failing.Impact of failure Today’s teleprotection applications were developed towards SDH/PDH 50ms failover time. All Rights Reserved © Alcatel-Lucent © 2011 .8032 and BGP-MH for IEC 61850 greatly enhances total availability of the applications. G. All ALU Service routers support FRR. for example with Microwave links Implementing MC-LAG. Integrating TDM in MPLS (as in 7705 SAR) brings the FRR to the application and limits the failure risks (compared to a 2 box solution) Providing extended rapid failover scenarios may be key in many applications. Support very rapid convergence time even in case of non direct connectivity (BFD).Conclusion on network failure impact Failover in the backbone can be limited to 50ms with FRR FRR has to be implemented in the first node to minimize failure risks. 3 Other design considerations for Teleprotection 16 | Teleprotection over IP/MPLS .Validation .March2011 All Rights Reserved © Alcatel-Lucent © 2011 . March2011 Tracking of LSP Path for a given service All Rights Reserved © Alcatel-Lucent © 2011 .Validation . Only trouble shooting tools may discover that.Packet flow asymmetry LSPs are unidirectional Topology changes or mis-configuration may result in different path being used in both directions. Bypass actual path Green and Red LSP follow different Path Active Path Bypass Logical Link Tracking of LSP status and history 17 | Teleprotection over IP/MPLS . 5620 CPAM allows this detection and raises alarms. Validation .March2011 All Rights Reserved © Alcatel-Lucent © 2011 . Many TDM applications require clock synchronization (as SDH) Some applications require Time of Day type of synchronization Packet networks can deliver clocking through different techniques : 18 | Teleprotection over IP/MPLS .Synchronization Information need to be delivered with time precision from an application standpoint. 4 Summary All Rights Reserved © Alcatel-Lucent © 2011 . very low jitter requirements Impact of a failure in the application Alcatel-Lucent demonstrates that its IP/MPLS can be used as the next generation network for such applications with : Native TDM integration (incl. …) Very high resistance to potential failures in network Several management tools to anticipate. synchronization. and manage the transition of existing mission critical applications 20 | Teleprotection over IP/MPLS . interfaces.Validation .Conclusion Teleprotection is the most stringent application that can be transported over networks due to : Low delays requirements. control and trouble shoot network IP/MPLS is the foundation to prepare the Smart Grid data explosion.March2011 All Rights Reserved © Alcatel-Lucent © 2011 . www.alcatel-lucent.com .