Technical DescriptionSymphony Plus Ethernet Networking Overview The Symphony Plus Plant Network, PN800, uses standard IP Ethernet technology for its communication backbone. PN800 replaces the INFI-NET loop from Symphony Harmony. Considering the fact that the network topology is specific to each project, this document outlines best practices, recommended topologies, and other implementation details and information. Summary This document is intended to guide plant engineers who are responsible for the safe and secure implementation of PN800 and the devices connected to it. These engineers are expected to have a basic familiarity with IP Ethernet networking technology and Symphony Harmony. It is not intended to be a definitive listing of how to configure Ethernet networks for Symphony Plus. It is intended to serve as a guide to a qualified industrial network engineer. Scope The primary scope of this document covers how to connect PN800 nodes to PN800 such that they can communicate with each other, and to operator consoles and engineering workstations. It also covers other Ethernet networks associated with Symphony Plus. Networking Overview The Network Stack IP Ethernet networking can be considered as a stack, a group of protocols which build upon each other. Each layer does not need to know about any of the other layers – they only need to be able to remove any additional overhead they add to a communication packet. ABB Inc. Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 1 of 18 Figure 1 The Networking Stack Physical Layer The physical layer defines the physical media (and its properties) which transmits the data.or multi-mode optical fibers). it will transmit the packet to the destination’s MAC address.docx Rev. IP Layer The IP layer handles communication between nodes that may not be directly connected together on the same layer 2 segment. Otherwise. connections between switches may use copper or fiber optic cable. As implied by the name. A Date: 4 April 2013 Page 2 of 18 . or the packet expires. CAT5E. 1000BASE-SX. 100BASE-TX or 1000BASE-TX over CAT5. 100BASE-SX. ABB Inc. A source module will compare its IP address and subnet mask to those of the destination. and if they are on the same IP network. A minimum speed of 100Mbps is required. Switches inspect the MAC address information. 1000Mbps or faster connectivity is recommended for switch interconnects. et al over single. Data Link Layer The Data Link Layer handles communication between two directly connected devices (e. This is the layer at which Ethernet devices’ MAC addresses are added to the packet. the IP layer adds IP addressing and subnet information to the packet. and only re-transmit the packet on the port to which the destination device is connected. However. Doc Id: 2VAA002993.g. The physical layer of all existing Ethernet-connected Symphony Plus modules is 100BASE-TX (using CAT5E at minimum). IP routing is currently not supported for PN800. two devices connected to the same switch).g. the source module will transmit the packet to its default gateway router (using the router’s MAC address). or CAT6 cable) or fiber optic cable (100BASE-FX. The media may be copper (e. which will then repeat this process until the destination node is directly connected to a router. ABB Inc.) is contained at the application layer. until the application receives just the data it is expecting. Putting It All Together Figure 2 Example Ethernet Packet As illustrated in Figure 2 above. process data). INFI-NET Term Symphony Plus Term Loop or Ring Segment Table 2 INFI-NET Concepts in PN800 Symphony Plus Ethernet Networks PN800 Plant Network PN800 is the network that connects Symphony Plus nodes together. INFI-NET message data (exception reports. It is functionally equivalent to INFI- NET loops in Symphony Harmony. each layer removes its encapsulation before passing the packet to the next layer up. a request and response).Transport Layer The transport layer is responsible for sessions which span multiple messages (for example. For example. Doc Id: 2VAA002993. each layer of the stack encapsulates the higher layer when being transmitted. Additional Documentation 2VAA002630 S+ Control: SPIEB800 INFI-NET to PN800 Ethernet Bridge User Manual IEC 62439-3 International Standard. with all the wrappers removed by the lower layers. When the packet is received. The protocol is defined by the individual applications. Industrial Communication Networks – High availability automation networks Table 1 Additional Documentation How INFI-NET concepts and terms translate to PN800 The table below lists several terms and definitions used in Symphony Harmony.g. Common protocols include Transmission Control Protocol (TCP) or Universal Datagram Protocol (UDP). A Date: 4 April 2013 Page 3 of 18 . and how they translate to Symphony Plus (as used in this document). Application Layer The application layer contains the actual data the running application is trying to communicate (e.docx Rev. etc. which provides an abstract example of an Ethernet packet. Symphony Plus system time mastership and distribution are managed using a proprietary protocol on PN800. only similar. It does not show any other networks. etc. independent.) can then be configured to assume the system time master role. a satellite clock) on SynchroNet using Simple Network Time Protocol (SNTP). and therefore provide similar time for Symphony Plus system time. non-redundant IP Ethernet network whose only purpose is to distribute highly accurate time to controller nodes for the purpose of time stamping Sequence of Events (SoE) data. SynchroNet is a dedicated. Note that Symphony Plus system time will not be as accurate as SynchroNet time in this scenario. a dedicated network interface on one (or more) console(s) could be connected to SynchroNet. L- SA SA HN800 CW800 HN800 CW800 SB SB HC800 CP800 HC800 CP800 MB810 COM MB810 COM L+ L+ L. The console’s operating system can then be configured to use the same time master as the controllers. CTB810 CTB811 F F F F R R R R 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 CTB810 CTB811 6 6 6 6 F F F F 7 7 7 7 R R R R 8 8 8 8 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 HN800 CW800 HN800 CW800 6 6 6 6 HC800 C P800 HC800 CP800 7 7 7 7 8 8 8 8 L+ L+ L. and the console software (Symphony Plus Operations. similar to the one used for Symphony Harmony. However. only those which need highly accurate time for SoE tracking should be connected. Not all controllers have to be connected to SynchroNet.g. Symphony Plus controllers receive time from the local time master (e. A Date: 4 April 2013 Page 4 of 18 . There are no special requirements for the network switches on SynchroNet. A SNTP client in the controller’s firmware connects to and receives time from a SNTP server. Doc Id: 2VAA002993. the number of switches between the time master and destination nodes should be kept to a minimum to minimize propagation delay (and therefore clock skew). L- SA SA SB SB MB810 COM MB810 COM Operations Console (optional) (optional) Operations Satellite Clock SynchroNet Switch Server Figure 3 Example SynchroNet Network ABB Inc.SOE Time Synchronization Network Every controller has a SOE Time Synchronization Network port (SynchroNet). SynchroNet is not used to provide Symphony Plus system-wide time.docx Rev. However. The figure below shows an example SynchroNet network. a PRP- enabled node uses the same MAC address on both networks A and B. Refer to IEC 62439-3 Clause 4 for the complete PRP standard used on PN800. Symphony Plus controllers have no special requirements for network switches on this connection. This results in zero failover time when one network fails. The figure below shows how the LRE is integrated into the network stack. the LRE sits between the physical Ethernet ports and the operating system on the node. because of this. the networks cannot be bridged together.Foreign Device Interface All Symphony Plus controllers have a dedicated. The topology and hardware of the second network should be identical to the first. with the destination node discarding the duplicate packet when received. Therefore. A Date: 4 April 2013 Page 5 of 18 . such that the higher layers of the stack are not aware that the connection is redundant. but. since both are always actively communicating. IEC 62439-3 Clause 4) to provide a redundant communication network. PRP Functional Overview PRP operates at the device level by transmitting effectively identical packets simultaneously on both networks. completely independent. referring to the two networks as “primary” and “backup” (or using similar terms) is inaccurate. non-redundant Ethernet Foreign Device Interface. or other IP Ethernet-based communications. Each PRP-enabled node has two Ethernet ports and a “Link Redundancy Entity” (LRE. which can be used for Modbus TCP. PN800 Network Redundancy PN800 uses Parallel Redundancy Protocol (PRP. we will use refer to the networks as “A” and “B” (per the IEC standard). PRP provides redundancy using a second. Logically. This section is intended to provide a brief overview of PRP. Doc Id: 2VAA002993. ABB Inc. Refer to the documentation of other devices on this network for any special requirements.docx Rev. HGS. As such. fault isolated network. independent. Such requirements are outside the scope of this document. either in software/firmware or hardware) which coordinates redundancy. To help facilitate the seamless redundancy. with one server in each redundant pair on either network A or B. PCs may connect to PN800 as DANs. Because PRP uses layer 2 broadcasts to coordinate network redundancy. Topology Restrictions AT NO TIME SHOULD NETWORKS A AND B BE CONNECTED TOGETHER! PRP operates at the data link layer (layer 2) of the network stack. is a device that is not PRP-enabled. it is currently not possible to extend redundant PN800 using layer 3 (IP) routers.docx Rev. or SAN. a SAN on network A cannot communicate with a SAN on network B). CP800. All Symphony Plus embedded modules which communicate on PN800 (e. With additional network cards and PRP driver software. and is transparent to devices that are not PRP-enabled. SANs can only communicate with DANs and SANs on the same network (e. PNI800. and is therefore attached to only one of the networks (either A or B). Doc Id: 2VAA002993. Figure 4 PRP LRE in the network stack PRP Device Types A device that is PRP-enabled and connected to both networks A and B is called a Doubly Attached Node. PCs and servers running Symphony Plus Operations and Engineering software are SANs. et al) are DANs. but their use is outside the scope of this document. A Date: 4 April 2013 Page 6 of 18 . A Singly Attached Node.g. S+ Operations servers may be installed redundantly. ABB Inc. but will not be fault-tolerant if there is a network failure on the network to which they are connected.g. or DAN. Third-party network drivers are available for PCs which enable them for PRP. Alternately. The best. Connecting between PN800 and other networks. or to any other public or private network (such as a corporate Intranet or other office network). and. or other software which needs to communicate on PN800 may be multi-homed (i. authenticated virtual private network (VPN) with limited user access would be a more secure solution to connect into PN800 from the outside.g. VPN appliance. therefore. A Date: 4 April 2013 Page 7 of 18 . where there is no direct electronic connection to outside networks. e. they may have network interfaces on PN800 and network interfaces on another network. deny entry to PN800. a firewall should be implemented as a minimum safety and security measure. a secure. and have explicit rules to allow only the necessary traffic to enter. Symphony Plus Engineering. encrypted. ABB Inc. by default.e. But Not Recommended.Network Security Do not directly connect PN800 to the public Internet. is outside the scope of this document. or any other device. The firewall should. Figure 5 Unacceptable Network Configurations If it is absolutely necessary to connect between PN800 and other networks. Doc Id: 2VAA002993. is neither recommended nor supported. using a firewall. most secure PN800 implementation is in the form of an air-gapped network. for the purpose of applying security updates to the operating system and other software from a trusted source on the other network). These computers must not route communications between PN800 and the other network. Figure 6 Acceptable. most robust..docx Rev. If possible. Network Connection Computers running Symphony Plus Operations. 2500. and the user assumes all responsibility for safety and security. regardless of the bridging technology. TCP and UDP Ports Used On PN800 If it is necessary to connect PN800 to outside networks. Doc Id: 2VAA002993. ABB Inc. is not recommended. communication to TCP ports 502. and 2502 should be blocked from entering PN800 from other outside networks. and 3000. and increases the risk of security breaches. A Date: 4 April 2013 Page 8 of 18 . Such a configuration reduces the overall security of the PN800 network. and UDP ports 123. using any method. Figure 7 Complete Redundant PN800 Network Overview Directly bridging PN800 to any other network. 2501. 2501. ABB strongly advises against such a configuration.docx Rev. g. it is neither supported nor recommended. there is no restriction on the assignment of loop numbers. To allow both the primary and backup controllers to maintain accurate time from the time master.g. Therefore.127. Though the IP addresses on SynchroNet can be assigned arbitrarily (i. and maximizes the loop/node address space. the third octet represents the PN800 segment number. it is recommended to use a similar addressing scheme as PN800 (e. they must have unique IP addresses. its IP address space should be similarly isolated.0. ABB Inc. Unlike INFI-NET. It is recommended that all primary modules use an even value in that octet (and allow the backup to assume an odd number).0 through 10.255.126. 10. Other Security Measures Implementing other active and/or passive security measures.0 to allow for maximum expandability. IP Addressing and Subnetting PN800 In PN800 the two least significant bytes of the IP address are equivalent to the INFI-NET Loop and Node number.255.0). As such. (e. This creates a single network address space equivalent to INFI-NET.e. such as intrusion detection systems. The backup’s address is set by the primary (via the redundancy link) by adding 1 to the least significant octet of the SynchroNet IP address.255). but because layer 3 routing within PN800 is not supported. A different network mask can be used to create a smaller network. SynchroNet Because the SynchroNet time synchronization network is completely independent of PN800. The subnet mask of all PN800 devices should be 255. A Date: 4 April 2013 Page 9 of 18 .255.g. is outside the scope of this document.e. it is recommended to set aside a /16 network segment for PN800 (e. and all network topologies are acceptable. Though layer 3 routing can be added between PN800 and outside networks.For optimum security. As such. the two different PN800 networks will not be able to communicate with each other..0. all incoming traffic should be blocked by default. there is no “central” or “satellite” loop. they do not have to correlate to the PN800 IP addresses in any way). it is outside the scope of this document. 10. and the fourth octet represents the node number on the segment).255.0.255 with a subnet mask of 255.docx Rev. This separation allows controllers in different PN800 segments to be connected to one time master. I.0. Doc Id: 2VAA002993. the IP address space should be isolated from the PN800 address space.126.127. with only specific paths opened on an as-needed basis. All IP addresses are valid.0 through 10. switches should be capable of 100Mbps Full-Duplex operation. such as VLAN tagging.g. NetGear. MOXA. 3COM. Linksys. the switch inspects the packet and reads the destination MAC address. Some managed switches are capable of performing basic IP routing. Managed vs. several vendors’ equipment has been used during product development and testing. It inspects each communication packet. These are called Layer 3 Switches. It will then re-transmit the packet on the interface which is connected to the destination device (and no others). trunking. and keeps the Symphony Plus system cost-effective. Network Cabling Electrically shielded copper Ethernet cables should be used when wiring PN800 and SynchroNet. If managed switches are desired (e. Refer to documentation for other devices connected to the Ethernet Foreign Device Interface to determine if shielded cables can be used. This simplicity reduces the number of possible misconfigurations and security vulnerabilities on PN800. ABB Inc.). Contemporary Controls CTRLink. HP. including Cisco.docx Rev. Additional care must be taken to ensure that switches with built-in redundancy do not adversely affect PN800 operation. Doc Id: 2VAA002993. and others. and reduce the size of MAC address tables in switches. Advantech. and re-transmits the packet (unmodified) on the port to which the destination device is connected. to collect statistics or other data using protocols such as SNMP). However. link aggregation. Unmanaged Switches PN800 is designed to be fully usable on networks employing only unmanaged switches (i. simultaneous spanning tree recalculations). This means that when a packet is received. care must be taken to ensure that switch failures do not affect both networks A and B (e. Use of any layer 3 routing is not supported on PN800. Routers A router allows an IP Ethernet network to be broken down into smaller Subnets to reduce load on switches. installing layer 3 routers between segments of PN800 is not supported.g. Types of Ethernet Devices Switches A switch is a device which allows two or more devices to communicate with each other over Ethernet. certain Hirschmann switches have additional redundancy features which must be disabled for operation on PN800. For example.Equipment Guide ABB does not recommend any particular network switch or router vendor. D-Link. no functionality relies on the capabilities of managed network switches.e. etc. Dell. Such configurations are outside the scope of this document. A Date: 4 April 2013 Page 10 of 18 . At minimum. Due to the method with which PRP coordinates redundancy. as ABB’s Symphony Plus product line adheres to standards-based Ethernet protocols. both ends of the cables should be clearly labeled with the network identifier (A or B) and the location of the other end (e. This document uses green lines to signify cables for SynchroNet. Refer to the IEB manual for additional implementation details. individual packets are only transmitted on the necessary switch ports in order for them to get to their destination (i. ABB Inc. If using INFI-NET as the central loop. and blue for network B cables. the IP address space for the individual PN800 segments (which are no longer directly interconnected) may be reduced. all rules for configuring INFI-NET must be followed. and be unique in the entire control system (e. for a PN800 segment that is satellite to a central INFI-NET loop. A Date: 4 April 2013 Page 11 of 18 . All modules will still receive broadcast messages. Node addresses must conform to INFI-NET requirements (numbered between 1 and 250).docx Rev. When combining PN800 segments with an existing Symphony Harmony system. individual modules should only see traffic that is destined for them). and subscribed modules will receive multicast messages. SynchroNet Time Masters ABB does not recommend any particular time master. any time master supporting SNTP can be used.2. with loop/segment 1 being the central network). a switch identifier and port number).3 of the PRP standard (IEC62439-3) defines red for network A cables. the IP address space should remain unchanged. the subnet mask may be changed to 255.Ethernet cables should also be color-coded to clearly identify a cable’s purpose.xxx and INFI-NET loop 45).0.g. either a PN800 segment or an INFI-NET loop.1. Doc Id: 2VAA002993. There must be one IEB800 between the central loop/segment and each satellite loop/segment. INFI-NET – PN800 Bridge IEB800 The INFI-NET – PN800 Bridge (IEB800) allows existing Symphony Harmony installations to expand by adding a Symphony Plus PN800 segment. During product development and testing. there cannot be both a PN800 subnet 45.e. including: Loop/segment numbers must adhere to INFI-NET requirements (numbered between 1 and 250. Additionally. Recommended Physical Topologies Thanks to the efficiencies of network switches compared to hubs. If a PN800 segment is used as the central network. a Symmetricom SyncServer S350 with a GPS antenna was used.255. There must be a single “central” network. For example. and default gateway for the nodes on that PN800 segment is the IP address of the IEB800. But. Section 4.255.g. A Date: 4 April 2013 Page 12 of 18 .or multiple- segment topology. there is only one network switch on each of the redundant networks. ABB Inc.docx Rev. an IEB800 can be connected to a spare port on the segment switches. with all PN800 nodes connected to the single switch.Single Segment Figure 8 Single Network Segment Topology In this topology. Doc Id: 2VAA002993. In this topology. It can seamlessly be expanded into a larger two. an IEB800 could be connected to either segment switch (preferably the one which will be ABB Inc. In this topology. Each network switch is connected to the PN800 nodes assigned to that segment. there are two redundant network switches. L- SA SA SB SB MB810 COM MB810 COM Figure 9 Two Segment Topology In this topology.Two Segments CTB810 CTB811 F F F F R R R R 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7 8 8 8 8 HN800 CW800 HN800 CW800 HC800 CP800 HC800 CP800 L+ L+ L. 4 SA SA 5 SB SB 6 7 8 PNI800 L+ L- SA SB CTB810 CTB811 F F R R 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 HN800 CW800 HN800 CW800 HC800 CP800 L+ L- SA SB MB810 COM PN800 “A” PN800 “B” Segment 2 CTB810 CTB811 F F F F R R R R Segment 2 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6 7 7 7 7 8 8 8 8 HN800 CW800 HN800 CW800 HC800 CP800 HC800 CP800 L+ L+ L. A Date: 4 April 2013 Page 13 of 18 . Doc Id: 2VAA002993.docx Rev. This ensures that only the traffic which needs to cross to the other segment goes across the link between the switches. one for each redundant segment. L. L- SA SA SB SB MB810 COM MB810 COM PN800 “A” PN800 “B” Segment 1 Segment 1 F R 1 2 3 4 5 6 7 8 PNI800 L+ L- CTB810 CTB811 F F SA R R SB 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 HN800 CW800 HN800 CW800 HC800 CP800 L+ L- SA SB MB810 COM F F R R 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 F PNI800 PNI800 R 1 2 L+ L+ 3 L. ABB Inc. A Date: 4 April 2013 Page 14 of 18 . Refer to the IEB800 manual for additional information and configuration restrictions. Alternatively. PNI800) on their respective segments.g.communicating with the INFI-NET loop more than the other). Several Segments In this topology. Expanding INFI-NET In this scenario. The segment switches are then connected to PN800 nodes (e.docx Rev. an IEB800 may be connected to the core switch. Doc Id: 2VAA002993. CP800. a customer has an existing Symphony Harmony system. the Several Segment topology could be employed to add an IEB800. with one segment switch per segment. there is a core switch which is connected to segment switches. In this topology. which is being expanded (using the IEB800) to add a PN800 segment. a single switch failure will not cause any disruption to the process. as one network will still be operating while the network with the failure re-calculates its spanning tree. to further minimize excess traffic passing through and between network switches. Figure 10 Expanding Symphony Harmony with Symphony Plus Other Topologies Many other network topology designs are possible.docx Rev. It may be better for performance to replace a smaller switch with a larger one. It is possible to sub-segment PN800 segments. as every switch through which a packet passes will add additional latency. A Date: 4 April 2013 Page 15 of 18 . Doc Id: 2VAA002993. ABB Inc. However. However. Using managed switches allows further redundancy to be added by utilizing Spanning Tree Protocol (or similar protocols) to create multiple paths from source to destination nodes. implementing a network using such technologies is outside the scope of this document. Because of PN800’s network redundancy. care must be taken. three PNI800s (two for S+ Operations and one for S+ Engineering). two S+ Operations consoles. a stand-alone S+ Engineering workstation. two (redundant) S+ Operations servers. and a satellite clock SNTP server for SynchroNet. A Date: 4 April 2013 Page 16 of 18 .Example Installation Below is an example installation. using four redundant HPC800s. Doc Id: 2VAA002993.docx Rev. Figure 11 Example Installation ABB Inc. The ideal PN800 network connection to a remote site is a dedicated. Device PN800 SynchroNet OpsNet HPC1 10.127.1.1.1.201 S+ Operations Console 1 N/A N/A 10. Doc Id: 2VAA002993.14 10.1.127.10 N/A 10.126.1.14 N/A 10.203 S+ Engineering 10.1.126. PN800. SynchroNet.255.101 N/A N/A PNI2 10. PSPG ABB Inc.202 S+ Operations Console 2 N/A N/A 10.1.0.1.0.1.1. Configurations which extend any Symphony Plus communication network using a third-party connection are neither recommended nor supported.127.127.126. hard-wired.1.127.127.126.111 N/A N/A Workstation SNTP Server N/A 10. point-to-point or mesh wireless.1.10 10.126. Addressing the Management Network and Corporate Intranet are beyond the scope of this document.1.126.126.2 N/A Table 3 IP Addresses for Example Installation Remote Network Connections IP Ethernet networking technology provides many options for connecting to remote sites (e. A Date: 4 April 2013 Page 17 of 18 .g.1.12 10.102 N/A N/A PNI3 10.1.127.200 10.15 HPC4 10.126.12 N/A 10.126.103 N/A N/A S+ Operations Server 1 10. an Internet service provider). et al).The table below lists IP addresses for all devices on all networks in the above diagram.1.125.1.13 HPC3 10. A All Initial 4Apr2013 C.docx Rev.126.17 PNI1 10.1.200 S+ Operations Server 2 10.16 10.1. VPN. (C) & Dept. direct link.127.201 10. which does not utilize any equipment or services owned or controlled by a third-party (e.16 N/A 10. and OpsNet subnet masks are 255.1.g. Revision Revision Page (P) Description Date Changed by Chap.1.11 HPC2 10.127.1.126.127.1.1. Marks.201 10.125.1.125.200 10.125. docx Rev. Doc Id: 2VAA002993. A Date: 4 April 2013 Page 18 of 18 . ABB Inc.
Report "2VAA002993 en S Control and IO Symphony Plus Ethernet Networking"