AdvancedTCA (ATCA) systems are used in many telecom, industrial, medical, security, and military applications because they allow system engineers to apply the processing horsepower of many standardized processor blades to large computing problems. One of the big issues associated with maintaining and enhancing these large systems is backwards interconnect compatibility. This problem isn't new. It has existed for years as the primary serial interconnect standards employed by ATCA blades evolve into faster versions. When the ATCA interconnect scheme is Ethernet (as opposed to PCI Express, Infiniband, Rapid IO, or StarFabric), then Ethernet autonegotiation becomes a key factor to ensure easy, blade-by-blade upgrading of existing system designs.
Ethernet autonegotiation allows dynamic selection of conflicting Ethernet features. For example, within the 1000BASE-X Ethernet specification, autonegotiation permits the automatic selection of:
- Duplex (half or full)
- Flow control (symmetric pause, asymmetric pause, or no flow control)
- Indicate partners failure
- Exchange status information
So Ethernet autonegotiation was important for ATCA system design even years ago when there was much less choice among Ethernet standards and when there was no IEEE standardization for Ethernet over the point-to-point backplane connections employed by ATCA platforms.
However, Ethernet technology marches on and choices have multiplied. The IEEE approved the 802.3ap standard in 2007. This standard includes the 1-Gbd Ethernet interface and two 10-Gbd (10 Gbps data rate) Ethernet interfaces: 10GBASE-KX4 and 10GBASE-KR. The 10GBASE-KX4 interface standard uses all four ports (lanes) of an AdvancedTCA blade fabric channel and splits the bit traffic over those four channels. All four lanes are required to make one blade-to-blade 10GBASE-KX4 connection, so there are none left over for additional signaling. The 10GBASE-KR specification uses only one lane so three remain in the channel and can be used for other purposes, giving each ATCA channel a potential bandwidth of 40 Gbd.
Autonegotiation establishes links with the fastest possible connection available at both endpoints, so 10GBASE-KX4 and 10GBASE-KR Ethernet interfaces must be able to negotiate a connection with one lane of 1000BASE-KX. Note that this is not true of PICMG's 10GBASE-BX4 specification but the PICMG 3.1 R2.0 subcommittee is working to ensure that existing 10GBASE-BX and -BX4 components will plug and play in -KX4 and -KX systems. The table below shows a few of the compatibility scenarios for 10GBASE-BX, 10GBASE-KX, and 10GBASE-KX4 ports.
Thus autonegotiation continues to play a key role in ATCA systems by enabling backward compatibility amongst ATCA blades with Ethernet connectivity. Consequently, the PICMG 3.1 R2.0 subcommittee has spent a lot of time on interconnect scenarios to ensure backward compatibility with existing ATCA blades and platforms while enabling higher-speed communications. The PICMG 3.1 R2.0 specification is still under development as of this writing.
The future is not limited to 10-Gbd Ethernet, however. Work is already underway for 40-Gbd connections over the backplane. The IEEE 802.3ba task force is developing a 40GBASE-KR4 standard that leverages the 10GBASE-KR Ethernet specification. Thus AdvancedTCA platforms that support 10GBASE-KR ports now should in theory be able to support 40GBASE-KR4 Ethernet ports in the future, but this standard is still a few years away from adoption.
The future won't stop there. A 10GBASE-KR Ethernet port employs advances in PHYs, equalization technology, forward-error-correction techniques, and other interconnect technologies to achieve backplane signaling speeds once thought impossible. The same developments certainly appear to be within reason for 40-Gbd connections. When 40-GBd interconnect for ATCA blades becomes possible, Ethernet autonegotiation will still be an important component in the quest for backwards compatibility with older ATCA blades.
How far do you think we can take the ATCA backplane beyond 10 Gbits/sec? Do you need to for your embedded applications? How soon?
Note: This blog post is based on the information in an excellent article titled "AdvancedTCA meets 10GBASE-KR" by Doug Sandy of Emerson Network Power Embedded Computing and a GE Fanuc application note titled "Ethernet Interoperability in AdvancedTCA, AdvancedMC and MicroTCA" written by Gene Juknevicius. Emerson Network Power is a Premier member of the Intel® Embedded and Communications Alliance (Intel® ECA) and GE Fanuc is an Associate member of Intel® ECA.
Intel® Embedded and Communications Alliance