12 Replies Latest reply on May 2, 2018 11:34 PM by cebula

    XL710 on ARM64 architecture

    cebula Green Belt

      Hello,

       

      I started to work on our new GPON embedded platform. HW is not available yet, but I want to be prepared. The board will have a Freescale's Layerscape processor LS2088 with 8 ARM64 cores. It will be connected (via 8xPCIe) to Ethernet controller FTXL710-BM1 (uplinks).

      On the other side the XL710 will be connected to the backplane with 4 x 10GBase-KR uplinks. We have chosen the XL710 controller because it supports PCIe->10GBASE-KR connection. We will use Freescale's Linux distribution on the system because it has support for Layerscape processor family.

       

      I have checked the i40e driver sources in our linux distribution and it is version 1.3.2. It doesn’t match any version in the "NVM and Software Compatibility". In the latest kernel version on kernel.org there is version 2.1.14 of i40e driver which also doesn't match any version in the SW compatibility table. So I downloaded the i40e sources (2.3.6) from Intel's download site  and successfully cross-compiled within our linux distribution. I also got NVM binary file (Version 6.01, CFG_ID=3.2) which is "XL710_6p01_KR_KX_NCSI_CFGID3p2_Quad.bin". Is this binary suitable for our configuration?

       

      Next question is how to put the NVM image in the FLASH? Do we need nvmupdate64e tool to burn the NVM image into flash? If yes, it must be compiled for ARM64 platform. Where can I get sources? Or ethtool is the right tool?

       

      And the last question: I expect that after "ifconfig up" I will get links up and running?

       

      Best regards

       

      Anton

        • Re: XL710 on ARM64 architecture
          AdolfoS Brown Belt

          Hello cebula

           

          I apologize for the delay in replying to your inquiry, I am consulting your case with our experts on ethernet controllers and I will provide more information as soon as possible.

           

          Best Regards,

          Adolfo Sanchez.

          • Re: XL710 on ARM64 architecture
            Carlos_A Brown Belt

            Hello, cebula:

             

            Thank you for contacting Intel Embedded Community.

             

            We suggest you follow the guidelines stated in the Intel(R) Ethernet NVM Update Tool Quick Usage Guide for Linux document # 332161.

             

            By the way, the Non-Volatile Memory (NVM) Update Utility for Intel(R) Ethernet Converged Network Adapter XL710 and X710 Series can be found at the following website:

             

            https://downloadcenter.intel.com/download/24769

             

            We hope that this information may help you.

             

            Best regards,

            Carlos_A.

              • Re: XL710 on ARM64 architecture
                cebula Green Belt

                Hello Carlos,

                 

                thank you for your suggestion. I have already seen the Intel(R) Ethernet NVM Update Tool Quick Usage Guide for Linux document before. That document doesn't help me much, because it is for x86 architecture.

                For updating NVM in linux there is a nvmupdate64e tool available (x86 binary). I downloaded it and started on our (old) target also with Layerscape CPU. Of course it didn't even start because the CPU is ARM64 arch.

                 

                root@ls2088ardb:~# file summ

                summ: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.14.0, BuildID[sha1]=1492b2f894dcd1533f441c5011cb9c6bf28d8df2, not stripped

                root@ls2088ardb:~# file nvmupdate64e

                nvmupdate64e: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.9, not stripped

                root@ls2088ardb:~# ./nvmupdate64e

                -sh: ./nvmupdate64e: cannot execute binary file: Exec format error

                root@ls2088ardb:~#

                 

                I also think that I don't need this utility. Can we update NVM with ethtool?

                 

                br, Anton

                  • Re: XL710 on ARM64 architecture
                    AdolfoS Brown Belt

                    Hello, cebula

                     

                    I am still working on your issue, and will provide more information as soon as possible.

                     

                    Best Regards,

                    Adolfo Sanchez

                    • Re: XL710 on ARM64 architecture
                      Carlos_A Brown Belt

                      Hello, cebula:

                       

                      Thanks for your reply.

                       

                      You will need to program the NVM device externally initially. To update the NVM after the device has been programmed, we can provide the source, but initial programming is not supported on the ARM at this moment.

                       

                      We hope that this information may help you.

                       

                      Best regards,

                      Carlos_A.

                        • Re: XL710 on ARM64 architecture
                          cebula Green Belt

                          Hi Carlos,

                           

                          Yes, I would like to have the sources. But what do you mean that initial programming is not supported on the ARM? If we compile your sources for ARM, we will have the tool, am I right?

                           

                          Our HW will be designed that the FLASH could be initially programmed through SPI, for later updating I suppose we will use the tool compiled for ARM. But, what prevent us that the tool is also used for the first programming?

                           

                          Best regards

                           

                          Anton

                            • Re: XL710 on ARM64 architecture
                              Carlos_A Brown Belt

                              Hello, cebula:

                               

                              Thanks for your reply.

                               

                              It is important to let you know that the XL710 is a complex device. The initial programming can be done via regular register reads/writes on the PCIe bus, much like how the tool EEPROM Access Tool (EAT) works. Ideally, EAT will be expanded to do initial programming of XL710. Though for updating the device, the device needs access to DMA which EAT does not support, even though EAT does support updating some older devices.

                               

                              Those are the reasons why initial programming is not possible.

                               

                              Best regards,

                              Carlos_A.

                      • Re: XL710 on ARM64 architecture
                        cebula Green Belt

                        Hello,

                         

                        now I have a board with XL710 and have the following problem regarding XL710. Three of four ports work, but the last one has a problem when I set the port up:

                         

                        root@ls2088ardb:~# ifconfig eth2 up

                        [   36.340371] 8021q: adding VLAN 0 to HW filter on device eth2

                        root@ls2088ardb:~#

                        root@ls2088ardb:~# ifconfig eth3 up

                        [   40.140668] 8021q: adding VLAN 0 to HW filter on device eth3

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~#

                        root@ls2088ardb:~# [   50.065491] ------------[ cut here ]------------

                        [   50.070122] WARNING: CPU: 1 PID: 0 at /users/rdss/zgor/FREESCALE/KERNEL_SPE_TRUNK/rte/sources/net/sched/sch_generic.c:321 dev_watchdog+0x268/0x274()

                        [   50.083439] NETDEV WATCHDOG: eth3 (i40e): transmit queue 1 timed out

                        [   50.089793] Modules linked in:

                        [   50.092849] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.35-lsp-1.0.70-0 #1

                        [   50.099895] Hardware name: Iskratel Layerscape 2088a GPON board SPE (DT)

                        [   50.106595] Call trace:

                        [   50.109042] [<ffff8000000898e4>] dump_backtrace+0x0/0x11c

                        [   50.114438] [<ffff800000089a14>] show_stack+0x14/0x1c

                        [   50.119488] [<ffff8000006d2238>] dump_stack+0x90/0xb0

                        [   50.124537] [<ffff8000000b3bc8>] warn_slowpath_common+0x98/0xd0

                        [   50.130455] [<ffff8000000b3c50>] warn_slowpath_fmt+0x50/0x58

                        [   50.136112] [<ffff8000005f3d70>] dev_watchdog+0x268/0x274

                        [   50.141509] [<ffff800000100a7c>] call_timer_fn.isra.29+0x28/0x94

                        [   50.147514] [<ffff800000100ca8>] run_timer_softirq+0x1c0/0x230

                        [   50.153345] [<ffff8000000b778c>] __do_softirq+0x108/0x244

                        [   50.158741] [<ffff8000000b7f74>] irq_exit+0x90/0xf4

                        [   50.163617] [<ffff8000000f17a8>] __handle_domain_irq+0x84/0x510

                        [   50.169535] [<ffff8000000824fc>] gic_handle_irq+0x84/0xe4

                        [   50.174930] Exception stack(0xffff80805b13bdf0 to 0xffff80805b13bf20)

                        [   50.181370] bde0:                                     00a02000 ffff8000 00000000 00010000

                        [   50.189549] be00: 5b13bf50 ffff8080 00086898 ffff8000 60000145 00000000 5f564000 00000080

                        [   50.197727] be20: 00000000 00000000 00000000 00000000 00000001 00000000 00000000 00000000

                        [   50.205906] be40: 00000000 00000000 5f564000 00000080 5f564000 00000080 009f2f40 ffff8000

                        [   50.214083] be60: 5b1307d0 ffff8080 5b13beb0 ffff8080 ffff9e5a 00000000 00000001 00000000

                        [   50.222261] be80: ce248c14 00000000 0000000a 00000000 00000000 00000000 00000000 00000000

                        [   50.230438] bea0: 00000000 00000000 00000000 00000000 00000000 00000000 00a02000 ffff8000

                        [   50.238616] bec0: 00a02960 ffff8000 00000000 00000000 009edbd8 ffff8000 5b13bf60 ffff8080

                        [   50.246793] bee0: 00000000 00000000 009f2318 ffff8000 006e9000 ffff8000 5b138000 ffff8080

                        [   50.254972] bf00: 00a02000 ffff8000 5b13bf50 ffff8080 00086894 ffff8000 5b13bf50 ffff8080

                        [   50.263150] [<ffff800000085700>] el1_irq+0x80/0x100

                        [   50.268024] [<ffff8000000e84b0>] cpu_startup_entry+0x1f8/0x264

                        [   50.273855] [<ffff8000000901a0>] secondary_start_kernel+0x114/0x120

                        [   50.280123] ---[ end trace f7be0532e0518ce7 ]---

                        [   50.284743] i40e 0000:01:00.3 eth3: tx_timeout: VSI_seid: 399, Q 1, NTC: 0x0, HWB: 0x1, NTU: 0x1, TAIL: 0x1, INT: 0x0

                        [   50.295359] i40e 0000:01:00.3 eth3: tx_timeout recovery level 1, hung_queue 1

                         

                        Message from syslogd@ls2088ardb at Mon Mar 12 18:33:23 2018 ...

                        ls2088ardb kernel: [   50.106595] Call trace:

                         

                        root@ls2088ardb:~# lspci

                        00:00.0 PCI bridge: Freescale Semiconductor Inc Device 8241 (rev 11)

                        01:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE backplane (rev 02)

                        01:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE backplane (rev 02)

                        01:00.2 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE backplane (rev 02)

                        01:00.3 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE backplane (rev 02)

                         

                         

                        Do you have any idea what should be wrong?

                        ===================================

                         

                        Best regards

                         

                        Anton