29 Replies Latest reply on Jan 6, 2014 10:40 PM by muthurajanjayakumar

    Data Plane Developer Kit

    Community Member

      The Data Plane SDK was announced last year (available 2Q2010) as Intel's solution for fast-path network processing. Previously, the only available solutions were proprietary via 3rd party vendors such as WindRiver. As far as I can tell, it is still not available. Does anyone have info regarding release dates for the SDK? We are interested in gaining knowledge on this subject for a future IPTV VOD-server project.

       

      Thanks,

      Steve

        • Re: Data Plane Developer Kit
          Felix_M BlackBelt

          Hi Steve:

           

          Welcome to the Intel® Embedded Community.

           

          Do you happen to have a link to info on this SDK?  Or do you recall where you saw the announcement?  I looked around intel.com and don't find anything.

           

          Thanks

          Felix

           

          J. Felix McNulty

          Community Moderator

          (Intel contractor)

          • Re: Data Plane Developer Kit
            Community Member

            Get Access to Intel® Data Plane Development Kit (Intel® DPDK) source code

            I’m so forested a week of search and  contacting the appropriate people and posting at premier help ( https://premier.intel.com/premier/PremierResources.aspx)

            I’ve not able to find or get an access to  Intel® Data Plane Development Kit (Intel® DPDK) source code.

            I got a copy of “Developer's Reference” but that is it.

            Is there anyone out there can help me please?

              • Re: Data Plane Developer Kit
                Community Member

                Hi Steven,

                 

                per my comments above, the Intel(r) Data Plane Development Kit is still under the guise of an Early Access program, where the code can only be released under NDA and only when a candidate company is accepted into the early access program. Please contact me directly if you'd like to know more.

                 

                Best regards,

                Michael

              • Re: Data Plane Developer Kit
                Community Member

                Hi Felix,

                     I'm working on DPDK, but I'm not very clear about the relation between the NIC driver and DPDK.

                     You know, DPDK can run as app on Linux.  So doen that means latest linux kernel driver of intel NIC has supported DPDK?

                 

                     Another problem: what platform does DPDK support?  What CPU does it support?

                 

                 

                    Thank you.

                  • Re: Data Plane Developer Kit
                    Green Belt

                    The Linux kernel drivers are not used by the DPDK. The DPDK includes its own drivers which are made of a different paradigm: "Polling" and "no OS".

                     

                    Both drivers are exclusive:

                      - either your PCI NIC device is managed by Linux (igb, ixgb)

                      - either your PCI NIC device is managed by the DPDK (librte_82576_pmd/ or librte_82599_pmd/)

                     

                    The DPDK looks like a library that your applications should link with and it should be used to bypass the OS services (so drivers). DPDK is optimized for Intel architectures.

                     

                    Best regards,

                      Vincent@6WIND

                      • Re: Data Plane Developer Kit
                        Community Member

                        Hi Vincent,

                           Thank you very much.

                            When dpdk run as linux app, how does the NIC works?

                          • Re: Data Plane Developer Kit
                            Green Belt

                            There are two parts, let's try to make it simple:

                              - probes -> main() / init()

                             

                            There is a userland PCI probing during the startup of the userland process (see the main()) and init() functions. Then, it maps the memory to read/write to the PCI NICs.

                             

                              - RX/TX -> main loop

                             

                            In order to receive packets, using the properly mapped memory during the init, it polls the queues of packets which have been DMA-ed into the memory.

                            In order to send, still using this mapped memory, it fills some queues with packets and it requests to the NICs to drain (DMA-ed) those queues of packets which are sent on the wire.

                             

                            This main loop is made of a while(1) { } in order to pump the packets and then to push the packets.

                             

                            The Linux kernel drivers are NOT used anymore when the PMD runs for the NICs which are managed by the DPDK. Other NICs can still be managed by the Linux kernel drivers.

                             

                            If you want to better understand some insides, there are some examples of code and a good documentation about it into the DPDK source code itself (I cannot post, you would need NDA with 6WIND or Intel or etc...). Or I can only advice you to play with the 6WINDGate dataplane that includes a fine grain integration of the DPDK for high speed and low latency packet processing.

                             

                            Best regards,

                              Vincent@6WIND

                          • Re: Data Plane Developer Kit
                            Community Member

                            Hi,   the Linux ixgbe drivers are already NAPI enabled, which supports a poll mode.   I am curious about why NAPI was  not considered  optimum enough for packet processing.  Would appreciate any insight. Thanks

                              • Re: Data Plane Developer Kit
                                Green Belt

                                NAPI is not pure PMD.

                                  • Re: Data Plane Developer Kit
                                    Community Member

                                    Ok,  would help  if you can throw in a few lines on why or how NAPI is not PMD.  

                                    • Re: Data Plane Developer Kit
                                      Green Belt

                                      Hi vincent,

                                      After I bound a net device to IGB_UIO driver, if I want to configure it for network access, use linux tools like ifconfig, what should I do? in DPDK, is kni response for this function?

                                      any response will be appericated!

                                        • Re: Data Plane Developer Kit
                                          Green Belt

                                          Hi Jiexie,

                                           

                                          KNI would provide path to your kernel (a kind of fast TUN/TAP channel), but it is not designed to manage (ifconfig'd) the ethernet ports with some IP addresses. In order to configure IP addresses, it means that you need a stack; but DPDK provides only library (librte_*). In order to get a stack, check: www.6wind.com which is the best I know , see: http://www.6wind.com/products/6windgate-protocols/

                                           

                                          Sorry,

                                            Vincent

                                            • Re: Data Plane Developer Kit
                                              Green Belt

                                              Hi vincent, thanks for responding.

                                              so I can transmit packets to kernel through KNI, let linux network protocol stack  process the related packets, then I can use ifconfig, tcpdump...

                                               

                                              besides, I met a problem when test the kni example provided by intel,  can you please give some advice?

                                              EAL: memzone_reserve_aligned_thread_unsafe(MP_mbuf_pool, 311173440, 0)No appropriate segment found

                                              EAL: Error - exiting with code: 1

                                              Cause: Could not initialise mbuf pool

                                                • Re: Data Plane Developer Kit
                                                  LynnZ Brown Belt

                                                  Hi Jiexie,

                                                  After the KNI user space example application and kernel module both successfully loaded, the KNI creates a vEth0net_dev that is very similar to regular Linux network device drive, so standard Linux tools like ifconfig, ip link, ethtool can be used to configure this vEth0 device. The Linux kernel stack will be used in this case.

                                                  The only difference of KNI vs. regular network device drive is that the KNI talks to user space PMD rather
                                                  than NIC via FIFOs.

                                                    • Re: Data Plane Developer Kit
                                                      LynnZ Brown Belt

                                                      Here is another resource for DPDK.  Intel DPDK is a set of libraries and drivers for fast packet processing on x86 platforms.  It runs mostly in Linux userland.  This project tracks Intel DPDK and includes all major public contributions.

                                                        • Re: Data Plane Developer Kit
                                                          Green Belt


                                                          Hi LynnZ,

                                                          Thanks for your response!

                                                          NOW I use kni crete two vEthX device, and configure with address, connect the two ports coressponding vEthX device, but ping fails, no ARP reply is sent, so no ICMP reply.

                                                          some docs said packets are sent to kernel stack through KNI interface vEthX device, so there should be ARP reply sent out, but not, or vEthx just receive and send out, no matter what types of packets.

                                                          the corresponding ports of vEth0 and vEth1 are connected.

                                                          [root@localhost ~]# ifconfig vEth0
                                                          vEth0     Link encap:Ethernet  HWaddr 00:1E:67:24:08:DA
                                                                    inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
                                                                    inet6 addr: fe80::21e:67ff:fe24:8da/64 Scope:Link
                                                                    UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
                                                                    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
                                                                   TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
                                                                    collisions:0 txqueuelen:1000
                                                                    RX bytes:0 (0.0 b)  TX bytes:1908 (1.8 KiB)

                                                          [root@localhost ~]# ifconfig vEth1
                                                          vEth1     Link encap:Ethernet  HWaddr 00:1E:67:24:08:DB
                                                                    inet addr:192.168.1.2  Bcast:192.168.1.255  Mask:255.255.255.0
                                                                    inet6 addr: fe80::21e:67ff:fe24:8db/64 Scope:Link
                                                                    UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
                                                                    RX packets:15 errors:0 dropped:0 overruns:0 frame:0
                                                                    TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
                                                                    collisions:0 txqueuelen:1000
                                                                    RX bytes:900 (900.0 b)  TX bytes:468 (468.0 b)

                                                          [root@localhost ~]#
                                                          [root@localhost ~]#
                                                          [root@localhost ~]# ping 192.168.1.1 -I vEth1
                                                          PING 192.168.1.1 (192.168.1.1) from 192.168.1.2 vEth1: 56(84) bytes of data.
                                                          From 192.168.1.2 icmp_seq=2 Destination Host Unreachable
                                                          From 192.168.1.2 icmp_seq=3 Destination Host Unreachable
                                                          From 192.168.1.2 icmp_seq=4 Destination Host Unreachable
                                                          From 192.168.1.2 icmp_seq=6 Destination Host Unreachable

                                                            • Re: Data Plane Developer Kit
                                                              LynnZ Brown Belt

                                                              Hi jiexie,  the DPDK team asked me to make you aware of  the developer mailing lists at www.dpdk.org where you will find a bigger DPDK open source community there.  They have discussions related to KNI happening there right now.  Can you check it out and let me know what you think?  LynnZ

                                                                • Re: Data Plane Developer Kit
                                                                  LynnZ Brown Belt

                                                                  By the way, jiexie, you need to register on the developer mailing lists to see those discussions at www.dpdk.org.

                                                                  • Re: Data Plane Developer Kit
                                                                    Green Belt

                                                                    OK,

                                                                    I will visit  www.dpdk.org. to check what can I get for my problem.

                                                                    Thanks very much!

                                                                    • Re: Data Plane Developer Kit
                                                                      Community Member
                                                                      Hi LynnZ,

                                                                       

                                                                      I am using DPDK 1.5.0 and trying to use KNI interface to measure throughput through it

                                                                       

                                                                      ----------------------------------------------------------------------------------------------------------

                                                                       

                                                                       

                                                                      I have machine with 2 lcores and 2 physical interface, and in running kni sample application --config parameter we are supposed to give
                                                                      1)port no,
                                                                      2)lcore_rx,
                                                                      3)lcore_tx, right?

                                                                       

                                                                      so can it be possible to give same lcore no. for both ports, like

                                                                       

                                                                      kni -c 0x3 -n 4 -- -P -p 0x3 --config"(0,0,1),(1,0,1)"
                                                                      As i have tried and after running this two interface vEth0 and vEth1 is created, but when i am giving ip with ifconfig to vEth1 then it is showing the below message ...

                                                                       

                                                                      SIOCSIFFLAGS: Timer expired,

                                                                       

                                                                      while i am able to bring up the the vEth0 interface successfully.

                                                                       

                                                                      -----------------------------------------------------------------------------------------------------
                                                                      i am pasting some details which prints while running the application ...

                                                                      Checking link status
                                                                      .....done
                                                                      Port 0 Link Up - speed 1000 Mbps - full-duplex
                                                                      Port 1 Link Up - speed 1000 Mbps - full-duplex
                                                                      APP: Lcore 1 is writing to port 0
                                                                      APP: Lcore 0 is reading from port 0

                                                                       

                                                                      and here in last two line Lcore 1 and Lcore 0 is writting and reading for port 0 and may be therefore i am able to bring up the vEth0. and there is no core for port 1 so it is failing.

                                                                       

                                                                      ---------------------------------------------------------------------------------------------------------
                                                                      So my question is why can't we use same lcores for reading and writing  for both ports, does DPDK restricts using same lcores for 2 ports (as it seems), and if so, then why??

                                                                       

                                                                       

                                                                      Thanks in advance

                                                                      Chandra

                                                                        • Re: Data Plane Developer Kit
                                                                          muthurajanjayakumar Green Belt

                                                                          Dear customer.

                                                                          Thank you for using Intel DPDK.

                                                                          You are correct - the way kni sample example application is designed - is the way what you have specified here "Locre 1 and Lcore 0 writing and reading for port 0". In agreement with what you have described, Chapter 8.1 overview of http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdk-sample-applications-user-guide.pdf indicates that "For a physical NIC port, one thread reads from the port and writes to the kni and another thread reads from KNI devices and writes the data unmodified to the port".

                                                                           

                                                                          We had TUN / TAP implementation for communication between user space app to kernel space. The need was better performance. Hence we have KNI. So, performance point of view, we have one core dedicated per port.

                                                                           

                                                                          Implementation [below] follow the design intent with kni_port_params having the mapping for 1 port ID.

                                                                           

                                                                          kni_ingress(struct kni_port_params *p)
                                                                          {
                                                                          uint8_t i, port_id;
                                                                          unsigned nb_rx, num;
                                                                          uint32_t nb_kni;
                                                                          struct rte_mbuf *pkts_burst[PKT_BURST_SZ];

                                                                          if (p == NULL)
                                                                          return;

                                                                          nb_kni = p->nb_kni;
                                                                          port_id = p->port_id;
                                                                          for (i = 0; i < nb_kni; i++) {
                                                                          /* Burst rx from eth */

                                                                          Thank you,

                                                                            • Re: Data Plane Developer Kit
                                                                              Community Member

                                                                              Hi Kumar,

                                                                               

                                                                              1.) What i can see into code is that, for one KNI interface creation we require two dedicated lcores (i.e one for reading and one for writting , that is two lcores for one KNI interface), so if i have to create two interface then i need four dedicated lcores(according to present design of KNI application)???

                                                                               

                                                                              2.) If it is so, then could you please write me why the designers had thought of using two dedicated lcores for one KNI interface, do the thread safety in mempool libraries and RX/TX in the PMD were their concern while design.

                                                                               

                                                                              3.) i really need some pointers from you, what area (in KNI application) i have to target if i need to create two KNI interface from two lcores (b'coz i have only two cores in my system).

                                                                               

                                                                              Thanks in advance,

                                                                              Chandra

                                                                                • Re: Data Plane Developer Kit
                                                                                  muthurajanjayakumar Green Belt

                                                                                  Dear Chandra,

                                                                                  Acknowledging your mail.

                                                                                  Will get back to you

                                                                                   

                                                                                  Thank you 

                                                                                    • Re: Data Plane Developer Kit
                                                                                      muthurajanjayakumar Green Belt

                                                                                      Hi,

                                                                                      the reason(s) why kni sample app uses a separate lcore for each port:

                                                                                      a) performance.

                                                                                      b) design simplicity.

                                                                                       

                                                                                      There is nothing that prevents same logical core to handle both RX and TX for multiple ports.

                                                                                       

                                                                                      It has best performance if a lcore in user space move packet descriptors from DPDK to KNI queue and another lcore in kernel space then COPY packet data to skb and then push skb to kernel stack.

                                                                                       

                                                                                      It should have no thread safe issue no matter it is multiple lcores poll same queue or a lcore polls multiple queues. For the former case, lockless queue design leverage atomic instruction “cmpxchg”(BTW, but we did see performance drops significantly for this case); for the second case, no shared data among queues.

                                                                                       

                                                                                      For the customer case of using same logical core to handle both RX and TX for multiple ports, basically the aggregated performance remains same while per port performance will be half of the one port case giving they have 2 ports to support

                                                                                        • Re: Data Plane Developer Kit
                                                                                          muthurajanjayakumar Green Belt

                                                                                          Assuming you are using DPDK 1.5, and pls take a look at  new command-line below on how to support multi-threaded KNI. Basically, it is ok to have single kernel thread(pin to a lcore) to handle multiple queues of multiple ports. But KNI application needs some changes as it assumes no overlapping of Rx/Tx lcore on multiple ports. The biggest change should be in main_loop() function as below, highlighted code snippet that need to change in YELLOW. ARE YOU ABLE to see the color in the advanced editor mode - "Use Advanced Editor"

                                                                                           

                                                                                           

                                                                                          static int

                                                                                          main_loop(__rte_unused void *arg)

                                                                                          {

                                                                                                 uint8_t i, nb_ports = rte_eth_dev_count();

                                                                                                 int32_t f_stop;

                                                                                                 const unsigned lcore_id = rte_lcore_id();

                                                                                                 enum lcore_rxtx {

                                                                                                        LCORE_NONE,

                                                                                                        LCORE_RX,

                                                                                                        LCORE_TX,

                                                                                                        LCORE_MAX

                                                                                                 };

                                                                                                 enum lcore_rxtx flag = LCORE_NONE;

                                                                                           

                                                                                                 nb_ports = (uint8_t)(nb_ports < RTE_MAX_ETHPORTS ?

                                                                                                                      nb_ports : RTE_MAX_ETHPORTS);

                                                                                                 for (i = 0; i < nb_ports; i++) {

                                                                                                        if (!kni_port_params_array[i])

                                                                                                               continue;

                                                                                                        if (kni_port_params_array[i]->lcore_rx == (uint8_t)lcore_id) {

                                                                                                               flag = LCORE_RX;

                                                                                                               break;

                                                                                                        } else if (kni_port_params_array[i]->lcore_tx ==

                                                                                                                                    (uint8_t)lcore_id) {

                                                                                                               flag = LCORE_TX;

                                                                                                               break;

                                                                                                        }

                                                                                                 }

                                                                                           

                                                                                                  /* code above get the port index and Rx/Tx direction, should get port mask if overlapping is allowed */

                                                                                           

                                                                                                 if (flag == LCORE_RX) {

                                                                                                        RTE_LOG(INFO, APP, "Lcore %u is reading from port %d\n",

                                                                                                                             kni_port_params_array[i]->lcore_rx,

                                                                                                                             kni_port_params_array[i]->port_id);

                                                                                                        while (1) {

                                                                                                               f_stop = rte_atomic32_read(&kni_stop);

                                                                                                               if (f_stop)

                                                                                                                      break;

                                                                                                               kni_ingress(kni_port_params_array[i]);

                                                                                           

                                                                                                                 /* code highlighted need to receive traffic from all ports specified in the port mask */

                                                                                                                 /* Note: suggest to have a lcore in charge of Rx for all ports. Having a lcore Rx port0 and Tx port1 would cause a lot more code change */

                                                                                                        }

                                                                                                 } else if (flag == LCORE_TX) {

                                                                                                        RTE_LOG(INFO, APP, "Lcore %u is writing to port %d\n",

                                                                                                                             kni_port_params_array[i]->lcore_tx,

                                                                                                                             kni_port_params_array[i]->port_id);

                                                                                                        while (1) {

                                                                                                               f_stop = rte_atomic32_read(&kni_stop);

                                                                                                               if (f_stop)

                                                                                                                      break;

                                                                                                               kni_egress(kni_port_params_array[i]);

                                                                                           

                                                                                                                 /* code highlighted need to transmit traffic to all ports specified in the port mask */

                                                                                                                 /* Note: suggest to have a lcore in charge of Rx for all ports. Having a lcore Rx port0 and Tx port1 would cause a lot more code change */

                                                                                                        }

                                                                                                 } else

                                                                                                        RTE_LOG(INFO, APP, "Lcore %u has nothing to do\n", lcore_id);

                                                                                           

                                                                                                 return 0;

                                                                                          }