2 Replies Latest reply on Jan 1, 2014 11:34 PM by danielkaminsky

    Intel DPDK (Xeon) - how to reserve a core for Linux Kernel

    Community Member



             I am working on a board with Dual Intel Xeon E5-2600L series processors. With our app, I am able to achieve 40Gbps processing at "X" bytes for a few minutes, with 4 LoadBalancer Cores and 16 Worker cores. However, during extended test-runs, I am seeing packet drops in large bursts (burst of a few thousand packets) even at 2X packet size over a 24hr traffic test (constant 40Gbps traffic).


              I do not think it is PPS performance issue. The drops are always on the 82599 due to FIFO over-flow i.e. indicate that the Load Balancer is not fast enough to pull it sometimes (so issue is not at Workers). All the Load Balancer threads are running on Socket 0 - which is directly connected to 2 82599 devices. Theoretically the LoadBalancers should be capable of pulling 40Gbps at "X" Bytes and I also see that for a few minutes there are no drops. As I keep increasing the Packet size, the duration for the bursty drops keeps increasing, so much so that at 2X it takes about 24 hrs for the drops to happen -  but I still eventually see the drops - which then again go away for next few hours and so on (I am not yet sure if there is a definite periodicity to the drops).


             Inorder, to improve the LoadBalancer thread performance, I do not have those threads Hyper-threaded with the Worker threads - i.e. the corresponding HyperThread is not allocated (Initially I had the Workers hyper-thread with LoadBalancer and that was impacting performance of the LoadBalancer - resulting in drops on 82599).  I believe, the non-allocated threads can be used by the Linux Kernel. I was therefore wondering if this sudden drop indicates that some Linux Kernel processing is happening on the free cores and holding off the LoadBalancer due to hyper-thereading. To test this, I wanted to fix the Linux cores, to the other 4 cores that I have available (and ensure that there is no hyper-threading with the LoadBalancer threads). Is there a way I can force the Linux Kernel to exclude certain cores?


             Any other suggestions as to what maybe causing these drops during an extended traffic run (at line-rate) are welcome. (I do plan to test with the sample LoadBalancer app soon).