From patchwork Wed Nov 11 06:26:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yingya Han X-Patchwork-Id: 83939 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7A4C4A09D2; Wed, 11 Nov 2020 07:31:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 748A737B4; Wed, 11 Nov 2020 07:31:24 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 675562BAB for ; Wed, 11 Nov 2020 07:31:21 +0100 (CET) IronPort-SDR: w0DWcHuHOpapiysHA53BP9Ca20mYq/AxpXQZvdBvFICf/vvt7tw25gufrvxUS/R8LR9NdXdtRi Afc8BJYFoJqQ== X-IronPort-AV: E=McAfee;i="6000,8403,9801"; a="149951482" X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="149951482" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2020 22:31:19 -0800 IronPort-SDR: yCSf7hioV2cDkRZkJJYCQrqKwaSVV90aI2IRmmbax6pFKIdd9iN7qnGrSAUm47oZJhLqzcyt7x ChP4x7tcDoXg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="308345733" Received: from dpdk-lijuan-purley2.sh.intel.com ([10.67.119.168]) by fmsmga008.fm.intel.com with ESMTP; 10 Nov 2020 22:31:18 -0800 From: hanyingya To: dts@dpdk.org Cc: hanyingya Date: Wed, 11 Nov 2020 14:26:59 +0800 Message-Id: <20201111062701.36217-2-yingyax.han@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201111062701.36217-1-yingyax.han@intel.com> References: <20201111062701.36217-1-yingyax.han@intel.com> Subject: [dts] [PATCH V1 1/3]test_plans: add vf single core test plan X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Signed-off-by: hanyingya --- test_plans/vf_single_core_perf_test_plan.rst | 122 +++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 test_plans/vf_single_core_perf_test_plan.rst diff --git a/test_plans/vf_single_core_perf_test_plan.rst b/test_plans/vf_single_core_perf_test_plan.rst new file mode 100644 index 00000000..53cf304a --- /dev/null +++ b/test_plans/vf_single_core_perf_test_plan.rst @@ -0,0 +1,122 @@ +.. Copyright (c) <2020>, Intel Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + - Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + - Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + + - Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + +====================================================== +Benchmark the performance of VF single core forwarding +====================================================== + +Prerequisites +============= + +1. Nic single core performance test requirements: + + 1.1) FVL25G: two dual port FVL25G nics, all installed on the same socket, + pick one port per nic. + 1.2) NNT10G: four 82599 nics, all installed on the same socket, + pick one port per nic. + 1.3) CVL100G: one CVL100G nics, all installed on the same socket, + pick one port per nic. + +2. Software:: + + dpdk: git clone http://dpdk.org/git/dpdk + scapy: http://www.secdev.org/projects/scapy/ + dts (next branch): git clone http://dpdk.org/git/tools/dts, + then "git checkout next" + Trex code: http://trex-tgn.cisco.com/trex/release/v2.84.tar.gz + (to be run in stateless Layer 2 mode, see section in + Getting Started Guide for more details) + python-prettytable: + apt install python-prettytable (for ubuntu os) + or dnf install python-prettytable (for fedora os). + +3. Connect all the selected nic ports to traffic generator(IXIA,TREX, + PKTGEN) ports(TG ports):: + + 2 TG 25g ports for FVL25G ports + 4 TG 10g ports for 4 NNT10G ports + 1 TG 100g ports for CVL100G port + +Test Case : Vf Single Core Performance Measurement +================================================== + +1. Bind PF ports to kernel driver, i40e/ixgbe/ice, then create 1 VF from each PF, + take XXV710 for example:: + + echo 1 > /sys/bus/pci/devices/0000\:af\:00.0/sriov_numvfs + echo 1 > /sys/bus/pci/devices/0000\:b1\:00.1/sriov_numvfs + +2. Set vf mac address:: + + ip link set ens5f0 vf 0 mac 00:12:34:56:78:01 + ip link set ens6f0 vf 0 mac 00:12:34:56:78:02 + +3. Bind all the created VFs to dpdk driver, vfio-pci or igb_uio:: + + ./usertools/dpdk-devbind.py -b vfio-pci af:02.0 b1:02.0 + +4. Start testpmd:: + + ./dpdk-testpmd -l 28,29 -n 4 -- -i --portmask=0x3 --txd=512 --rxd=512 \ + --txq=2 --rxq=2 --nb-cores=1 + + testpmd> set fwd mac + testpmd> start + +5. Configure traffic generator to send traffic:: + + dst mac: peer nic port mac + src ip : random + dst ip : fixed + frame size: 64 byte + transmit rate: 100% + +6. Result tables. + + +-----------+------------+---------+-------------+---------+---------------------+ + | Fwd_core | Frame Size | TXD/RXD | Throughput | Rate | Expected Throughput | + +===========+============+=========+=============+=========+=====================+ + | 1C/1T | 64 | 512 | xxxxx Mpps | xxx % | xxxxxxx Mpps | + +-----------+------------+---------+-------------+---------+---------------------+ + | 1C/1T | 64 | 2048 | xxxxx Mpps | xxx % | xxxxxxx Mpps | + +-----------+------------+---------+-------------+---------+---------------------+ + | 1C/2T | 64 | 512 | xxxxx Mpps | xxx % | xxxxxxx Mpps | + +-----------+------------+---------+-------------+---------+---------------------+ + | 1C/2T | 64 | 2048 | xxxxx Mpps | xxx % | xxxxxxx Mpps | + +-----------+------------+---------+-------------+---------+---------------------+ + + Check throughput and compare it with the expected value. Case will raise failure + if actual throughputs have more than 1Mpps gap from expected ones. + +Note : + The values for the expected throughput may vary due to different platform and OS, + and traffic generator, please correct threshold values accordingly. From patchwork Wed Nov 11 06:27:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yingya Han X-Patchwork-Id: 83941 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E42ABA09D2; Wed, 11 Nov 2020 07:31:26 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DAA0B5913; Wed, 11 Nov 2020 07:31:25 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id B6F9837B4 for ; Wed, 11 Nov 2020 07:31:22 +0100 (CET) IronPort-SDR: j19lYD95NxwLRuUoRERi+IV61SAMHYpydhmTM65KpUWlbF0Nk/qkAJvcToJbCqNffGmJcJJ31h IuvwSQPBQ5Uw== X-IronPort-AV: E=McAfee;i="6000,8403,9801"; a="149951484" X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="149951484" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2020 22:31:21 -0800 IronPort-SDR: PiinPf4cZdPDCKUf1tQwrCDtvTzURJwyUOitS3iZXeoixXh71EEyKnWKDrkUqgeNQVq3jA92sU asGzYfQ7h7hg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="308345745" Received: from dpdk-lijuan-purley2.sh.intel.com ([10.67.119.168]) by fmsmga008.fm.intel.com with ESMTP; 10 Nov 2020 22:31:19 -0800 From: hanyingya To: dts@dpdk.org Cc: hanyingya Date: Wed, 11 Nov 2020 14:27:00 +0800 Message-Id: <20201111062701.36217-3-yingyax.han@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201111062701.36217-1-yingyax.han@intel.com> References: <20201111062701.36217-1-yingyax.han@intel.com> Subject: [dts] [PATCH V1 2/3]conf: add vf single core configuration file X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Signed-off-by: hanyingya --- conf/vf_single_core_perf.cfg | 64 ++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 conf/vf_single_core_perf.cfg diff --git a/conf/vf_single_core_perf.cfg b/conf/vf_single_core_perf.cfg new file mode 100644 index 00000000..041faa87 --- /dev/null +++ b/conf/vf_single_core_perf.cfg @@ -0,0 +1,64 @@ +# Throughput numbers vary in different environment. +# Users could change these configuration on demand: +# +# - test_parameters defines the combination of frame size and descriptor +# numbers, and the pattern is +# {'frame size': ['descriptor number #1', 'descriptor number #2']} +# +# - test_duration is how many seconds each combination performance will +# be recorded. +# +# - accepted_tolerance defines the accepted tolerance between test +# results and expected numbers. +# +# - expected_throughput is a dictionary defining expected throughput +# numbers based on NIC, and the pattern is +# {'NIC': {'frame size': {'descriptor number': 'excepted throughput'}}} +# Every user should fill it out with your actual numbers. To keep the +# expected throughput private, dts takes 0.00 as default. +# +# - if update_expected == Ture, and add argument "--update-expected" in +# bash command, all objects in this file will changed after the run +# for example: ./dts --update-expected +# +# Highlights: +# At the begining, please change test_parameters according to your +# requirements, then run ./dts --update-expected to get the absolute +# results which will replace the default numbers 0.00 in this configuration. +# So you will have your own private configuration, and could start your +# tests as ususal. + +[suite] +update_expected = True +test_parameters = {'1C/1T': {64: [512, 2048]}, + '1C/2T': {64: [512, 2048]}} +test_duration = 60 +accepted_tolerance = 1 +expected_throughput = { + 'fortville_spirit': { + '1C/1T': {64: {512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {512: 0.00, 2048: 0.00}}}, + 'fortville_eagle': { + '1C/1T':{64: {512: 0.00, 2048: 0.00}}, + '1C/2T':{64: {512: 0.00, 2048: 0.00}}}, + 'niantic': { + '1C/1T': {64: {128: 0.00, 512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {128: 0.00, 512: 0.00, 2048: 0.00}}}, + 'fortville_25g': { + '1C/1T': {64: {512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {512: 0.00, 2048: 0.00}}}, + 'columbiaville_100g': { + '1C/1T': {64: {512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {512: 0.00, 2048: 0.00}}}, + 'columbiaville_25g': { + '1C/1T': {64: {512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {512: 0.00, 2048: 0.00}}}, + 'ConnectX4_LX_MT4117': {'40G': { + '1C/1T': {64: {128: 0.00, 256: 0.00, 512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {128: 0.00, 256: 0.00, 512: 0.00, 2048: 0.00}}}, + '25G': { + '1C/1T': {64: {128: 0.00, 256: 0.00, 512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {128: 0.00, 256: 0.00, 512: 0.00, 2048: 0.00}}}}, + 'ConnectX5_MT4121': { + '1C/1T': {64: {128: 0.00, 256: 0.00, 512: 0.00, 2048: 0.00}}, + '1C/2T': {64: {128: 0.00, 256: 0.00, 512: 0.00, 2048: 0.00}}}} From patchwork Wed Nov 11 06:27:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yingya Han X-Patchwork-Id: 83940 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B238FA09D3; Wed, 11 Nov 2020 07:31:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ADDBC567C; Wed, 11 Nov 2020 07:31:24 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 2CE7E2BAB for ; Wed, 11 Nov 2020 07:31:23 +0100 (CET) IronPort-SDR: 3CoIZN/kSltW39g296WeIIv9L8a72X8VrTukwL+71BQXGu2HOntJhN6/sKCdhEs722wMmXBv6t Utn3XQmmjI4A== X-IronPort-AV: E=McAfee;i="6000,8403,9801"; a="149951486" X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="149951486" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2020 22:31:22 -0800 IronPort-SDR: 0bhBrc4CNoyWJAlHRz1/JHnBkciiHCZvQfzeyeXw5UadvzUlJ9YgTIyUVr32d2qJlSicP9/NcZ s93hLbaUqTHQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="308345750" Received: from dpdk-lijuan-purley2.sh.intel.com ([10.67.119.168]) by fmsmga008.fm.intel.com with ESMTP; 10 Nov 2020 22:31:21 -0800 From: hanyingya To: dts@dpdk.org Cc: hanyingya Date: Wed, 11 Nov 2020 14:27:01 +0800 Message-Id: <20201111062701.36217-4-yingyax.han@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201111062701.36217-1-yingyax.han@intel.com> References: <20201111062701.36217-1-yingyax.han@intel.com> Subject: [dts] [PATCH V1 3/3]tests: add vf single core test script X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Signed-off-by: hanyingya --- tests/TestSuite_vf_single_core_perf.py | 399 +++++++++++++++++++++++++ 1 file changed, 399 insertions(+) create mode 100644 tests/TestSuite_vf_single_core_perf.py diff --git a/tests/TestSuite_vf_single_core_perf.py b/tests/TestSuite_vf_single_core_perf.py new file mode 100644 index 00000000..57ab7240 --- /dev/null +++ b/tests/TestSuite_vf_single_core_perf.py @@ -0,0 +1,399 @@ +# +# Copyright(c) 2020 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +""" +DPDK Test suite. +""" + +import utils +import json +import os +from test_case import TestCase +from settings import HEADER_SIZE, UPDATE_EXPECTED, load_global_setting +from pmd_output import PmdOutput +from copy import deepcopy +import rst +from pktgen import PacketGeneratorHelper + + +class TestVfSingleCorePerf(TestCase): + + def set_up_all(self): + """ + Run at the start of each test suite. + PMD prerequisites. + """ + # Based on h/w type, choose how many ports to use + self.dut_ports = self.dut.get_ports() + self.verify(len(self.dut_ports) >= 1, "At least 1 port is required to test") + self.socket = self.dut.get_numa_id(self.dut_ports[0]) + self.vfs_mac = ["00:12:34:56:78:0%d" % (i + 1) for i in self.dut_ports] + self.pmdout = PmdOutput(self.dut) + + # set vf assign method and vf driver + self.vf_driver = self.get_suite_cfg()['vf_driver'] + if self.vf_driver is None: + self.vf_driver = self.drivername + + # get dts output path + if self.logger.log_path.startswith(os.sep): + self.output_path = self.logger.log_path + else: + cur_path = os.path.dirname( + os.path.dirname(os.path.realpath(__file__))) + self.output_path = os.sep.join([cur_path, self.logger.log_path]) + # create an instance to set stream field setting + self.pktgen_helper = PacketGeneratorHelper() + + # determine if to save test result as a separated file + self.save_result_flag = True + + def set_up(self): + """ + Run before each test case. + It's more convenient to load suite configuration here than + set_up_all in debug mode. + """ + # test parameters include: frames size, descriptor numbers + self.test_parameters = self.get_suite_cfg()['test_parameters'] + + # traffic duraion in second + self.test_duration = self.get_suite_cfg()['test_duration'] + self.expected_throughput = self.get_suite_cfg()['expected_throughput'][self.nic] + + # Accepted tolerance in Mpps + self.gap = self.get_suite_cfg()['accepted_tolerance'] + + # header to print test result table + self.table_header = ['Fwd_core', 'Frame Size', 'TXD/RXD', 'Throughput', 'Rate', + 'Expected Throughput', 'Throughput Difference'] + self.test_result = {'header': [], 'data': []} + + # initilize throughput attribution + self.throughput = {} + + def setup_vf_env(self): + """ + require enough PF ports,using kernel or dpdk driver, create 1 VF from each PF. + """ + self.used_dut_port = [port for port in self.dut_ports] + self.sriov_vfs_port = [] + for i in self.dut_ports: + host_driver = self.dut.ports_info[i]['port'].default_driver + self.dut.generate_sriov_vfs_by_port(self.used_dut_port[i], 1, driver=host_driver) + sriov_vfs_port = self.dut.ports_info[self.used_dut_port[i]]['vfs_port'] + self.sriov_vfs_port.append(sriov_vfs_port) + + # set vf mac address. + for i in self.dut_ports: + pf_intf = self.dut.ports_info[i]['port'].get_interface_name() + self.dut.send_expect("ip link set %s vf 0 mac %s" % (pf_intf, self.vfs_mac[i]), "#") + + # bind vf to vf driver + try: + for i in self.dut_ports: + for port in self.sriov_vfs_port[i]: + port.bind_driver(self.vf_driver) + except Exception as e: + self.destroy_vf_env() + raise Exception(e) + + def destroy_vf_env(self): + """ + destroy the setup VFs + """ + for i in self.dut_ports: + self.dut.destroy_sriov_vfs_by_port(self.dut_ports[i]) + + def flows(self): + """ + Return a list of packets that implements the flows described in l3fwd. + """ + return [ + 'IP(src="1.2.3.4",dst="192.18.1.0")', + 'IP(src="1.2.3.4",dst="192.18.1.1")', + 'IP(src="1.2.3.4",dst="192.18.0.0")', + 'IP(src="1.2.3.4",dst="192.18.0.1")', + 'IP(src="1.2.3.4",dst="192.18.3.0")', + 'IP(src="1.2.3.4",dst="192.18.3.1")', + 'IP(src="1.2.3.4",dst="192.18.2.0")', + 'IP(src="1.2.3.4",dst="192.18.2.1")'] + + def create_pacap_file(self, frame_size, port_num): + """ + Prepare traffic flow + """ + payload_size = frame_size - HEADER_SIZE['ip'] - HEADER_SIZE['eth'] + pcaps = {} + for _port in self.dut_ports: + if 1 == port_num: + flow = ['Ether(dst="%s")/%s/("X"*%d)' % (self.vfs_mac[_port], self.flows()[_port], payload_size)] + pcap = os.sep.join([self.output_path, "dst{0}.pcap".format(_port)]) + self.tester.scapy_append('wrpcap("%s", [%s])' % (pcap, ','.join(flow))) + self.tester.scapy_execute() + pcaps[_port] = [] + pcaps[_port].append(pcap) + else: + index = self.dut_ports[_port] + cnt = 0 + for layer in self.flows()[_port * 2:(_port + 1) * 2]: + flow = ['Ether(dst="%s")/%s/("X"*%d)' % (self.vfs_mac[_port], layer, payload_size)] + pcap = os.sep.join([self.output_path, "dst{0}_{1}.pcap".format(index, cnt)]) + self.tester.scapy_append('wrpcap("%s", [%s])' % (pcap, ','.join(flow))) + self.tester.scapy_execute() + if index not in pcaps: + pcaps[index] = [] + pcaps[index].append(pcap) + cnt += 1 + return pcaps + + def prepare_stream(self, pcaps, port_num): + """ + create streams for ports,one port one stream + """ + tgen_input = [] + if 1 == port_num: + txIntf = self.tester.get_local_port(self.dut_ports[0]) + rxIntf = txIntf + for pcap in pcaps[0]: + tgen_input.append((txIntf, rxIntf, pcap)) + else: + for rxPort in range(port_num): + if rxPort % port_num == 0 or rxPort ** 2 == port_num: + txIntf = self.tester.get_local_port(self.dut_ports[rxPort + 1]) + port_id = self.dut_ports[rxPort + 1] + else: + txIntf = self.tester.get_local_port(self.dut_ports[rxPort - 1]) + port_id = self.dut_ports[rxPort - 1] + rxIntf = self.tester.get_local_port(self.dut_ports[rxPort]) + for pcap in pcaps[port_id]: + tgen_input.append((txIntf, rxIntf, pcap)) + return tgen_input + + def test_perf_vf_single_core(self): + """ + Run nic single core performance + """ + self.setup_vf_env() + port_num = len(self.dut_ports) + self.perf_test(port_num) + self.handle_expected() + self.handle_results() + + def handle_expected(self): + """ + Update expected numbers to configurate file: conf/$suite_name.cfg + """ + if load_global_setting(UPDATE_EXPECTED) == "yes": + for fwd_config in list(self.test_parameters.keys()): + for frame_size in list(self.test_parameters[fwd_config].keys()): + for nb_desc in self.test_parameters[fwd_config][frame_size]: + self.expected_throughput[fwd_config][frame_size][nb_desc] = \ + round(self.throughput[fwd_config][frame_size][nb_desc], 3) + + def perf_test(self, port_num): + """ + Single core Performance Benchmarking test + """ + # ports whitelist + eal_para = "" + for i in range(port_num): + eal_para += " -w " + self.sriov_vfs_port[i][0].pci + port_mask = utils.create_mask(self.dut_ports) + # parameters for application/testpmd + param = " --portmask=%s" % (port_mask) + + for fwd_config in list(self.test_parameters.keys()): + # the fwd_config just the config for fwd core + # to start testpmd should add 1C to it + core_config = '1S/%s' % fwd_config + thread_num = int(fwd_config[fwd_config.find('/')+1: fwd_config.find('T')]) + core_list = self.dut.get_core_list(core_config, socket=self.socket) + self.verify(len(core_list) >= thread_num, "the Hyper-threading not open, please open it to test") + + # need add one more core for start testpmd + core_list = [core_list[0]] + [str(int(i) + 1) for i in core_list] + + self.logger.info("Executing Test Using cores: %s of config %s" % (core_list, fwd_config)) + + nb_cores = thread_num + + # fortville has to use 2 queues at least to get the best performance + if self.nic in ["fortville_25g", "fortville_spirit"] or thread_num == 2: + param += " --rxq=2 --txq=2" + # columbiaville use one queue per port for best performance. + elif self.nic in ["columbiaville_100g", "columbiaville_25g"]: + param += " --rxq=1 --txq=1" + # workaround for that testpmd can't forward packets in io forward mode + param += " --port-topology=loop" + + self.throughput[fwd_config] = dict() + for frame_size in list(self.test_parameters[fwd_config].keys()): + self.throughput[fwd_config][frame_size] = dict() + pcaps = self.create_pacap_file(frame_size, port_num) + tgenInput = self.prepare_stream(pcaps, port_num) + for nb_desc in self.test_parameters[fwd_config][frame_size]: + self.logger.info("Test running at parameters: " + "framesize: {}, rxd/txd: {}".format(frame_size, nb_desc)) + parameter = param + " --txd=%d --rxd=%d --nb-cores=%d" % (nb_desc, nb_desc, nb_cores) + self.pmdout.start_testpmd( + core_list, parameter, eal_para, socket=self.socket) + self.dut.send_expect("set fwd mac", "testpmd> ", 15) + self.dut.send_expect("start", "testpmd> ", 15) + + vm_config = self.set_fields() + # clear streams before add new streams + self.tester.pktgen.clear_streams() + + # run packet generator + streams = self.pktgen_helper.prepare_stream_from_tginput(tgenInput, 100, vm_config, self.tester.pktgen) + # set traffic option + traffic_opt = {'duration': self.test_duration} + _, packets_received = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt) + self.verify(packets_received > 0, "No traffic detected") + throughput = packets_received / 1000000.0 + self.throughput[fwd_config][frame_size][nb_desc] = throughput + + self.dut.send_expect("stop", "testpmd> ") + self.dut.send_expect("quit", "# ", 30) + + self.verify(throughput, "No traffic detected, please check your configuration") + self.logger.info("Trouthput of " + "framesize: {}, rxd/txd: {} is :{} Mpps".format( + frame_size, nb_desc, throughput)) + + return self.throughput + + def handle_results(self): + """ + results handled process: + 1, save to self.test_results + 2, create test results table + 3, save to json file for Open Lab + """ + # save test results to self.test_result + header = self.table_header + for fwd_config in list(self.test_parameters.keys()): + ret_datas = {} + for frame_size in list(self.test_parameters[fwd_config].keys()): + wirespeed = self.wirespeed(self.nic, frame_size, len(self.dut_ports)) + ret_datas[frame_size] = {} + for nb_desc in self.test_parameters[fwd_config][frame_size]: + ret_data = {} + ret_data[header[0]] = fwd_config + ret_data[header[1]] = frame_size + ret_data[header[2]] = nb_desc + ret_data[header[3]] = "{:.3f} Mpps".format( + self.throughput[fwd_config][frame_size][nb_desc]) + ret_data[header[4]] = "{:.3f}%".format( + self.throughput[fwd_config][frame_size][nb_desc] * 100 / wirespeed) + ret_data[header[5]] = "{:.3f} Mpps".format( + self.expected_throughput[fwd_config][frame_size][nb_desc]) + ret_data[header[6]] = "{:.3f} Mpps".format( + self.throughput[fwd_config][frame_size][nb_desc] - + self.expected_throughput[fwd_config][frame_size][nb_desc]) + + ret_datas[frame_size][nb_desc] = deepcopy(ret_data) + self.test_result[fwd_config] = deepcopy(ret_datas) + + # Create test results table + self.result_table_create(header) + for fwd_config in list(self.test_parameters.keys()): + for frame_size in list(self.test_parameters[fwd_config].keys()): + for nb_desc in self.test_parameters[fwd_config][frame_size]: + table_row = list() + for i in range(len(header)): + table_row.append( + self.test_result[fwd_config][frame_size][nb_desc][header[i]]) + self.result_table_add(table_row) + # present test results to screen + self.result_table_print() + + # save test results as a file + if self.save_result_flag: + self.save_result(self.test_result) + + def save_result(self, data): + """ + Saves the test results as a separated file named with + self.nic+_single_core_perf.json in output folder + if self.save_result_flag is True + """ + json_obj = dict() + case_name = self.running_case + json_obj[case_name] = list() + status_result = [] + for fwd_config in list(self.test_parameters.keys()): + for frame_size in list(self.test_parameters[fwd_config].keys()): + for nb_desc in self.test_parameters[fwd_config][frame_size]: + row_in = self.test_result[fwd_config][frame_size][nb_desc] + row_dict0 = dict() + row_dict0['performance'] = list() + row_dict0['parameters'] = list() + result_throughput = float(row_in['Throughput'].split()[0]) + expected_throughput = float(row_in['Expected Throughput'].split()[0]) + # delta value and accepted tolerance in percentage + delta = result_throughput - expected_throughput + if delta > -self.gap: + row_dict0['status'] = 'PASS' + else: + row_dict0['status'] = 'FAIL' + row_dict1 = dict(name="Throughput", value=result_throughput, unit="Mpps", delta=delta) + row_dict2 = dict(name="Txd/Rxd", value=row_in["TXD/RXD"], unit="descriptor") + row_dict3 = dict(name="frame_size", value=row_in["Frame Size"], unit="bytes") + row_dict4 = dict(name="Fwd_core", value=row_in["Fwd_core"]) + row_dict0['performance'].append(row_dict1) + row_dict0['parameters'].append(row_dict2) + row_dict0['parameters'].append(row_dict3) + row_dict0['parameters'].append(row_dict4) + json_obj[case_name].append(row_dict0) + status_result.append(row_dict0['status']) + with open(os.path.join(rst.path2Result, + '{0:s}_single_core_perf.json'.format( + self.nic)), 'w') as fp: + json.dump(json_obj, fp) + self.verify("FAIL" not in status_result, "Excessive gap between test results and expectations") + + def set_fields(self): + """ + set ip protocol field behavior + """ + fields_config = {'ip': {'src': {'action': 'random'}, }, } + return fields_config + + def tear_down(self): + """ + Run after each test case. + """ + self.destroy_vf_env() + + def tear_down_all(self): + """ + Run after each test suite. + """ + self.dut.kill_all()