From patchwork Tue Jul 18 02:33:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ke Xu X-Patchwork-Id: 129600 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0A6F942EA1; Tue, 18 Jul 2023 04:33:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D4AF6427E9; Tue, 18 Jul 2023 04:33:13 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id B3C3340223 for ; Tue, 18 Jul 2023 04:33:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1689647591; x=1721183591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lb//Gi8tIIpQQZxNHWJ1+oZVimiVm2CBfKc9gRosgtY=; b=IFYM3XIAp1FUgb0WdtO6rfEr/BYtrxK3wsz3tYXCz40oTI6YO98uwaBq gk89Jyf508TS7FBfvtaGNHQIaJFnpkdGOfxYOI/vQQ/as54CY/GJMj25p mejSo1sHVicP+UVOJHFGoZSavypb1QSzcJ5Lhnrg6RXpoHEoLp6jwri/F Xm8sVENEOnnfks8sdeGR6d6yFLXA0XaV6qf6s4PwBj/f/bjLompXkyQMh 5SJEv/j6m1wlQSFPaXrjeNz8AIvQdwAQ6+hwFni2irOHDUcoZAiXQg/do b79H3Ku5YNW523SjfOa4PKAC9/QHSajCjH1n2cmX7eq+mPLbUvtQ6r4HZ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10774"; a="396926976" X-IronPort-AV: E=Sophos;i="6.01,213,1684825200"; d="scan'208";a="396926976" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Jul 2023 19:33:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10774"; a="700722127" X-IronPort-AV: E=Sophos;i="6.01,213,1684825200"; d="scan'208";a="700722127" Received: from dpdk-xuke-host.sh.intel.com ([10.67.114.220]) by orsmga006.jf.intel.com with ESMTP; 17 Jul 2023 19:33:08 -0700 From: Ke Xu To: dts@dpdk.org Cc: ke1.xu@intel.com, tarcadia@qq.com Subject: [DTS][Patch V3 0/6] Rewrite test suite for checksum_offload, tso and vf_offload Date: Tue, 18 Jul 2023 02:33:05 +0000 Message-Id: <20230718023311.2480869-1-ke1.xu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230718012629.2471811-7-ke1.xu@intel.com> References: <20230718012629.2471811-7-ke1.xu@intel.com> MIME-Version: 1.0 X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Checksum offload cases were in bad situation. Cases are not well organized in structure. Cases were added one by one along with the development of features. The very beginning cases are designed to verifying RX Checksum Offload enabling / disabling. The followed cases are to verify the TX Offload, and found a gap when TX introduces differences to IP and L4. Then IP and L4 RX cases are added again. There were also a gap found on the flag verifying and a seperate ol_flag case was added. To better organizing the checksum offload cases. I introduced the scenario based common cases and specific cases named with sufix to cover specific scenarios of special requirements. And we placed all available test scopes introduced in any existing cases to each of our common test execution to ensure no leakage to our test scopes. For scenarios, we introduce the very commonly used configurations for checksum as our test scenarios. These scenarios cover combined usage of HW checksum Offload, SW checksum and differently configured tunneling scenario, w/wo VLAN. These cases formed a organized web for future cases that if a new configuration is introduced, cases of new configuration combinations are to be added but existing configurations for different scopes are not to be added. For test scopes, we introduce verifying ol_flag, fwd stats, other verbose lines, and forwarded packet capturing and analyzing. All these scopes are recorder for verifying in each common checksum offload execution. This ensures no scope would be missed in this rewrite and in future case updating. The newly organized cases also helps better covering our existing test requirements. A common packet sending and recording helps cut down the repeating time cost of different executions for scope based redundant cases. This rewrite also helps formating a data-process-seperated suite structure. The test parameters and required packets are listed in front and cases are referring to the configured parameters. This helps better reading the suite code. In the test plan, new cases are added and old ones are merged with the new cases. Packets, ol_flags, stats and related test scopes are listed in table to show where the expected result is from. Similar modifications are also made to TSO cases. Common TSO executions are introduced and scenarios are designed based on configurations with test scopes covering all cases. Common packets are introduced. In vf_offload, checksum_offload and tso cases are synced with those in checksum_offload and tso suites. During the release DPDK 22.11 and 23.03, several main features are synced from PF to VF. This means the cases and scopes in checksum_offload and tso are available in vf_offload now. This patch also helped introducing these common test execution and scopes to vf_offload suite. Generally speaking, this case rewrite helps formating the cases structure and helps better maintaining the cases in future. Ke Xu (6): test_plans/checksum_offload: Refactor checksum_offload case. tests/checksum_offload: Refactor checksum_offload case. test_plans/tso: Refactor TSO case. tests/tso: Refactor TSO cases. test_plans/vf_offload: Refactor checksum_offload case. tests/vf_offload: Refactor vf_offload cases. test_plans/checksum_offload_test_plan.rst | 1478 ++++------ test_plans/tso_test_plan.rst | 489 +++- test_plans/vf_offload_test_plan.rst | 1356 +++++---- tests/TestSuite_checksum_offload.py | 1790 ++++++------ tests/TestSuite_tso.py | 1100 +++++--- tests/TestSuite_vf_offload.py | 3053 +++++++++++++-------- 6 files changed, 5371 insertions(+), 3895 deletions(-)