From patchwork Wed Jul 12 20:33:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ke Xu X-Patchwork-Id: 129512 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B462B42E58; Wed, 12 Jul 2023 22:33:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AE05F406BA; Wed, 12 Jul 2023 22:33:30 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 0EB8D400D5 for ; Wed, 12 Jul 2023 22:33:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1689194010; x=1720730010; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=iGX7iG+p9Wm//+6gn8F1QY+qtWt3DJWDfGAtLMkShUc=; b=aahnK/Q8Pyh+6HpVynoOCGvgxnklTYgP6w89oaHrea/QSCovrH01fKEi PC/tGv5E8pMQghOqBLYyCdwRKhjHsJxq9J+aKWKXJJHnWhRS+vSI6uUW3 1H0VluAdhpCIj3rZ5eN0HJnbczpwPnWHSGcIczqPgPmqhevkXXmaa8H4Q WH6b2sTvjUB0N1oZZ/ADeiW7PTQ+T0idHv4krWN1ngxH/01ADurMFePnD 1R1nX4/VClCavhrvYJy/+0p2ezLLL1T7VeAOYvdtSfPz9CzMRPsjEK0Kd 5nYwg1ua3uTpX1Heegzmx+rnuNpD25ZuBCyDfdPaGR6BWp2Kp+2Kw1klK w==; X-IronPort-AV: E=McAfee;i="6600,9927,10769"; a="362467038" X-IronPort-AV: E=Sophos;i="6.01,200,1684825200"; d="scan'208";a="362467038" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jul 2023 13:33:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10769"; a="715696949" X-IronPort-AV: E=Sophos;i="6.01,200,1684825200"; d="scan'208";a="715696949" Received: from dpdk-xuke-host.sh.intel.com ([10.67.114.220]) by orsmga007.jf.intel.com with ESMTP; 12 Jul 2023 13:33:26 -0700 From: Ke Xu To: dts@dpdk.org Cc: ke1.xu@intel.com, tarcadia@qq.com Subject: [DTS][Patch V1 0/2] Rewrite test suite for checksum_offload Date: Wed, 12 Jul 2023 20:33:32 +0000 Message-Id: <20230712203334.2032445-1-ke1.xu@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Checksum offload cases were in bad situation. Cases are not well organized in structure. Cases were added one by one along with the development of features. The very beginning cases are designed to verifying RX Checksum Offload enabling / disabling. The followed cases are to verify the TX Offload, and found a gap when TX introduces differences to IP and L4. Then IP and L4 RX cases are added again. There were also a gap found on the flag verifying and a seperate ol_flag case was added. To better organizing the checksum offload cases. I introduced the scenario based common cases and specific cases named with sufix to cover specific scenarios of special requirements. And we placed all available test scopes introduced in any existing cases to each of our common test execution to ensure no leakage to our test scopes. For scenarios, we introduce the very commonly used configurations for checksum as our test scenarios. These scenarios cover combined usage of HW checksum Offload, SW checksum and differently configured tunneling scenario, w/wo VLAN. These cases formed a organized web for future cases that if a new configuration is introduced, cases of new configuration combinations are to be added but existing configurations for different scopes are not to be added. For test scopes, we introduce verifying ol_flag, fwd stats, other verbose lines, and forwarded packet capturing and analyzing. All these scopes are recorder for verifying in each common checksum offload execution. This ensures no scope would be missed in this rewrite and in future case updating. The newly organized cases also helps better covering our existing test requirements. A common packet sending and recording helps cut down the repeating time cost of different executions for scope based redundant cases. This rewrite also helps formating a data-process-seperated suite structure. The test parameters and required packets are listed in front and cases are referring to the configured parameters. This helps better reading the suite code. In the test plan, new cases are added and old ones are merged with the new cases. Packets, ol_flags, stats and related test scopes are listed in table to show where the expected result is from. Generally speaking, this case rewrite helps formating the case structure and helps better maintaining the cases in future. Ke Xu (2): test_plans/checksum_offload: Refactor checksum_offload case. tests/checksum_offload: Refactor checksum_offload case. test_plans/checksum_offload_test_plan.rst | 1478 +++++++---------- tests/TestSuite_checksum_offload.py | 1766 +++++++++++---------- 2 files changed, 1549 insertions(+), 1695 deletions(-)