Message ID | 20210402142424.1353789-1-ciara.power@intel.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C905A0548; Fri, 2 Apr 2021 16:24:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EFF104069E; Fri, 2 Apr 2021 16:24:42 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 2851E40142 for <dev@dpdk.org>; Fri, 2 Apr 2021 16:24:40 +0200 (CEST) IronPort-SDR: 8MdwRGPQ/f42C9UkzAwX/rrvNXM1LH+FApLvE56jR/SJfcp8mOlE0OIEOxMCXKf3YpV/N+ozrJ P0gI5QOB0g8Q== X-IronPort-AV: E=McAfee;i="6000,8403,9942"; a="192512441" X-IronPort-AV: E=Sophos;i="5.81,300,1610438400"; d="scan'208";a="192512441" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2021 07:24:39 -0700 IronPort-SDR: tt4d4CVvR54hQzsBp++8Yk0mMChcTP0zkzffOyn0LhJy67+wpnAHMKC8ZOvHhKUCjfDDNtOp3W slhvEsPurmrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,300,1610438400"; d="scan'208";a="413199549" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.223.148]) by fmsmga008.fm.intel.com with ESMTP; 02 Apr 2021 07:24:36 -0700 From: Ciara Power <ciara.power@intel.com> To: dev@dpdk.org Cc: declan.doherty@intel.com, gakhil@marvell.com, aconole@redhat.com, hemant.agrawal@nxp.com, anoobj@marvell.com, ruifeng.wang@arm.com, asomalap@amd.com, ajit.khaparde@broadcom.com, g.singh@nxp.com, Ciara Power <ciara.power@intel.com> Date: Fri, 2 Apr 2021 14:24:18 +0000 Message-Id: <20210402142424.1353789-1-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2 0/6] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
test: refactor crypto unit test framework
|
|
Message
Power, Ciara
April 2, 2021, 2:24 p.m. UTC
v2: - Added macro in place of testcase/testsuite loops. - Added more detail in the summary output. - Moved testcase counts to the testsuite structure. - Flattened testsuite structure to remove union. - Added patch for fix of blockcipher test return value. - Squashed release note into last patch. The current crypto unit test framework is not granular enough to accurately track unit test results. This is caused by one testcase in a suite actually running multiple testcases, but only returning one result. The approach taken in this patchset allows a test suite have either a list of sub-testsuites, or a list of testcases as previously used. The unit test suite runner can then recursively iterate and run the sub-testsuites, until it reaches a suite with testcases, and it then runs each testcase as it had done previously. By allowing this further breakdown into sub-testsuites, a refactor of the crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites can be used in place of the testcases that had multiple testcases hidden on a sub level. The blockcipher tests previously had these hidden testcases, but are now sub-testsuites that are dynamically created and added to a parent test suite, allowing for each testcase status to be reported directly to the runner. The cryptodev test suite is broken down into smaller suites that are used as sub-testsuites, which allows for more flexibility choosing which sub-testsuites should run for the current device autotest. The introduction of sub-testsuites also allows for more precise setup/teardown functions, rather than general ones loaded with conditions as was seen with the initial setup function used for all crypto testsuites. For example, when running the cryptodev_aesni_mb_autotest, the AESNI MB parent test suite has its own setup function to initialise the AESNI MB device. Various sub-testsuites are added to the parent test suite, such as some of the static suites that were once in the cryptodev_testsuite, and blockcipher suites. The unit test runner can then run the AESNI MB parent test suite, which in turn will run the sub-testsuites. Documentation will be added in a later version of the patchset, adding to the test document that isn't yet merged. [1] --- [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ Ciara Power (6): app/test: refactor of unit test suite runner test: introduce parent testsuite format test/crypto: refactor to use sub-testsuites test/crypto: move testsuite params to header file test/crypto: fix return value on test skipped test/crypto: dynamically build blockcipher suite app/test/test.c | 226 ++- app/test/test.h | 23 +- app/test/test_cryptodev.c | 1964 ++++++++++++++++++------ app/test/test_cryptodev.h | 20 + app/test/test_cryptodev_asym.c | 93 +- app/test/test_cryptodev_blockcipher.c | 127 +- app/test/test_cryptodev_blockcipher.h | 12 +- app/test/test_ipsec.c | 32 +- doc/guides/rel_notes/release_21_05.rst | 5 + 9 files changed, 1831 insertions(+), 671 deletions(-)
Comments
> -----Original Message----- > From: Ciara Power <ciara.power@intel.com> > Sent: Friday, April 2, 2021 10:24 PM > To: dev@dpdk.org > Cc: declan.doherty@intel.com; gakhil@marvell.com; aconole@redhat.com; > hemant.agrawal@nxp.com; anoobj@marvell.com; Ruifeng Wang > <Ruifeng.Wang@arm.com>; asomalap@amd.com; Ajit Khaparde > (ajit.khaparde@broadcom.com) <ajit.khaparde@broadcom.com>; > g.singh@nxp.com; Ciara Power <ciara.power@intel.com> > Subject: [PATCH v2 0/6] test: refactor crypto unit test framework > > v2: > - Added macro in place of testcase/testsuite loops. > - Added more detail in the summary output. > - Moved testcase counts to the testsuite structure. > - Flattened testsuite structure to remove union. > - Added patch for fix of blockcipher test return value. > - Squashed release note into last patch. > > The current crypto unit test framework is not granular enough to accurately > track unit test results. This is caused by one testcase in a suite actually > running multiple testcases, but only returning one result. > > The approach taken in this patchset allows a test suite have either a list of > sub-testsuites, or a list of testcases as previously used. > The unit test suite runner can then recursively iterate and run the sub- > testsuites, until it reaches a suite with testcases, and it then runs each > testcase as it had done previously. > > By allowing this further breakdown into sub-testsuites, a refactor of the > crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites > can be used in place of the testcases that had multiple testcases hidden on a > sub level. > The blockcipher tests previously had these hidden testcases, but are now > sub-testsuites that are dynamically created and added to a parent test suite, > allowing for each testcase status to be reported directly to the runner. > The cryptodev test suite is broken down into smaller suites that are used as > sub-testsuites, which allows for more flexibility choosing which sub- > testsuites should run for the current device autotest. > The introduction of sub-testsuites also allows for more precise > setup/teardown functions, rather than general ones loaded with conditions > as was seen with the initial setup function used for all crypto testsuites. > > For example, when running the cryptodev_aesni_mb_autotest, the AESNI > MB parent test suite has its own setup function to initialise the AESNI MB > device. > Various sub-testsuites are added to the parent test suite, such as some of > the static suites that were once in the cryptodev_testsuite, and blockcipher > suites. > The unit test runner can then run the AESNI MB parent test suite, which in > turn will run the sub-testsuites. > > Documentation will be added in a later version of the patchset, adding to the > test document that isn't yet merged. [1] > > --- > [1] > https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1- > aconole@redhat.com/ > > Ciara Power (6): > app/test: refactor of unit test suite runner > test: introduce parent testsuite format > test/crypto: refactor to use sub-testsuites > test/crypto: move testsuite params to header file > test/crypto: fix return value on test skipped > test/crypto: dynamically build blockcipher suite > > app/test/test.c | 226 ++- > app/test/test.h | 23 +- > app/test/test_cryptodev.c | 1964 ++++++++++++++++++------ > app/test/test_cryptodev.h | 20 + > app/test/test_cryptodev_asym.c | 93 +- > app/test/test_cryptodev_blockcipher.c | 127 +- > app/test/test_cryptodev_blockcipher.h | 12 +- > app/test/test_ipsec.c | 32 +- > doc/guides/rel_notes/release_21_05.rst | 5 + > 9 files changed, 1831 insertions(+), 671 deletions(-) > > -- > 2.25.1 Tested against armv8crypto PMD and the result looks good. Tested-by: Ruifeng Wang <ruifeng.wang@arm.com>