From patchwork Mon Aug 9 12:40:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 96731 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E5CFA0C4C; Mon, 9 Aug 2021 14:41:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 12AEE4068A; Mon, 9 Aug 2021 14:41:08 +0200 (CEST) Received: from mail-ua1-f98.google.com (mail-ua1-f98.google.com [209.85.222.98]) by mails.dpdk.org (Postfix) with ESMTP id BD9914003C for ; Mon, 9 Aug 2021 14:41:06 +0200 (CEST) Received: by mail-ua1-f98.google.com with SMTP id f25so643504uam.1 for ; Mon, 09 Aug 2021 05:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RHPQueSVEZgJ+RhWvCNE8wh1qmtkboPxW6k895Gheg0=; b=eSw5/qUG4gsM2RXbDIavcLRUGGGW3gEPbd3Z+6mcpFCK4HQLrYcF6ouVDfhW51ft1l g9CKpGKH0QQHe2XxkAjKXZgwidxz+nqjRAdVbONUJeh0FxobMzsDPvyMU8Oxizv1+I3s esHvDvg2aAqpqIxqBaPEN0+sDB+0A+B1umBDs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=RHPQueSVEZgJ+RhWvCNE8wh1qmtkboPxW6k895Gheg0=; b=r+ZLSB4+05eMmgWydHCExkYtBy0KfxT9YrzrNlMAfF3TqxZYJp3kDZuD/yRi0gjn0C oecwW/QHylRymD26SsJKOV27NXuxyq271eVRQqL4WUR430mzNw1FCV/iDi+5pHP7IpUe CIIi5ZODmTcpKmakuZjbPrOm3jH266DQ67viWV0ZP3dspwDC+St4fiJOQBMTm3zoOY3f +WjDT7SEx+vPLDUfwN+MDt+f26sDmNSUlFEMQGnLubeGg47bX6lS4YcDMCuwJ08kMSJf u4Bod4gy26LMzSj3MrpwLifX0wAj6T+gMdOOwAyt9sWNCO0yCZNP6AOkyZ9Zr6rBCzQZ UY8Q== X-Gm-Message-State: AOAM531l5Pk3F7MY3TsW9Zqu0GIzPDIvedkvuPYcUqcneI/UWw4aUKtR TFOFg0aZ48Sd5erD8AeFHF+s3ZhiV4xVnYICNXu46QWd+g2eCTcN9C5r3ckwv9tbcGUwHprbwt5 tKstlGqnUFSeJV+qMgyiAYlBQL3BeiQWUS9w50UobjSX6rAl2/LwfdHgY3Jv9ebn97H1kS9Gct1 YJD8RUN9cVRS2VXA== X-Google-Smtp-Source: ABdhPJxzMnDbqxxHWGzC3vKheve8cyx4LYbIlWZ07w36SycAW2NXxwd16XsUgtgqH95qMawat2ffwQ+vNNTl X-Received: by 2002:ab0:906:: with SMTP id w6mr15891094uag.37.1628512866154; Mon, 09 Aug 2021 05:41:06 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id a35sm5063362uax.2.2021.08.09.05.41.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Aug 2021 05:41:06 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1220:6f7f:5992:405a:bfd7]) by postal.iol.unh.edu (Postfix) with ESMTP id 87642605248B; Mon, 9 Aug 2021 08:41:05 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Mon, 9 Aug 2021 08:40:59 -0400 Message-Id: <20210809124101.13719-1-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [dts] [PATCH v2 1/3] framework/test_result: Fix circular import X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard This circular import is normally fine because the debugger module is initialized before test_case is imported, but since that is not necessarily the case with the dependency script, the circular import needed to be removed. Signed-off-by: Owen Hilyard --- framework/test_case.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/framework/test_case.py b/framework/test_case.py index 98b716b9..3d7bc30e 100644 --- a/framework/test_case.py +++ b/framework/test_case.py @@ -33,7 +33,6 @@ A base class for creating DTF test cases. """ import re -import debugger import traceback import signal import time @@ -374,6 +373,10 @@ class TestCase(object): """ Execute all test cases in one suite. """ + + # local import to avoid circular import + import debugger + # prepare debugger rerun case environment if self._enable_debug or self._debug_case: debugger.AliveSuite = self From patchwork Mon Aug 9 12:41:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 96732 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3538A0C52; Mon, 9 Aug 2021 14:41:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 39E09410F4; Mon, 9 Aug 2021 14:41:08 +0200 (CEST) Received: from mail-oi1-f228.google.com (mail-oi1-f228.google.com [209.85.167.228]) by mails.dpdk.org (Postfix) with ESMTP id 6EB5E4003C for ; Mon, 9 Aug 2021 14:41:07 +0200 (CEST) Received: by mail-oi1-f228.google.com with SMTP id h11so8043711oie.0 for ; Mon, 09 Aug 2021 05:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O6l8TXMmofBCMy24pXqjuBVQbtoBuBwbmH1JREBVG8A=; b=R36gCPrnmfmXcyQMm5pAjrPoS+lsjnBiIUBnyBN5uGDhhxC8s/WdmomCnNjPkt814T lO0XBKZlcyWKYRgxLYq3VFPPJlXNaoUKGq3aThY6snIcjP9R0UOeoQw3Q7TvwSoSyiTE tDZpy6qvvm4uoAaiGHmWOXYyyvjsBPswa+6rs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O6l8TXMmofBCMy24pXqjuBVQbtoBuBwbmH1JREBVG8A=; b=R3rSSNO3RaZ1VqsEDiuahWf7oxCmrT0KmSWd81ExkJNsewLNDx2TKTpRBkMGHSuNKK 8ouRECj+O1bCO/NpHve/sb8clbj6Q+xWmSachfF4VtW7EJ5jkFCsaDmWqDwJ6h0s9tD3 3f9YJY277elDxlnhHwWX3xOW3gc8QVbZNzq2THSwtF2vWHvK+NAnGdjNOEs1HJVW91Jq jCuPUgRVmK5b7zVeG8buyFyqu11N6DKUspyH89PkgBSrLe4J40MXwIy4TSLm/uzxdICE TqwCMYwPahOUhpEzK87dCy+CeF/Rj8Wkgty6RnSnROPuXDMLnCOXTviCzusPQCqmOgyp s0Sw== X-Gm-Message-State: AOAM5330j0GficI8lSUD3kye2LTjc+LpgdQrg3X6VMQa99UhfAa+7qYr NyNTMk6nCQ31EynjMeQ2mplqMpcxUl/bHzQqVouSpe6ajT4T0TJdK3Vk+sYyEm/3HiLna5lj4fR CKfNb3E2wxKPcrV22s1rjkIYSc3nZ9B9KDqz6EDVXu/NsoRRKkoXBUfMSyEyiVYpkIoeSJjzRWA == X-Google-Smtp-Source: ABdhPJzPKkVUMoxT1xw0pvMRHoGgGSHKd0jEEEyVbTxrp3R+Io2ln+49589U2TvOMq166fVVHF6Y2Rj9P5bB X-Received: by 2002:aca:4007:: with SMTP id n7mr15191783oia.58.1628512866784; Mon, 09 Aug 2021 05:41:06 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id a10sm3477216oon.20.2021.08.09.05.41.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Aug 2021 05:41:06 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1220:6f7f:5992:405a:bfd7]) by postal.iol.unh.edu (Postfix) with ESMTP id 370F4605248E; Mon, 9 Aug 2021 08:41:06 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Mon, 9 Aug 2021 08:41:00 -0400 Message-Id: <20210809124101.13719-2-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210809124101.13719-1-ohilyard@iol.unh.edu> References: <20210809124101.13719-1-ohilyard@iol.unh.edu> MIME-Version: 1.0 Subject: [dts] [PATCH v2 2/3] requirements: Added required modules to requirements X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard Since the dependency script imports every module it looks at, it found a few requirements that are not listed. Signed-off-by: Owen Hilyard --- requirements.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/requirements.txt b/requirements.txt index fae0ace8..d88ccd26 100644 --- a/requirements.txt +++ b/requirements.txt @@ -38,3 +38,6 @@ pcapy xlrd scapy==2.4.4 threadpool +matplotlib +gitpython +pydes \ No newline at end of file From patchwork Mon Aug 9 12:41:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Owen Hilyard X-Patchwork-Id: 96733 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6D34FA0C4C; Mon, 9 Aug 2021 14:41:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5FAD3410ED; Mon, 9 Aug 2021 14:41:10 +0200 (CEST) Received: from mail-pl1-f228.google.com (mail-pl1-f228.google.com [209.85.214.228]) by mails.dpdk.org (Postfix) with ESMTP id DC78F4003C for ; Mon, 9 Aug 2021 14:41:08 +0200 (CEST) Received: by mail-pl1-f228.google.com with SMTP id j3so16183345plx.4 for ; Mon, 09 Aug 2021 05:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CxIH8N0S5Iv7TQ2jMsDU96RQZzEVgCAHoLsAhKrnmKU=; b=f6uWIVSeJg2eGKCo3WdHRQ16mw9L6InhQVFZ/fgNkXdi18FMK4JM7i3TocXvOCReqW pd6G9tGl0j/OajZ7XWr49oyoiHdT79Jamz50TeyQRgRqRIMPEeyDBUWD3bD8a6wwKrzb NBNybPXu5bgEYb2jOfFV0dVqpljc0ph+3eo/s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CxIH8N0S5Iv7TQ2jMsDU96RQZzEVgCAHoLsAhKrnmKU=; b=YZ3FtfJVKnSOBANsVBV19j+VzeWBKO3Bo567Vw5ugurmbv/XD0Q7s4gpAxcYJHgi9D oP5cj6Yb+WNwAuoM9DD6pa5k2Zy1nbOMbRaN/0ycM+YqRfGGc/J+u1MfoBYcWgfFoUfF 6WOFmcSOrEg6pRYEpLvryyeE7AZSQGnK8OKtIcLZ09D9TjsT6l81/+VAa1WhC+hGLcVm AuWY/rdKItpf8HsCFjI42RlvreDzXS/mHFc/eS8cBiUMJeXtp6+i+p/dgfp87m6UeuVI w9LAFHX/C6wuoZBQotjoo2IlRXWxGq0gvfWevTobHX2FOle+IQQUos/MMa9+60ojmxr9 14oA== X-Gm-Message-State: AOAM533tum8qs0kX+3j9l8I3jzMHuXsZRmAG7kae60keNK71eamXeBy3 sFHpitMeSatEoCZMPplBYvikxQjH6uFoZOAXFsIYV+s/HMa+FCCUrrrS0XFAqvX4zVCDsSccjve f75nDUu2aSm02AZ4R0wedNXjCkIOZNNHrRzLwV6SgKC5Kre0FLv4eJt6z06jP/zG2lKQI4GepXg 250GRrf+3nfzBcIQ== X-Google-Smtp-Source: ABdhPJxQLHcntc3bCTZarhVSgAnvUjSOEi6KHa4/n9l0Q2HaKStVQeh1tzSuCSf+jnU/Xn5dvfdXBKzpg5zQ X-Received: by 2002:a17:90a:b009:: with SMTP id x9mr8582087pjq.97.1628512867856; Mon, 09 Aug 2021 05:41:07 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id ce8sm1475141pjb.10.2021.08.09.05.41.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Aug 2021 05:41:07 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1220:6f7f:5992:405a:bfd7]) by postal.iol.unh.edu (Postfix) with ESMTP id DF159605248B; Mon, 9 Aug 2021 08:41:06 -0400 (EDT) From: ohilyard@iol.unh.edu To: dts@dpdk.org Cc: lijuan.tu@intel.com, Owen Hilyard Date: Mon, 9 Aug 2021 08:41:01 -0400 Message-Id: <20210809124101.13719-3-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210809124101.13719-1-ohilyard@iol.unh.edu> References: <20210809124101.13719-1-ohilyard@iol.unh.edu> MIME-Version: 1.0 Subject: [dts] [PATCH v2 3/3] ci/initial: Added script to get the tests for a patchset X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" From: Owen Hilyard This script should be run after the patchset has been applied. It will check all files that have a diff to the git ref in DTS_MAIN_BRANCH_REF (currently origin/master). It will also issue warnings to standard error if a "protected path" is changed. This is currently configured to only by the ci scripts folder, since under most circumstances a patch should not need to change anything in there. This warning will be in the format: "WARNING: {file_name} is protected" The script will also issue a warning if a config file is changed. This warning is also sent to standard error and takes the form of: "WARNING: {file_name} is a config file and was changed" The script will output a list of the test suites to run to standard out, with each entry having one line. Signed-off-by: Owen Hilyard --- ci/README.txt | 37 +++++++ ci/get_tests_for_patchset.py | 195 +++++++++++++++++++++++++++++++++++ ci/requirements.txt | 33 ++++++ 3 files changed, 265 insertions(+) create mode 100644 ci/README.txt create mode 100644 ci/get_tests_for_patchset.py create mode 100644 ci/requirements.txt diff --git a/ci/README.txt b/ci/README.txt new file mode 100644 index 00000000..281329f7 --- /dev/null +++ b/ci/README.txt @@ -0,0 +1,37 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +This directory contains scripts used for ci in DTS. Nothing in this directory +should be modified during the course of submitting a patch to DPDK. + +Additional python requirements only needed for running DTS in ci are present in the +requirements.txt file in this directory. + diff --git a/ci/get_tests_for_patchset.py b/ci/get_tests_for_patchset.py new file mode 100644 index 00000000..fdcf2371 --- /dev/null +++ b/ci/get_tests_for_patchset.py @@ -0,0 +1,195 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +import argparse +import os +import pkgutil +import re +import subprocess +import sys +from pkgutil import walk_packages +from types import ModuleType +from typing import List, Iterable + +DTS_MAIN_BRANCH_REF: str = "origin/master" + +DTS_TEST_MODULE_PATH: str = "tests" +# Will be unnecessary once DTS moves to a normal module structure +DTS_MODULE_PATHS: List[str] = ["framework", "nic", "dep", DTS_TEST_MODULE_PATH] + +# This is primarily intended for folders which contain CI scripts, as these +# should not be modified in the course of normal CI. A file in any of these +# folders being modified will cause a warning to be emitted to standard error +DTS_PROTECTED_PATHS: List[str] = [ + "ci", +] + +# This is intended to help detect when a config files have been changed. +# The intention behind this detection is to enable additional regression +# tests related to ensuring stable config file formats +DTS_CONFIG_PATHS: List[str] = [ + "conf", + "execution.cfg", +] + + +def get_args() -> str: + parser: argparse.ArgumentParser = argparse.ArgumentParser( + description="After a patchset is applied, run this script" + "It will then output a list " + "of applicable test suites to standard output, with 1 " + "test suite per line. All other output, such as warnings," + " errors or informational messages will be sent to " + "standard error. This script may produce no output at all," + "in which case it should be assumed that there are no " + "applicable test suites.\n\n " + "Exit Codes:\n" + "value | meaning\n" + "----- | -------\n" + " 0 | OK \n" + " 1 | Error, check standard error", + ) + + dts_directory: str = os.path.dirname(os.path.dirname(os.path.join(os.getcwd(), __file__))) + parser.add_argument("-d", "--dts-directory", type=str, default=dts_directory, required=False) + args = parser.parse_args() + if not os.path.isdir(args.dts_directory): + print(f"{args.dts_directory} is not a directory.", file=sys.stderr) + exit(1) + + return args.dts_directory + + +def get_modified_files() -> List[str]: + cmd = ['git', 'diff', '--name-only', DTS_MAIN_BRANCH_REF, 'HEAD'] + process: subprocess.CompletedProcess = subprocess.run(cmd, capture_output=True) + if process.returncode != 0: + print(f"{' '.join(cmd)} returned {process.returncode}") + exit(1) + + # This explicit conversion to utf8 will catch encoding errors + stdout: bytes = process.stdout + stdout_str: str = stdout.decode("utf-8") + + return stdout_str.splitlines() + + +def get_names_of_modified_python_files(files: List[str]) -> List[str]: + return list( + map( + lambda f: str(re.sub("\\.py", "", f)), + map( + lambda f: os.path.basename(f), + filter( + lambda f: f.endswith(".py"), files + ) + ) + ) + ) + + +def get_modules() -> List[ModuleType]: + return list(map(lambda m: pkgutil.resolve_name(m.name), walk_packages(DTS_MODULE_PATHS))) + + +def get_module_imports(mod: ModuleType) -> Iterable[ModuleType]: + imports = [] + for attribute_label in dir(mod): + try: + attribute = getattr(mod, attribute_label) + except ModuleNotFoundError as _: # some standard library modules don't like being directly imported + continue + if isinstance(attribute, type(mod)): + imports.append(attribute) + return imports + + +def get_modules_in_tree(mod: ModuleType) -> Iterable[ModuleType]: + yield from get_module_imports(mod) + + +def get_only_test_suites(modules: Iterable[ModuleType]) -> Iterable[ModuleType]: + test_package_path = os.path.join(os.getcwd(), DTS_TEST_MODULE_PATH) + mod: ModuleType + return filter(lambda mod: mod.__name__.startswith("TestSuite_"), + filter(lambda mod: mod.__file__.startswith(test_package_path), modules)) + + +def get_test_suite_names(modules: Iterable[ModuleType]) -> Iterable[str]: + # Moving this into a set is there to ensure that there are no duplicates + return set(list(map(lambda mod: re.sub("TestSuite_", "", mod.__name__), modules))) + + +def get_tests_to_run() -> List[str]: + dts_directory: str + dts_directory = get_args() + + # This all needs to be done at the top level, so I have to do it here. + + # chdir to the DTS directory, since we want that to be + # the context for any commands that are run. + os.chdir(dts_directory) + + for path in DTS_MODULE_PATHS: + sys.path.append(os.path.join(dts_directory, path)) + + files: List[str] = get_modified_files() + changed_module_name = get_names_of_modified_python_files(files) + + for protected_folder in DTS_PROTECTED_PATHS: + for file_name in files: + if file_name.startswith(protected_folder): + print(f"WARNING: {file_name} is protected", file=sys.stderr) + + for config_file_path in DTS_CONFIG_PATHS: + for file_name in files: + if file_name.startswith(config_file_path): + print(f"WARNING: {file_name} is a config file and was changed", file=sys.stderr) + + # Each index is 1 level of the tree + module_list: List[ModuleType] = [pkgutil.resolve_name(name) for name in changed_module_name] + current_index: int = 0 + while current_index < len(module_list) and len(module_list) > 0: + mod = module_list[current_index] + if module_list.count(mod) == 1: + module_list = module_list + list(get_modules_in_tree(mod)) + current_index += 1 + + test_suites_to_run: List[str] = list(get_test_suite_names(get_only_test_suites(module_list))) + test_suites_to_run.sort() + return test_suites_to_run + +def main(): + test_suites_to_run = get_tests_to_run() + print("\n".join(test_suites_to_run)) + + +if __name__ == "__main__": + main() diff --git a/ci/requirements.txt b/ci/requirements.txt new file mode 100644 index 00000000..92d57c30 --- /dev/null +++ b/ci/requirements.txt @@ -0,0 +1,33 @@ +# BSD LICENSE +# +# Copyright(c) 2021 University of New Hampshire Interoperability Laboratory. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +argparse==1.4.0 +python>=3.8.0 \ No newline at end of file