From patchwork Mon Dec 4 10:24:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Juraj_Linke=C5=A1?= X-Patchwork-Id: 134794 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B7C54366A; Mon, 4 Dec 2023 11:25:19 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E5B1841101; Mon, 4 Dec 2023 11:24:40 +0100 (CET) Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by mails.dpdk.org (Postfix) with ESMTP id 7CC3340DF5 for ; Mon, 4 Dec 2023 11:24:37 +0100 (CET) Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-40c09f4bea8so9309595e9.1 for ; Mon, 04 Dec 2023 02:24:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1701685477; x=1702290277; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HwU+LFHRLrk+IQzVJORkHjCvcMai8+Em9v6JYz1n0S0=; b=s7SGDW2RSWesA5wtzCjFs2UTLxXeMGfxp4A60QmL5IOUAsaE3ePw/vPBcJIbunOaZx +l3ixCDWWDBJBU9wm9uulbqIqCvFQtwVuNgn1Wt0X4WirEf7f+93g3Nn4ZDZJGouoxc7 5DRLCKlKrZVr8RopQztfTAv2mdp00g/44vx4QC75gGGq09D9ms9rE1wql1lsv45OBMoP q3we0vBMkOwrOMaaqZgmAxkRQzPUpqDmPxBW/lpGa0felFRBw7p0wlPy+cPRF3CkWJvb c6KNL8rG+W+1GZGpFq88z9UuZNl2RrU6ur+KSPP0i1vBLmjAxRJViYjTG9nLUMu79JkE 63lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701685477; x=1702290277; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HwU+LFHRLrk+IQzVJORkHjCvcMai8+Em9v6JYz1n0S0=; b=FpHEFWsLStztLpHpGT7d7grwHo0ui9btg0HRphRsmvG0WNNJKspwuDs9Ky+JOqIGeJ mHFRmbGCB0qUjiU9Ec/dMGOhh9X/76NFOfcrqZfcJ7KM33iTlm5PeX1KuWbVN80zUAPj vRKLTG4rz+8/N2u10MLkIssdZreKTPfNGjF79TwE2LWwEYgJbX21KdEk+mWJrfIDcGqc wIYcprn1NcPJijz9XpKcUFGEhZiOLe6VSxthpmTEIbNearM9g10PHM8bW1z1doRF4i8s YzlkAUFjt8hRZ9C0j9HguZhUpAO9xTKejV/gJidpg/nB6HAIKzoMmbmN6CVTSSIAWfiz lf7A== X-Gm-Message-State: AOJu0Yw+klItUfYzo1T2lg8sjOYc0jV1/FTwiyJiyvaTplvQDLtmox/L sqcOad4W6NOHB9nd6swvIzq7Vg== X-Google-Smtp-Source: AGHT+IH1uVgmitNN/f45TtEPbXCCaRsP+Bt+rTSvE+GJ6LCmiPXQkK97jxPHMCb/yoNKsa5NAVLMpg== X-Received: by 2002:a05:600c:46c7:b0:40b:35aa:bfdd with SMTP id q7-20020a05600c46c700b0040b35aabfddmr2209558wmo.27.1701685477142; Mon, 04 Dec 2023 02:24:37 -0800 (PST) Received: from jlinkes-PT-Latitude-5530.pantheon.local ([81.89.53.154]) by smtp.gmail.com with ESMTPSA id m28-20020a05600c3b1c00b0040b2b38a1fasm14255415wms.4.2023.12.04.02.24.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 02:24:36 -0800 (PST) From: =?utf-8?q?Juraj_Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com, Luca.Vizzarro@arm.com Cc: dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= Subject: [PATCH v9 06/21] dts: logger and utils docstring update Date: Mon, 4 Dec 2023 11:24:14 +0100 Message-Id: <20231204102429.106709-7-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204102429.106709-1-juraj.linkes@pantheon.tech> References: <20231123151344.162812-1-juraj.linkes@pantheon.tech> <20231204102429.106709-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Format according to the Google format and PEP257, with slight deviations. Signed-off-by: Juraj Linkeš --- dts/framework/logger.py | 72 ++++++++++++++++++++++----------- dts/framework/utils.py | 88 +++++++++++++++++++++++++++++------------ 2 files changed, 113 insertions(+), 47 deletions(-) diff --git a/dts/framework/logger.py b/dts/framework/logger.py index bb2991e994..cfa6e8cd72 100644 --- a/dts/framework/logger.py +++ b/dts/framework/logger.py @@ -3,9 +3,9 @@ # Copyright(c) 2022-2023 PANTHEON.tech s.r.o. # Copyright(c) 2022-2023 University of New Hampshire -""" -DTS logger module with several log level. DTS framework and TestSuite logs -are saved in different log files. +"""DTS logger module. + +DTS framework and TestSuite logs are saved in different log files. """ import logging @@ -18,19 +18,21 @@ stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" -class LoggerDictType(TypedDict): - logger: "DTSLOG" - name: str - node: str - +class DTSLOG(logging.LoggerAdapter): + """DTS logger adapter class for framework and testsuites. -# List for saving all using loggers -Loggers: list[LoggerDictType] = [] + The :option:`--verbose` command line argument and the :envvar:`DTS_VERBOSE` environment + variable control the verbosity of output. If enabled, all messages will be emitted to the + console. + The :option:`--output` command line argument and the :envvar:`DTS_OUTPUT_DIR` environment + variable modify the directory where the logs will be stored. -class DTSLOG(logging.LoggerAdapter): - """ - DTS log class for framework and testsuite. + Attributes: + node: The additional identifier. Currently unused. + sh: The handler which emits logs to console. + fh: The handler which emits logs to a file. + verbose_fh: Just as fh, but logs with a different, more verbose, format. """ _logger: logging.Logger @@ -40,6 +42,15 @@ class DTSLOG(logging.LoggerAdapter): verbose_fh: logging.FileHandler def __init__(self, logger: logging.Logger, node: str = "suite"): + """Extend the constructor with additional handlers. + + One handler logs to the console, the other one to a file, with either a regular or verbose + format. + + Args: + logger: The logger from which to create the logger adapter. + node: An additional identifier. Currently unused. + """ self._logger = logger # 1 means log everything, this will be used by file handlers if their level # is not set @@ -92,26 +103,43 @@ def __init__(self, logger: logging.Logger, node: str = "suite"): super(DTSLOG, self).__init__(self._logger, dict(node=self.node)) def logger_exit(self) -> None: - """ - Remove stream handler and logfile handler. - """ + """Remove the stream handler and the logfile handler.""" for handler in (self.sh, self.fh, self.verbose_fh): handler.flush() self._logger.removeHandler(handler) +class _LoggerDictType(TypedDict): + logger: DTSLOG + name: str + node: str + + +# List for saving all loggers in use +_Loggers: list[_LoggerDictType] = [] + + def getLogger(name: str, node: str = "suite") -> DTSLOG: + """Get DTS logger adapter identified by name and node. + + An existing logger will be returned if one with the exact name and node already exists. + A new one will be created and stored otherwise. + + Args: + name: The name of the logger. + node: An additional identifier for the logger. + + Returns: + A logger uniquely identified by both name and node. """ - Get logger handler and if there's no handler for specified Node will create one. - """ - global Loggers + global _Loggers # return saved logger - logger: LoggerDictType - for logger in Loggers: + logger: _LoggerDictType + for logger in _Loggers: if logger["name"] == name and logger["node"] == node: return logger["logger"] # return new logger dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node) - Loggers.append({"logger": dts_logger, "name": name, "node": node}) + _Loggers.append({"logger": dts_logger, "name": name, "node": node}) return dts_logger diff --git a/dts/framework/utils.py b/dts/framework/utils.py index a0f2173949..cc5e458cc8 100644 --- a/dts/framework/utils.py +++ b/dts/framework/utils.py @@ -3,6 +3,16 @@ # Copyright(c) 2022-2023 PANTHEON.tech s.r.o. # Copyright(c) 2022-2023 University of New Hampshire +"""Various utility classes and functions. + +These are used in multiple modules across the framework. They're here because +they provide some non-specific functionality, greatly simplify imports or just don't +fit elsewhere. + +Attributes: + REGEX_FOR_PCI_ADDRESS: The regex representing a PCI address, e.g. ``0000:00:08.0``. +""" + import atexit import json import os @@ -19,12 +29,20 @@ def expand_range(range_str: str) -> list[int]: - """ - Process range string into a list of integers. There are two possible formats: - n - a single integer - n-m - a range of integers + """Process `range_str` into a list of integers. + + There are two possible formats of `range_str`: + + * ``n`` - a single integer, + * ``n-m`` - a range of integers. - The returned range includes both n and m. Empty string returns an empty list. + The returned range includes both ``n`` and ``m``. Empty string returns an empty list. + + Args: + range_str: The range to expand. + + Returns: + All the numbers from the range. """ expanded_range: list[int] = [] if range_str: @@ -37,6 +55,14 @@ def expand_range(range_str: str) -> list[int]: def get_packet_summaries(packets: list[Packet]) -> str: + """Format a string summary from `packets`. + + Args: + packets: The packets to format. + + Returns: + The summary of `packets`. + """ if len(packets) == 1: packet_summaries = packets[0].summary() else: @@ -45,27 +71,36 @@ def get_packet_summaries(packets: list[Packet]) -> str: class StrEnum(Enum): + """Enum with members stored as strings.""" + @staticmethod def _generate_next_value_(name: str, start: int, count: int, last_values: object) -> str: return name def __str__(self) -> str: + """The string representation is the name of the member.""" return self.name class MesonArgs(object): - """ - Aggregate the arguments needed to build DPDK: - default_library: Default library type, Meson allows "shared", "static" and "both". - Defaults to None, in which case the argument won't be used. - Keyword arguments: The arguments found in meson_options.txt in root DPDK directory. - Do not use -D with them, for example: - meson_args = MesonArgs(enable_kmods=True). - """ + """Aggregate the arguments needed to build DPDK.""" _default_library: str def __init__(self, default_library: str | None = None, **dpdk_args: str | bool): + """Initialize the meson arguments. + + Args: + default_library: The default library type, Meson supports ``shared``, ``static`` and + ``both``. Defaults to :data:`None`, in which case the argument won't be used. + dpdk_args: The arguments found in ``meson_options.txt`` in root DPDK directory. + Do not use ``-D`` with them. + + Example: + :: + + meson_args = MesonArgs(enable_kmods=True). + """ self._default_library = f"--default-library={default_library}" if default_library else "" self._dpdk_args = " ".join( ( @@ -75,6 +110,7 @@ def __init__(self, default_library: str | None = None, **dpdk_args: str | bool): ) def __str__(self) -> str: + """The actual args.""" return " ".join(f"{self._default_library} {self._dpdk_args}".split()) @@ -96,24 +132,14 @@ class _TarCompressionFormat(StrEnum): class DPDKGitTarball(object): - """Create a compressed tarball of DPDK from the repository. - - The DPDK version is specified with git object git_ref. - The tarball will be compressed with _TarCompressionFormat, - which must be supported by the DTS execution environment. - The resulting tarball will be put into output_dir. + """Compressed tarball of DPDK from the repository. - The class supports the os.PathLike protocol, + The class supports the :class:`os.PathLike` protocol, which is used to get the Path of the tarball:: from pathlib import Path tarball = DPDKGitTarball("HEAD", "output") tarball_path = Path(tarball) - - Arguments: - git_ref: A git commit ID, tag ID or tree ID. - output_dir: The directory where to put the resulting tarball. - tar_compression_format: The compression format to use. """ _git_ref: str @@ -128,6 +154,17 @@ def __init__( output_dir: str, tar_compression_format: _TarCompressionFormat = _TarCompressionFormat.xz, ): + """Create the tarball during initialization. + + The DPDK version is specified with `git_ref`. The tarball will be compressed with + `tar_compression_format`, which must be supported by the DTS execution environment. + The resulting tarball will be put into `output_dir`. + + Args: + git_ref: A git commit ID, tag ID or tree ID. + output_dir: The directory where to put the resulting tarball. + tar_compression_format: The compression format to use. + """ self._git_ref = git_ref self._tar_compression_format = tar_compression_format @@ -196,4 +233,5 @@ def _delete_tarball(self) -> None: os.remove(self._tarball_path) def __fspath__(self) -> str: + """The os.PathLike protocol implementation.""" return str(self._tarball_path)