From patchwork Fri Aug 2 12:44:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 1190 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7F5F94571A; Fri, 2 Aug 2024 14:44:24 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6F2F940E20; Fri, 2 Aug 2024 14:44:24 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) by mails.dpdk.org (Postfix) with ESMTP id 3F860402D3 for ; Fri, 2 Aug 2024 14:44:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1722602663; x=1754138663; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LN+RzowtW8uT6ZbN0MbwlJUHSo/Me36fb/slX3KXDkU=; b=L78ec0hh4QdCjn1zjl9wiVFiGepEg209KeRwOywKcu6LN2FFDgbt7Z+a wx6riso8tXS2lvwDphmib4+tsMHyVEPrCJvjVzrVNhH3j0qfZc54g3EBb u8fLISfd/qi9agTvX70+ysrPOzvZecIf+Ih7lRUgx3jlB1BKlwjESP6Jj DFq7O+BvH6vp3toxeKLApwUitiURxTPOqEbCAcJEawg3OIHMsiJ9rRmz5 djp42lOAU67SXHcyRsw0Kp/9LIxKO+YJfU+jzG80tSPTfhSxzc3CcVrx3 p2aI1WWAYLWwWco/DI8noK095d02ElkuuXOqL++i2euw+h+LiKjsRLrmX g==; X-CSE-ConnectionGUID: CPL5+F8eT7OswEeQh60NoQ== X-CSE-MsgGUID: 22mcrg1dTNe7ZUTjFxRPmw== X-IronPort-AV: E=McAfee;i="6700,10204,11152"; a="20499754" X-IronPort-AV: E=Sophos;i="6.09,257,1716274800"; d="scan'208";a="20499754" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2024 05:44:22 -0700 X-CSE-ConnectionGUID: 4Ty8g/1dTEGc1XQWjeNllQ== X-CSE-MsgGUID: nPWwK0ecQ2m/krjyEbXbBQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,257,1716274800"; d="scan'208";a="56149886" Received: from silpixa00401385.ir.intel.com ([10.237.214.25]) by orviesa008.jf.intel.com with ESMTP; 02 Aug 2024 05:44:20 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: ferruh.yigit@amd.com, konstantin.ananyev@huawei.com, anatoly.burakov@intel.com, Bruce Richardson Subject: [PATCH v2 0/7] record and rework component dependencies Date: Fri, 2 Aug 2024 13:44:04 +0100 Message-ID: <20240802124411.485430-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730145508.551075-1-bruce.richardson@intel.com> References: <20240730145508.551075-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org As part of the meson build, we can record the dependencies for each component as we process it, logging them to a file. This file can be used as input to a number of other scripts and tools, for example, to graph the dependencies, or to allow higher-level build-config tools to automatically enable component requirements, etc. The first patch of this set generates the basic dependency tree. The second patch does some processing of that dependency tree to identify cases where dependencies are being unnecessarily specified. Reducing these makes it easier to have readable dependency graphs in future, without affecting the build. The following 4 patches are based on the output of the second patch, and greatly cut down the number of direct dependency links between components. Even with the cut-down dependencies, the full dependency graph is nigh-unreadable, so the final patch adds a new script to generate dependency tree subgraphs, creating dot files for e.g. the dependencies of a particular component, or a component class such as mempool drivers. Bruce Richardson (7): build: output a dependency log in build directory devtools: add script to flag unneeded dependencies build: remove kvargs from driver class dependencies build: reduce library dependencies build: reduce driver dependencies build: reduce app dependencies devtools: add script to generate DPDK dependency graphs app/dumpcap/meson.build | 2 +- app/graph/meson.build | 2 +- app/meson.build | 1 + app/pdump/meson.build | 2 +- app/proc-info/meson.build | 2 +- app/test-crypto-perf/meson.build | 2 +- app/test-fib/meson.build | 2 +- app/test-sad/meson.build | 2 +- app/test/meson.build | 2 +- buildtools/log-deps.py | 43 +++++++ buildtools/meson.build | 2 + devtools/draw-dependency-graphs.py | 136 +++++++++++++++++++++ devtools/find-duplicate-deps.py | 53 ++++++++ drivers/baseband/fpga_5gnr_fec/meson.build | 2 +- drivers/baseband/fpga_lte_fec/meson.build | 2 +- drivers/baseband/la12xx/meson.build | 2 +- drivers/baseband/null/meson.build | 2 +- drivers/baseband/turbo_sw/meson.build | 2 +- drivers/bus/auxiliary/meson.build | 2 - drivers/bus/dpaa/meson.build | 2 +- drivers/bus/fslmc/meson.build | 2 +- drivers/bus/ifpga/meson.build | 2 +- drivers/bus/pci/meson.build | 4 +- drivers/bus/platform/meson.build | 1 - drivers/bus/uacce/meson.build | 2 - drivers/bus/vdev/meson.build | 2 - drivers/common/cnxk/meson.build | 4 +- drivers/common/cpt/meson.build | 2 +- drivers/common/idpf/meson.build | 2 +- drivers/common/mlx5/meson.build | 2 +- drivers/compress/mlx5/meson.build | 2 +- drivers/compress/nitrox/meson.build | 2 +- drivers/compress/octeontx/meson.build | 2 +- drivers/crypto/bcmfs/meson.build | 2 +- drivers/crypto/cnxk/meson.build | 2 +- drivers/crypto/dpaa_sec/meson.build | 2 +- drivers/crypto/ipsec_mb/meson.build | 2 +- drivers/crypto/mlx5/meson.build | 2 +- drivers/crypto/nitrox/meson.build | 2 +- drivers/dma/cnxk/meson.build | 2 +- drivers/dma/dpaa/meson.build | 2 +- drivers/dma/dpaa2/meson.build | 2 +- drivers/dma/odm/meson.build | 2 +- drivers/dma/skeleton/meson.build | 2 +- drivers/event/cnxk/meson.build | 2 +- drivers/event/dlb2/meson.build | 2 +- drivers/event/dpaa2/meson.build | 2 +- drivers/event/meson.build | 2 +- drivers/event/octeontx/meson.build | 3 +- drivers/event/sw/meson.build | 2 +- drivers/mempool/cnxk/meson.build | 2 +- drivers/mempool/dpaa/meson.build | 2 +- drivers/mempool/dpaa2/meson.build | 2 +- drivers/mempool/octeontx/meson.build | 2 +- drivers/meson.build | 1 + drivers/net/cnxk/meson.build | 3 +- drivers/net/iavf/meson.build | 2 +- drivers/net/ice/meson.build | 2 +- drivers/net/mana/meson.build | 2 +- drivers/net/meson.build | 2 +- drivers/net/mlx5/meson.build | 2 +- drivers/net/sfc/meson.build | 2 +- drivers/net/softnic/meson.build | 2 +- drivers/raw/cnxk_bphy/meson.build | 2 +- drivers/raw/cnxk_gpio/meson.build | 2 +- drivers/raw/ntb/meson.build | 2 +- drivers/raw/skeleton/meson.build | 2 +- drivers/regex/meson.build | 2 +- drivers/regex/mlx5/meson.build | 2 +- drivers/vdpa/ifc/meson.build | 2 +- drivers/vdpa/meson.build | 3 +- drivers/vdpa/mlx5/meson.build | 2 +- drivers/vdpa/sfc/meson.build | 2 +- lib/argparse/meson.build | 2 +- lib/bbdev/meson.build | 2 +- lib/bitratestats/meson.build | 2 +- lib/bpf/meson.build | 2 +- lib/cmdline/meson.build | 2 +- lib/compressdev/meson.build | 2 +- lib/cryptodev/meson.build | 2 +- lib/dispatcher/meson.build | 2 +- lib/distributor/meson.build | 2 +- lib/dmadev/meson.build | 2 - lib/eal/meson.build | 5 +- lib/efd/meson.build | 2 +- lib/ethdev/meson.build | 2 +- lib/eventdev/meson.build | 3 +- lib/fib/meson.build | 2 +- lib/gpudev/meson.build | 2 +- lib/graph/meson.build | 2 +- lib/gro/meson.build | 2 +- lib/gso/meson.build | 2 +- lib/hash/meson.build | 4 +- lib/ip_frag/meson.build | 2 +- lib/ipsec/meson.build | 2 +- lib/kvargs/meson.build | 2 +- lib/latencystats/meson.build | 2 +- lib/lpm/meson.build | 3 +- lib/mbuf/meson.build | 2 +- lib/member/meson.build | 2 +- lib/mempool/meson.build | 2 +- lib/meson.build | 1 + lib/metrics/meson.build | 2 +- lib/mldev/meson.build | 2 +- lib/net/meson.build | 2 +- lib/node/meson.build | 2 +- lib/pcapng/meson.build | 2 +- lib/pdcp/meson.build | 2 +- lib/pdump/meson.build | 2 +- lib/pipeline/meson.build | 2 +- lib/port/meson.build | 2 +- lib/power/meson.build | 2 +- lib/rawdev/meson.build | 2 - lib/rcu/meson.build | 2 +- lib/regexdev/meson.build | 2 +- lib/reorder/meson.build | 2 +- lib/rib/meson.build | 2 +- lib/ring/meson.build | 1 - lib/sched/meson.build | 2 +- lib/security/meson.build | 2 +- lib/table/meson.build | 2 +- lib/telemetry/meson.build | 2 +- lib/vhost/meson.build | 2 +- 123 files changed, 347 insertions(+), 134 deletions(-) create mode 100644 buildtools/log-deps.py create mode 100755 devtools/draw-dependency-graphs.py create mode 100755 devtools/find-duplicate-deps.py --- 2.43.0