From patchwork Thu Mar 23 17:15:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 125465 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C9FC342824; Thu, 23 Mar 2023 18:15:14 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 69E384111C; Thu, 23 Mar 2023 18:15:13 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 4FE4F4111C for ; Thu, 23 Mar 2023 18:15:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679591710; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5itaAo97rceHknvwpgS4KMpZIkjkQEGbYtJqHyaJD9M=; b=al2AOhT0SXYkIKV1Rkbysr8yfKQiDN6xTFd/Joxq1cg0bDwNQld21gnww5wR61/uixq9R8 Y1Sk9gyUanoFqrnAoZfm4VMktUjbj4bhoknb9zYfr6vwHZBqbBEIlK0SgmlEqujLjQ+JR7 j5FY+O6JD6ElT3FSrPLincIps+/m2rM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-524-R2333QZ_NoG4-YJKf6HuSQ-1; Thu, 23 Mar 2023 13:15:08 -0400 X-MC-Unique: R2333QZ_NoG4-YJKf6HuSQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 45D0C886468; Thu, 23 Mar 2023 17:15:08 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.224.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 96DD4140E950; Thu, 23 Mar 2023 17:15:07 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: Thomas Monjalon Subject: [PATCH v2 1/2] devtools: unify configuration for ABI check Date: Thu, 23 Mar 2023 18:15:01 +0100 Message-Id: <20230323171502.3124049-2-david.marchand@redhat.com> In-Reply-To: <20230323171502.3124049-1-david.marchand@redhat.com> References: <20221103154717.22368-1-david.marchand@redhat.com> <20230323171502.3124049-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org We have been skipping removed libraries in the ABI check by updating the check-abi.sh script itself. See, for example, commit 33584c19ddc2 ("raw/dpaa2_qdma: remove driver"). Having two places for exception is a bit confusing, and those exceptions are best placed in a single configuration file out of the check script. Besides, a next patch will switch the check from comparing ABI xml files to directly comparing .so files. In this mode, libabigail does not support the soname_regexp syntax used for the mlx glue libraries. Let's handle these special cases in libabigail.abignore using comments. Taking the raw/dpaa2_qdma driver as an example, it would be possible to skip it by adding: ; SKIP_LIBRARY=librte_net_mlx4_glue +; SKIP_LIBRARY=librte_raw_dpaa2_qdma Signed-off-by: David Marchand --- devtools/check-abi.sh | 9 +++++++-- devtools/libabigail.abignore | 12 +++++++++--- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh index d253a12768..f74432be5d 100755 --- a/devtools/check-abi.sh +++ b/devtools/check-abi.sh @@ -10,7 +10,8 @@ fi refdir=$1 newdir=$2 warnonly=${3:-} -ABIDIFF_OPTIONS="--suppr $(dirname $0)/libabigail.abignore --no-added-syms" +ABIDIFF_SUPPRESSIONS=$(dirname $(readlink -f $0))/libabigail.abignore +ABIDIFF_OPTIONS="--suppr $ABIDIFF_SUPPRESSIONS --no-added-syms" if [ ! -d $refdir ]; then echo "Error: reference directory '$refdir' does not exist." >&2 @@ -34,10 +35,14 @@ else ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2" fi -export newdir ABIDIFF_OPTIONS +export newdir ABIDIFF_OPTIONS ABIDIFF_SUPPRESSIONS export diff_func='run_diff() { dump=$1 name=$(basename $dump) + if grep -q "; SKIP_LIBRARY=${name%.dump}\>" $ABIDIFF_SUPPRESSIONS; then + echo "Skipped $name" >&2 + return 0 + fi dump2=$(find $newdir -name $name) if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then echo "Error: cannot find $name in $newdir" >&2 diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore index 7a93de3ba1..3ff51509de 100644 --- a/devtools/libabigail.abignore +++ b/devtools/libabigail.abignore @@ -16,9 +16,15 @@ [suppress_variable] name_regexp = _pmd_info$ -; Ignore changes on soname for mlx glue internal drivers -[suppress_file] - soname_regexp = ^librte_.*mlx.*glue\. +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; Special rules to skip libraries ; +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +; +; This is not a libabigail rule (see check-abi.sh). +; This is used for driver removal and other special cases like mlx glue libs. +; +; SKIP_LIBRARY=librte_common_mlx5_glue +; SKIP_LIBRARY=librte_net_mlx4_glue ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Experimental APIs exceptions ; From patchwork Thu Mar 23 17:15:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 125466 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50A8242824; Thu, 23 Mar 2023 18:15:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8777042B7E; Thu, 23 Mar 2023 18:15:19 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id CD4A84021E for ; Thu, 23 Mar 2023 18:15:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679591717; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2HLlU1xSAaDLxOp6SHVBo3nDsm1gRFuTiU9FbcFyT4U=; b=I4u3qGjlvaCZt6adQLfD1OQXzNjkNIu+mcWDrQ7ezOCgLj7TywRItl+hCElD2yb6pBIbAU v2Tb+9SQnDDb2TM8AsQxadfpkcGWa85G8QDj3P7ndRlg8piD8WSFtujH4+gfq+jnBpT1rH OvxxkI+09sirvuOGLjwr6k3vmiD3CtM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-596-pbYJ9B88Ph2p8PS-z_K5Ew-1; Thu, 23 Mar 2023 13:15:11 -0400 X-MC-Unique: pbYJ9B88Ph2p8PS-z_K5Ew-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5491E85530C; Thu, 23 Mar 2023 17:15:11 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.224.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D388492B0A; Thu, 23 Mar 2023 17:15:10 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: Aaron Conole , Michael Santana , Thomas Monjalon , Bruce Richardson Subject: [PATCH v2 2/2] devtools: stop depending on libabigail xml format Date: Thu, 23 Mar 2023 18:15:02 +0100 Message-Id: <20230323171502.3124049-3-david.marchand@redhat.com> In-Reply-To: <20230323171502.3124049-1-david.marchand@redhat.com> References: <20221103154717.22368-1-david.marchand@redhat.com> <20230323171502.3124049-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org A ABI reference depends on: - DPDK build options, - toolchain compiler and versions, - libabigail version. The reason for the latter point is that, when the ABI reference was generated, ABI xml files were dumped in a format depending on the libabigail version. Those xml files were then later used to compare against modified code. There are a few disadvantages with this method: - since the xml files are dependent on the libabigail version, when updating CI environments, a change in the libabigail package requires regenerating the ABI references, - comparing xml files with abidiff is not well tested, as we (DPDK) uncovered bugs in libabigail that were not hit with comparing .so, Switch to comparing .so directly, remove this dependence and update GHA script. Signed-off-by: David Marchand --- .ci/linux-build.sh | 4 ---- .github/workflows/build.yml | 2 +- MAINTAINERS | 1 - devtools/check-abi.sh | 17 +++++++++-------- devtools/gen-abi.sh | 27 --------------------------- devtools/test-meson-builds.sh | 5 ----- 6 files changed, 10 insertions(+), 46 deletions(-) delete mode 100755 devtools/gen-abi.sh diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 150b38bd7a..9631e342b5 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -130,8 +130,6 @@ fi if [ "$ABI_CHECKS" = "true" ]; then if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then rm -rf libabigail - # if we change libabigail, invalidate existing abi cache - rm -rf reference fi if [ ! -d libabigail ]; then @@ -153,7 +151,6 @@ if [ "$ABI_CHECKS" = "true" ]; then meson setup $OPTS -Dexamples= $refsrcdir $refsrcdir/build ninja -C $refsrcdir/build DESTDIR=$(pwd)/reference ninja -C $refsrcdir/build install - devtools/gen-abi.sh reference find reference/usr/local -name '*.a' -delete rm -rf reference/usr/local/bin rm -rf reference/usr/local/share @@ -161,7 +158,6 @@ if [ "$ABI_CHECKS" = "true" ]; then fi DESTDIR=$(pwd)/install ninja -C build install - devtools/gen-abi.sh install devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-} fi diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index bbcb535afb..e24e47a216 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -70,7 +70,7 @@ jobs: run: | echo 'ccache=ccache-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W) >> $GITHUB_OUTPUT echo 'libabigail=libabigail-${{ matrix.config.os }}' >> $GITHUB_OUTPUT - echo 'abi=abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.LIBABIGAIL_VERSION }}-${{ env.REF_GIT_TAG }}' >> $GITHUB_OUTPUT + echo 'abi=abi-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.cross }}-${{ env.REF_GIT_TAG }}' >> $GITHUB_OUTPUT - name: Retrieve ccache cache uses: actions/cache@v3 with: diff --git a/MAINTAINERS b/MAINTAINERS index 1a33ad8592..280058adfc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -94,7 +94,6 @@ F: devtools/check-spdx-tag.sh F: devtools/check-symbol-change.sh F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh -F: devtools/gen-abi.sh F: devtools/get-maintainer.sh F: devtools/git-log-fixes.sh F: devtools/load-devel-config diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh index f74432be5d..39e3798931 100755 --- a/devtools/check-abi.sh +++ b/devtools/check-abi.sh @@ -37,20 +37,21 @@ fi export newdir ABIDIFF_OPTIONS ABIDIFF_SUPPRESSIONS export diff_func='run_diff() { - dump=$1 - name=$(basename $dump) - if grep -q "; SKIP_LIBRARY=${name%.dump}\>" $ABIDIFF_SUPPRESSIONS; then + lib=$1 + name=$(basename $lib) + if grep -q "; SKIP_LIBRARY=${name%.so.*}\>" $ABIDIFF_SUPPRESSIONS; then echo "Skipped $name" >&2 return 0 fi - dump2=$(find $newdir -name $name) - if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then + # Look for a library with the same major ABI version + lib2=$(find $newdir -name "${name%.*}.*" -a ! -type l) + if [ -z "$lib2" ] || [ ! -e "$lib2" ]; then echo "Error: cannot find $name in $newdir" >&2 return 1 fi - abidiff $ABIDIFF_OPTIONS $dump $dump2 || { + abidiff $ABIDIFF_OPTIONS $lib $lib2 || { abiret=$? - echo "Error: ABI issue reported for abidiff $ABIDIFF_OPTIONS $dump $dump2" >&2 + echo "Error: ABI issue reported for abidiff $ABIDIFF_OPTIONS $lib $lib2" >&2 if [ $(($abiret & 3)) -ne 0 ]; then echo "ABIDIFF_ERROR|ABIDIFF_USAGE_ERROR, this could be a script or environment issue." >&2 fi @@ -65,7 +66,7 @@ export diff_func='run_diff() { }' error= -find $refdir -name "*.dump" | +find $refdir -name "*.so.*" -a ! -type l | xargs -n1 -P0 sh -c 'eval "$diff_func"; run_diff $0' || error=1 diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh deleted file mode 100755 index 61f7510ea1..0000000000 --- a/devtools/gen-abi.sh +++ /dev/null @@ -1,27 +0,0 @@ -#!/bin/sh -e -# SPDX-License-Identifier: BSD-3-Clause -# Copyright (c) 2019 Red Hat, Inc. - -if [ $# != 1 ]; then - echo "Usage: $0 installdir" >&2 - exit 1 -fi - -installdir=$1 -if [ ! -d $installdir ]; then - echo "Error: install directory '$installdir' does not exist." >&2 - exit 1 -fi - -dumpdir=$installdir/dump -rm -rf $dumpdir -mkdir -p $dumpdir -for f in $(find $installdir -name "*.so.*"); do - if test -L $f; then - continue - fi - - libname=$(basename $f) - echo $dumpdir/${libname%.so*}.dump $f -done | -xargs -n2 -P0 abidw --out-file diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index 48f4e52df3..9131088c9d 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -204,7 +204,6 @@ build () # [meson options] -Dexamples= $* compile $abirefdir/build install_target $abirefdir/build $abirefdir/$targetdir - $srcdir/devtools/gen-abi.sh $abirefdir/$targetdir # save disk space by removing static libs and apps find $abirefdir/$targetdir/usr/local -name '*.a' -delete @@ -215,10 +214,6 @@ build () # [meson options] install_target $builds_dir/$targetdir \ $(readlink -f $builds_dir/$targetdir/install) echo "Checking ABI compatibility of $targetdir" >&$verbose - echo $srcdir/devtools/gen-abi.sh \ - $(readlink -f $builds_dir/$targetdir/install) >&$veryverbose - $srcdir/devtools/gen-abi.sh \ - $(readlink -f $builds_dir/$targetdir/install) >&$veryverbose echo $srcdir/devtools/check-abi.sh $abirefdir/$targetdir \ $(readlink -f $builds_dir/$targetdir/install) >&$veryverbose $srcdir/devtools/check-abi.sh $abirefdir/$targetdir \