From patchwork Fri Apr 1 18:02:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 109086 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6B711A0507; Fri, 1 Apr 2022 20:03:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 10EF840E03; Fri, 1 Apr 2022 20:03:03 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id B2A8B4067E for ; Fri, 1 Apr 2022 20:03:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648836180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P1qvlJJHq0xIfbrEFk8LSZSflpeRon0kq/5acbMAaQs=; b=ix8/RKHqjLijd7wYWuA/+mxOOVprz7SdKy3i66hGkcm2UenN/jwTrlwlZbwolSjBseLHrJ MlTPSqpY3LbeZ8KsP7aTaQH6RLEs9ce5LLd5X0k1fy+cdQOLucO/T2l+HCSNhoOgu4SvFg JtK500AOZa9Ch0ur5Bv6h5PT9vhqyQM= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-307-Y_YKFY9lNGKSmbICAsMQ5A-1; Fri, 01 Apr 2022 14:02:59 -0400 X-MC-Unique: Y_YKFY9lNGKSmbICAsMQ5A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D88861C01E86; Fri, 1 Apr 2022 18:02:58 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1624F4292D2; Fri, 1 Apr 2022 18:02:57 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: Aaron Conole , Michael Santana Subject: [PATCH v2] ci: add Fedora 35 container in GHA Date: Fri, 1 Apr 2022 20:02:53 +0200 Message-Id: <20220401180253.17876-1-david.marchand@redhat.com> In-Reply-To: <20211116123102.19237-1-david.marchand@redhat.com> References: <20211116123102.19237-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Build DPDK with Fedora 35 containers. GHA container support does not allow caching images and docker hub seems to limit image pulls. On the other hand, the Fedora project hub does not seem to limit them, so prefer this hub. Nevertheless, let's try to be good citizens and cache (once a day) a prepared image for subsequent builds. Differences with the Ubuntu GHA vm images: - tasks are run as root in containers, no need for sudo, - compiler must be explicitly installed, - GHA artifacts can't contain a ':' in their name, and must be filtered, Signed-off-by: David Marchand --- Changes since v1: - switched to Fedora project hub, - cached container image, --- .ci/linux-build.sh | 3 + .ci/linux-setup.sh | 3 + .github/workflows/build.yml | 106 +++++++++++++++++++++++++++++++++++- 3 files changed, 110 insertions(+), 2 deletions(-) diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 69940dec8b..774a1441bf 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -1,5 +1,8 @@ #!/bin/sh -xe +# Builds are run as root in containers, no need for sudo +[ "$(id -u)" != '0' ] || alias sudo= + on_error() { if [ $? = 0 ]; then exit diff --git a/.ci/linux-setup.sh b/.ci/linux-setup.sh index ca35b4b38d..f5ef6b1b98 100755 --- a/.ci/linux-setup.sh +++ b/.ci/linux-setup.sh @@ -1,5 +1,8 @@ #!/bin/sh -xe +# Builds are run as root in containers, no need for sudo +[ "$(id -u)" != '0' ] || alias sudo= + # need to install as 'root' since some of the unit tests won't run without it sudo python3 -m pip install --upgrade 'meson==0.49.2' diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 02819aa5de..b01868ad76 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -10,7 +10,7 @@ defaults: shell: bash --noprofile --norc -exo pipefail {0} jobs: - build: + ubuntu-vms: name: ${{ join(matrix.config.*, '-') }} runs-on: ${{ matrix.config.os }} env: @@ -145,7 +145,109 @@ jobs: with: name: meson-logs-${{ join(matrix.config.*, '-') }} path: | + build/.ninja_log + build/gdb.log + build/meson-logs/meson-log.txt build/meson-logs/testlog.txt + + rpm-containers: + name: ${{ join(matrix.config.*, '-') }} + runs-on: ubuntu-latest + + strategy: + fail-fast: false + matrix: + config: + - image: fedora:35 + compiler: gcc + library: static + - image: fedora:35 + compiler: gcc + library: shared + - image: fedora:35 + compiler: clang + library: static + - image: fedora:35 + compiler: clang + library: shared + + steps: + - name: Checkout sources + uses: actions/checkout@v2 + - name: Generate various keys + id: get_keys + run: | + echo -n '::set-output name=ccache::' + echo 'ccache-${{ matrix.config.image }}-${{ matrix.config.compiler }}-'$(date -u +%Y-w%W) + echo -n '::set-output name=image::' + echo 'image-${{ matrix.config.image }}-'$(date -u +%Y-%m-%d) + echo -n '::set-output name=logs::' + echo 'meson-logs-${{ join(matrix.config.*, '-') }}' | tr -d ':' + - name: Retrieve ccache cache + uses: actions/cache@v2 + with: + path: ~/.ccache + key: ${{ steps.get_keys.outputs.ccache }}-${{ github.ref }} + restore-keys: | + ${{ steps.get_keys.outputs.ccache }}-refs/heads/main + - name: Retrieve image cache + id: image_cache + uses: actions/cache@v2 + with: + path: ~/.image + key: ${{ steps.get_keys.outputs.image }} + - name: Prepare working directory + run: | + mkdir -p ~/.ccache + mkdir -p ~/.image + > ~/env + echo CC=ccache ${{ matrix.config.compiler }} >> ~/env + echo DEF_LIB=${{ matrix.config.library }} >> ~/env + - name: Pull and prepare a fresh image + if: steps.image_cache.outputs.cache-hit != 'true' + run: | + docker pull registry.fedoraproject.org/${{ matrix.config.image }} + docker run \ + --mount type=bind,src=$HOME/.ccache,dst=/root/.ccache \ + --mount type=bind,src=$(pwd),dst=/root/dpdk \ + --env-file ~/env -w /root/dpdk -d -i --name dpdk \ + registry.fedoraproject.org/${{ matrix.config.image }} \ + bash -li + - name: Load the cached image + if: steps.image_cache.outputs.cache-hit == 'true' + run: | + docker load -i ~/.image/${{ matrix.config.image }}.tar + docker run \ + --mount type=bind,src=$HOME/.ccache,dst=/root/.ccache \ + --mount type=bind,src=$(pwd),dst=/root/dpdk \ + --env-file ~/env -w /root/dpdk -d -i --name dpdk \ + local:dpdk \ + bash -li + - name: Update + run: docker exec -i dpdk dnf update -y || true + - name: Install packages + run: docker exec -i dpdk dnf install -y ccache numactl-devel + python3-setuptools python3-wheel python3-pip python3-pyelftools + ninja-build libatomic libbpf-devel libfdt-devel libpcap-devel + openssl-devel rdma-core-devel zlib-devel + - name: Save image in cache + if: steps.image_cache.outputs.cache-hit != 'true' + run: | + docker commit dpdk local:dpdk + docker save -o ~/.image/${{ matrix.config.image }}.tar local:dpdk + - name: Install compiler + run: docker exec -i dpdk dnf install -y ${{ matrix.config.compiler }} + - name: Run setup + run: docker exec -i dpdk .ci/linux-setup.sh + - name: Build + run: docker exec -i dpdk .ci/linux-build.sh + - name: Stop image + run: docker kill dpdk + - name: Upload logs on failure + if: failure() + uses: actions/upload-artifact@v2 + with: + name: ${{ steps.get_keys.outputs.logs }} + path: | build/.ninja_log build/meson-logs/meson-log.txt - build/gdb.log