[v2] ci: add Fedora 35 container in GHA
Checks
Commit Message
Build DPDK with Fedora 35 containers.
GHA container support does not allow caching images and docker hub
seems to limit image pulls.
On the other hand, the Fedora project hub does not seem to limit them,
so prefer this hub.
Nevertheless, let's try to be good citizens and cache (once a day) a
prepared image for subsequent builds.
Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed,
- GHA artifacts can't contain a ':' in their name, and must be filtered,
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changes since v1:
- switched to Fedora project hub,
- cached container image,
---
.ci/linux-build.sh | 3 +
.ci/linux-setup.sh | 3 +
.github/workflows/build.yml | 106 +++++++++++++++++++++++++++++++++++-
3 files changed, 110 insertions(+), 2 deletions(-)
Comments
On Fri, Apr 1, 2022 at 8:03 PM David Marchand <david.marchand@redhat.com> wrote:
>
> Build DPDK with Fedora 35 containers.
>
> GHA container support does not allow caching images and docker hub
> seems to limit image pulls.
> On the other hand, the Fedora project hub does not seem to limit them,
> so prefer this hub.
> Nevertheless, let's try to be good citizens and cache (once a day) a
> prepared image for subsequent builds.
We could move to a pipeline approach, with a first stage job preparing
the image and pushing to the cache, and the second stage pulling the
image from the cache and starting compilations.
Food for thoughts...
>
> Differences with the Ubuntu GHA vm images:
> - tasks are run as root in containers, no need for sudo,
> - compiler must be explicitly installed,
> - GHA artifacts can't contain a ':' in their name, and must be filtered,
>
@@ -1,5 +1,8 @@
#!/bin/sh -xe
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
on_error() {
if [ $? = 0 ]; then
exit
@@ -1,5 +1,8 @@
#!/bin/sh -xe
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
# need to install as 'root' since some of the unit tests won't run without it
sudo python3 -m pip install --upgrade 'meson==0.49.2'
@@ -10,7 +10,7 @@ defaults:
shell: bash --noprofile --norc -exo pipefail {0}
jobs:
- build:
+ ubuntu-vms:
name: ${{ join(matrix.config.*, '-') }}
runs-on: ${{ matrix.config.os }}
env:
@@ -145,7 +145,109 @@ jobs:
with:
name: meson-logs-${{ join(matrix.config.*, '-') }}
path: |
+ build/.ninja_log
+ build/gdb.log
+ build/meson-logs/meson-log.txt
build/meson-logs/testlog.txt
+
+ rpm-containers:
+ name: ${{ join(matrix.config.*, '-') }}
+ runs-on: ubuntu-latest
+
+ strategy:
+ fail-fast: false
+ matrix:
+ config:
+ - image: fedora:35
+ compiler: gcc
+ library: static
+ - image: fedora:35
+ compiler: gcc
+ library: shared
+ - image: fedora:35
+ compiler: clang
+ library: static
+ - image: fedora:35
+ compiler: clang
+ library: shared
+
+ steps:
+ - name: Checkout sources
+ uses: actions/checkout@v2
+ - name: Generate various keys
+ id: get_keys
+ run: |
+ echo -n '::set-output name=ccache::'
+ echo 'ccache-${{ matrix.config.image }}-${{ matrix.config.compiler }}-'$(date -u +%Y-w%W)
+ echo -n '::set-output name=image::'
+ echo 'image-${{ matrix.config.image }}-'$(date -u +%Y-%m-%d)
+ echo -n '::set-output name=logs::'
+ echo 'meson-logs-${{ join(matrix.config.*, '-') }}' | tr -d ':'
+ - name: Retrieve ccache cache
+ uses: actions/cache@v2
+ with:
+ path: ~/.ccache
+ key: ${{ steps.get_keys.outputs.ccache }}-${{ github.ref }}
+ restore-keys: |
+ ${{ steps.get_keys.outputs.ccache }}-refs/heads/main
+ - name: Retrieve image cache
+ id: image_cache
+ uses: actions/cache@v2
+ with:
+ path: ~/.image
+ key: ${{ steps.get_keys.outputs.image }}
+ - name: Prepare working directory
+ run: |
+ mkdir -p ~/.ccache
+ mkdir -p ~/.image
+ > ~/env
+ echo CC=ccache ${{ matrix.config.compiler }} >> ~/env
+ echo DEF_LIB=${{ matrix.config.library }} >> ~/env
+ - name: Pull and prepare a fresh image
+ if: steps.image_cache.outputs.cache-hit != 'true'
+ run: |
+ docker pull registry.fedoraproject.org/${{ matrix.config.image }}
+ docker run \
+ --mount type=bind,src=$HOME/.ccache,dst=/root/.ccache \
+ --mount type=bind,src=$(pwd),dst=/root/dpdk \
+ --env-file ~/env -w /root/dpdk -d -i --name dpdk \
+ registry.fedoraproject.org/${{ matrix.config.image }} \
+ bash -li
+ - name: Load the cached image
+ if: steps.image_cache.outputs.cache-hit == 'true'
+ run: |
+ docker load -i ~/.image/${{ matrix.config.image }}.tar
+ docker run \
+ --mount type=bind,src=$HOME/.ccache,dst=/root/.ccache \
+ --mount type=bind,src=$(pwd),dst=/root/dpdk \
+ --env-file ~/env -w /root/dpdk -d -i --name dpdk \
+ local:dpdk \
+ bash -li
+ - name: Update
+ run: docker exec -i dpdk dnf update -y || true
+ - name: Install packages
+ run: docker exec -i dpdk dnf install -y ccache numactl-devel
+ python3-setuptools python3-wheel python3-pip python3-pyelftools
+ ninja-build libatomic libbpf-devel libfdt-devel libpcap-devel
+ openssl-devel rdma-core-devel zlib-devel
+ - name: Save image in cache
+ if: steps.image_cache.outputs.cache-hit != 'true'
+ run: |
+ docker commit dpdk local:dpdk
+ docker save -o ~/.image/${{ matrix.config.image }}.tar local:dpdk
+ - name: Install compiler
+ run: docker exec -i dpdk dnf install -y ${{ matrix.config.compiler }}
+ - name: Run setup
+ run: docker exec -i dpdk .ci/linux-setup.sh
+ - name: Build
+ run: docker exec -i dpdk .ci/linux-build.sh
+ - name: Stop image
+ run: docker kill dpdk
+ - name: Upload logs on failure
+ if: failure()
+ uses: actions/upload-artifact@v2
+ with:
+ name: ${{ steps.get_keys.outputs.logs }}
+ path: |
build/.ninja_log
build/meson-logs/meson-log.txt
- build/gdb.log