Message ID | 1559638792-8608-1-git-send-email-david.marchand@redhat.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3A1241BB3A; Tue, 4 Jun 2019 11:00:10 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 8071E1BB32 for <dev@dpdk.org>; Tue, 4 Jun 2019 11:00:08 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A1D21C1EB1FE; Tue, 4 Jun 2019 09:00:07 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-116-185.ams2.redhat.com [10.36.116.185]) by smtp.corp.redhat.com (Postfix) with ESMTP id EDD7B5D9D2; Tue, 4 Jun 2019 09:00:02 +0000 (UTC) From: David Marchand <david.marchand@redhat.com> To: dev@dpdk.org Cc: thomas@monjalon.net, aconole@redhat.com, msantana@redhat.com Date: Tue, 4 Jun 2019 10:59:38 +0200 Message-Id: <1559638792-8608-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 04 Jun 2019 09:00:07 +0000 (UTC) Subject: [dpdk-dev] [PATCH 00/14] Unit tests fixes for CI X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series |
Unit tests fixes for CI
|
|
Message
David Marchand
June 4, 2019, 8:59 a.m. UTC
This is a joint effort to make the unit tests ready for CI. The first 8 patches are fixes that I had accumulated. Then the second part of the series focuses on skipping tests when some requirements are not fulfilled so that we can start them in a restrained environment like Travis virtual machines that gives us two cores and does not have specific hw devices. We are still not ready for enabling those tests in Travis. At least, the following issues remain: - some fixes on librte_acl have not been merged yet [1], - the tests on --file-prefix are still ko, and have been isolated in a test that we could disable while waiting for the fixes, - rwlock_autotest and hash_readwrite_lf_autotest are taking a little more than 10s, - librte_table unit test crashes on ipv6 [2], - the "perf" tests are taking way too long for my taste, - the shared build unit tests all fail when depending on mempool since the mempool drivers are not loaded, 1: http://patchwork.dpdk.org/project/dpdk/list/?series=4242 2: https://bugs.dpdk.org/show_bug.cgi?id=285 Comments/reviews welcome!
Comments
On 6/4/19 4:59 AM, David Marchand wrote: > This is a joint effort to make the unit tests ready for CI. > The first 8 patches are fixes that I had accumulated. > Then the second part of the series focuses on skipping tests when some > requirements are not fulfilled so that we can start them in a restrained > environment like Travis virtual machines that gives us two cores and does > not have specific hw devices. > > We are still not ready for enabling those tests in Travis. > At least, the following issues remain: > - some fixes on librte_acl have not been merged yet [1], > - the tests on --file-prefix are still ko, and have been isolated in a > test that we could disable while waiting for the fixes, > - rwlock_autotest and hash_readwrite_lf_autotest are taking a little more > than 10s, It would be a good idea to increase timeout. These are probably not the only tests that would pass with just a little bit more time > - librte_table unit test crashes on ipv6 [2], > - the "perf" tests are taking way too long for my taste, We should still fix them. However I don't know if we should be running the perf test for every job and every patch on travis. It takes too long. The travis queue will be delayed too far behind for it to be of any use. OTOH we could have one job as part of the travis build dedicated to running tests (or just perf test). It's still time consuming but better than running the test on every travis job. For this to work we would need to decreased the timeout for the perf tests as the timeout for it and the travis are both 10 minutes > - the shared build unit tests all fail when depending on mempool since > the mempool drivers are not loaded, > > 1: http://patchwork.dpdk.org/project/dpdk/list/?series=4242 > 2: https://bugs.dpdk.org/show_bug.cgi?id=285 > > Comments/reviews welcome! >
04/06/2019 17:49, Michael Santana Francisco: > On 6/4/19 4:59 AM, David Marchand wrote: > > - the "perf" tests are taking way too long for my taste, > > We should still fix them. However I don't know if we should be running > the perf test for every job and every patch on travis. It takes too > long. The travis queue will be delayed too far behind for it to be of > any use. > > OTOH we could have one job as part of the travis build dedicated to > running tests (or just perf test). It's still time consuming but better > than running the test on every travis job. For this to work we would > need to decreased the timeout for the perf tests as the timeout for it > and the travis are both 10 minutes +Cc ci@dpdk.org I don't think we should run the perf tests in basic CI like Travis. We can run perf tests if the purpose is to compare the performance with previous releases, as some other tests in the community lab.
Thomas Monjalon <thomas@monjalon.net> writes: > 04/06/2019 17:49, Michael Santana Francisco: >> On 6/4/19 4:59 AM, David Marchand wrote: >> > - the "perf" tests are taking way too long for my taste, +1 here. >> >> We should still fix them. However I don't know if we should be running >> the perf test for every job and every patch on travis. It takes too >> long. The travis queue will be delayed too far behind for it to be of >> any use. >> >> OTOH we could have one job as part of the travis build dedicated to >> running tests (or just perf test). It's still time consuming but better >> than running the test on every travis job. For this to work we would >> need to decreased the timeout for the perf tests as the timeout for it >> and the travis are both 10 minutes > > +Cc ci@dpdk.org > > I don't think we should run the perf tests in basic CI like Travis. > We can run perf tests if the purpose is to compare the performance > with previous releases, as some other tests in the community lab. +1 - some of the perf tests aren't going to complete in any sort of reasonable time. While we could claim it's a separate problem, we should also not enable something that will make the travis runs so much longer. I do like the idea of running tests in the travis build, and I think it would make sense to have just a single job for it (or maybe one for clang and one for gcc? maybe even that is overkill). I would rather not do performance tests during the travis run, though. It doesn't really make sense. Travis isn't any kind of an 'optimized' environment, so I don't know what 'performance' should mean.