Message ID | 20200313081614.195335-1-ruifeng.wang@arm.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7BCE1A0567; Fri, 13 Mar 2020 09:16:56 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DBA221C00E; Fri, 13 Mar 2020 09:16:55 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 49FA73B5 for <dev@dpdk.org>; Fri, 13 Mar 2020 09:16:54 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A942431B; Fri, 13 Mar 2020 01:16:53 -0700 (PDT) Received: from net-arm-thunderx2-02.shanghai.arm.com (net-arm-thunderx2-02.shanghai.arm.com [10.169.40.171]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8C0703F67D; Fri, 13 Mar 2020 01:16:49 -0700 (PDT) From: Ruifeng Wang <ruifeng.wang@arm.com> To: aconole@redhat.com, maicolgabriel@hotmail.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, cristian.dumitrescu@intel.com, yipeng1.wang@intel.com, sameh.gobriel@intel.com Cc: dev@dpdk.org, david.marchand@redhat.com, anatoly.burakov@intel.com, gavin.hu@arm.com, honnappa.nagarahalli@arm.com, juraj.linkes@pantheon.tech, nd@arm.com, Ruifeng Wang <ruifeng.wang@arm.com> Date: Fri, 13 Mar 2020 16:16:10 +0800 Message-Id: <20200313081614.195335-1-ruifeng.wang@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200225073236.135581-1-ruifeng.wang@arm.com> References: <20200225073236.135581-1-ruifeng.wang@arm.com> Subject: [dpdk-dev] [PATCH v3 0/4] no-huge unit test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series | no-huge unit test | |
Message
Ruifeng Wang
March 13, 2020, 8:16 a.m. UTC
For environments (such as containers) where hugetlbfs are not available, some unit tests can be run with 'no-huge' option. fast-tests suites is generated dynamically according to hugetlbfs availability in building environment. This allows unit test to run in different environments using the same suite name. Several test cases are fixed to be able to run in no-huge mode. v3: Use a single suite instead of create additional one for no-huge. (Aaron) Fix several test cases in no-huge mode. v2: Add a patch to enable running multiple suites in a job. (David) Ruifeng Wang (4): test: enable tests to run in no-huge mode ci: generate fast-tests suite base on hugepage availability ci: proceed with verification without hugepage ci: enable unit test for aarch64 .ci/linux-setup.sh | 11 +- .travis.yml | 5 +- app/test/meson.build | 216 ++++++++++++++++++--------------- app/test/test_acl.c | 22 ++-- app/test/test_hash.c | 7 +- app/test/test_table_pipeline.c | 12 +- 6 files changed, 152 insertions(+), 121 deletions(-)
Comments
Ruifeng Wang <ruifeng.wang@arm.com> writes: > For environments (such as containers) where hugetlbfs are not available, > some unit tests can be run with 'no-huge' option. > > fast-tests suites is generated dynamically according to hugetlbfs > availability in building environment. This allows unit test to run > in different environments using the same suite name. > > Several test cases are fixed to be able to run in no-huge mode. This looks great! Thanks, Ruifeng. I'm going to ack it once I see it run under the robot :) > v3: > Use a single suite instead of create additional one for no-huge. (Aaron) > Fix several test cases in no-huge mode. > > v2: > Add a patch to enable running multiple suites in a job. (David) > > > Ruifeng Wang (4): > test: enable tests to run in no-huge mode > ci: generate fast-tests suite base on hugepage availability > ci: proceed with verification without hugepage > ci: enable unit test for aarch64 > > .ci/linux-setup.sh | 11 +- > .travis.yml | 5 +- > app/test/meson.build | 216 ++++++++++++++++++--------------- > app/test/test_acl.c | 22 ++-- > app/test/test_hash.c | 7 +- > app/test/test_table_pipeline.c | 12 +- > 6 files changed, 152 insertions(+), 121 deletions(-)
Aaron Conole <aconole@redhat.com> writes: > Ruifeng Wang <ruifeng.wang@arm.com> writes: > >> For environments (such as containers) where hugetlbfs are not available, >> some unit tests can be run with 'no-huge' option. >> >> fast-tests suites is generated dynamically according to hugetlbfs >> availability in building environment. This allows unit test to run >> in different environments using the same suite name. >> >> Several test cases are fixed to be able to run in no-huge mode. > > This looks great! Thanks, Ruifeng. > > I'm going to ack it once I see it run under the robot :) Just looking through the robot's run, it seems that on the statically linked Arm64 build, the disk quota is getting exceeded. Do we need to request some more disk quota for this somehow? Is the build getting too large? >> v3: >> Use a single suite instead of create additional one for no-huge. (Aaron) >> Fix several test cases in no-huge mode. >> >> v2: >> Add a patch to enable running multiple suites in a job. (David) >> >> >> Ruifeng Wang (4): >> test: enable tests to run in no-huge mode >> ci: generate fast-tests suite base on hugepage availability >> ci: proceed with verification without hugepage >> ci: enable unit test for aarch64 >> >> .ci/linux-setup.sh | 11 +- >> .travis.yml | 5 +- >> app/test/meson.build | 216 ++++++++++++++++++--------------- >> app/test/test_acl.c | 22 ++-- >> app/test/test_hash.c | 7 +- >> app/test/test_table_pipeline.c | 12 +- >> 6 files changed, 152 insertions(+), 121 deletions(-)
On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote: > > Aaron Conole <aconole@redhat.com> writes: > > > Ruifeng Wang <ruifeng.wang@arm.com> writes: > > > >> For environments (such as containers) where hugetlbfs are not available, > >> some unit tests can be run with 'no-huge' option. > >> > >> fast-tests suites is generated dynamically according to hugetlbfs > >> availability in building environment. This allows unit test to run > >> in different environments using the same suite name. > >> > >> Several test cases are fixed to be able to run in no-huge mode. > > > > This looks great! Thanks, Ruifeng. > > > > I'm going to ack it once I see it run under the robot :) > > Just looking through the robot's run, it seems that on the statically > linked Arm64 build, the disk quota is getting exceeded. Do we need to > request some more disk quota for this somehow? Is the build getting too > large? It seems to repeat. https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975 Do you know how much space we have in travis? Is the (c?)cache getting too big? You can find out the per job cache size via the travis cli.
> -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Friday, March 13, 2020 23:54 > To: Aaron Conole <aconole@redhat.com> > Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana > <maicolgabriel@hotmail.com>; Bruce Richardson > <bruce.richardson@intel.com>; Ananyev, Konstantin > <konstantin.ananyev@intel.com>; Cristian Dumitrescu > <cristian.dumitrescu@intel.com>; Wang, Yipeng1 > <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; > dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>; > Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli > <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd > <nd@arm.com> > Subject: Re: [PATCH v3 0/4] no-huge unit test > > On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote: > > > > Aaron Conole <aconole@redhat.com> writes: > > > > > Ruifeng Wang <ruifeng.wang@arm.com> writes: > > > > > >> For environments (such as containers) where hugetlbfs are not > > >> available, some unit tests can be run with 'no-huge' option. > > >> > > >> fast-tests suites is generated dynamically according to hugetlbfs > > >> availability in building environment. This allows unit test to run > > >> in different environments using the same suite name. > > >> > > >> Several test cases are fixed to be able to run in no-huge mode. > > > > > > This looks great! Thanks, Ruifeng. > > > > > > I'm going to ack it once I see it run under the robot :) > > > > Just looking through the robot's run, it seems that on the statically > > linked Arm64 build, the disk quota is getting exceeded. Do we need to > > request some more disk quota for this somehow? Is the build getting > > too large? > > It seems to repeat. > https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975 > > Do you know how much space we have in travis? > Is the (c?)cache getting too big? Yes, it is probably caused by cache. I hit the disk quota issue as well when running Travis against the latest Master code. After deleting caches, the issue was gone. Then Travis run with this series of patches also got a pass. Hi Aaron, Is is OK to clear cache of robot and re-run the build? Thanks. /Ruifeng > You can find out the per job cache size via the travis cli. > > > -- > David Marchand
Ruifeng Wang <Ruifeng.Wang@arm.com> writes: >> -----Original Message----- >> From: David Marchand <david.marchand@redhat.com> >> Sent: Friday, March 13, 2020 23:54 >> To: Aaron Conole <aconole@redhat.com> >> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Michael Santana >> <maicolgabriel@hotmail.com>; Bruce Richardson >> <bruce.richardson@intel.com>; Ananyev, Konstantin >> <konstantin.ananyev@intel.com>; Cristian Dumitrescu >> <cristian.dumitrescu@intel.com>; Wang, Yipeng1 >> <yipeng1.wang@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; >> dev <dev@dpdk.org>; Burakov, Anatoly <anatoly.burakov@intel.com>; >> Gavin Hu <Gavin.Hu@arm.com>; Honnappa Nagarahalli >> <Honnappa.Nagarahalli@arm.com>; juraj.linkes@pantheon.tech; nd >> <nd@arm.com> >> Subject: Re: [PATCH v3 0/4] no-huge unit test >> >> On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote: >> > >> > Aaron Conole <aconole@redhat.com> writes: >> > >> > > Ruifeng Wang <ruifeng.wang@arm.com> writes: >> > > >> > >> For environments (such as containers) where hugetlbfs are not >> > >> available, some unit tests can be run with 'no-huge' option. >> > >> >> > >> fast-tests suites is generated dynamically according to hugetlbfs >> > >> availability in building environment. This allows unit test to run >> > >> in different environments using the same suite name. >> > >> >> > >> Several test cases are fixed to be able to run in no-huge mode. >> > > >> > > This looks great! Thanks, Ruifeng. >> > > >> > > I'm going to ack it once I see it run under the robot :) >> > >> > Just looking through the robot's run, it seems that on the statically >> > linked Arm64 build, the disk quota is getting exceeded. Do we need to >> > request some more disk quota for this somehow? Is the build getting >> > too large? >> >> It seems to repeat. >> https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975 >> >> Do you know how much space we have in travis? >> Is the (c?)cache getting too big? > > Yes, it is probably caused by cache. > I hit the disk quota issue as well when running Travis against the latest Master code. > After deleting caches, the issue was gone. Then Travis run with this > series of patches also got a pass. > > Hi Aaron, > Is is OK to clear cache of robot and re-run the build? I can do this, but I am concerned that we will need to disable the cache to avoid this error on a consistent basis. > Thanks. > /Ruifeng > >> You can find out the per job cache size via the travis cli. >> >> >> -- >> David Marchand
David Marchand <david.marchand@redhat.com> writes: > On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote: >> >> Aaron Conole <aconole@redhat.com> writes: >> >> > Ruifeng Wang <ruifeng.wang@arm.com> writes: >> > >> >> For environments (such as containers) where hugetlbfs are not available, >> >> some unit tests can be run with 'no-huge' option. >> >> >> >> fast-tests suites is generated dynamically according to hugetlbfs >> >> availability in building environment. This allows unit test to run >> >> in different environments using the same suite name. >> >> >> >> Several test cases are fixed to be able to run in no-huge mode. >> > >> > This looks great! Thanks, Ruifeng. >> > >> > I'm going to ack it once I see it run under the robot :) >> >> Just looking through the robot's run, it seems that on the statically >> linked Arm64 build, the disk quota is getting exceeded. Do we need to >> request some more disk quota for this somehow? Is the build getting too >> large? > > It seems to repeat. > https://travis-ci.com/github/ovsrobot/dpdk/jobs/297840285#L2975 Yes. > Do you know how much space we have in travis? Suppposedly we have 18G on that container... :-/ > Is the (c?)cache getting too big? > You can find out the per job cache size via the travis cli. When using the travis CLI: $ travis cache no caches found I know this must be untrue, but it seems to not want to send me details on my system. I tried going through the API, but the largest cache file I see is 200M, so I must be misunderstanding something.
On Mon, Mar 16, 2020 at 10:13:23AM -0400, Aaron Conole wrote: > David Marchand <david.marchand@redhat.com> writes: > > > On Fri, Mar 13, 2020 at 2:04 PM Aaron Conole <aconole@redhat.com> wrote: > >> > >> Aaron Conole <aconole@redhat.com> writes: > >> > >> > Ruifeng Wang <ruifeng.wang@arm.com> writes: > >> > > >> >> For environments (such as containers) where hugetlbfs are not available, > >> >> some unit tests can be run with 'no-huge' option. > >> >> > >> >> fast-tests suites is generated dynamically according to hugetlbfs > >> >> availability in building environment. This allows unit test to run > >> >> in different environments using the same suite name. > >> >> > >> >> Several test cases are fixed to be able to run in no-huge mode. > >> > > >> > This looks great! Thanks, Ruifeng. > >> > > >> > I'm going to ack it once I see it run under the robot :) > >> > >> Just looking through the robot's run, it seems that on the statically > >> linked Arm64 build, the disk quota is getting exceeded. Do we need to > >> request some more disk quota for this somehow? Is the build getting too > >> large? > > I think as a general rule we need to limit the number of static builds we do, and possibly skip building all the examples for the static builds. Given we have almost 50 examples, that's a lot of linking of libraries into binaries. For meson builds, perhaps we only pass -Dexamples=all for the shared builds, and maybe do -Dexamples=l3fwd or similar for the static ones. One or two examplse to check that the linking works with examples should be enough if we test the build of the examples themselves in the shared build jobs. Thoughts? /Bruce
On 2020-03-13 01:16, Ruifeng Wang wrote: > For environments (such as containers) where hugetlbfs are not > available, > some unit tests can be run with 'no-huge' option. > > fast-tests suites is generated dynamically according to hugetlbfs > availability in building environment. This allows unit test to run > in different environments using the same suite name. > > Several test cases are fixed to be able to run in no-huge mode. > > v3: > Use a single suite instead of create additional one for no-huge. > (Aaron) > Fix several test cases in no-huge mode. > > v2: > Add a patch to enable running multiple suites in a job. (David) > > > Ruifeng Wang (4): > test: enable tests to run in no-huge mode > ci: generate fast-tests suite base on hugepage availability > ci: proceed with verification without hugepage > ci: enable unit test for aarch64 > > .ci/linux-setup.sh | 11 +- > .travis.yml | 5 +- > app/test/meson.build | 216 ++++++++++++++++++--------------- > app/test/test_acl.c | 22 ++-- > app/test/test_hash.c | 7 +- > app/test/test_table_pipeline.c | 12 +- > 6 files changed, 152 insertions(+), 121 deletions(-) I added Ruifeng's patches on top of my ppc64le ci patches and updated the matrix. Ci ran fine on ppc64le. https://travis-ci.org/github/djlwilder/dpdk/jobs/663573687 Tested-by: David Wilder <dwilder@us.ibm.com>