Message ID | 20220406145606.2913834-1-juraj.linkes@pantheon.tech (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F3588A0507; Wed, 6 Apr 2022 16:56:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C26E40689; Wed, 6 Apr 2022 16:56:11 +0200 (CEST) Received: from lb.pantheon.sk (lb.pantheon.sk [46.229.239.20]) by mails.dpdk.org (Postfix) with ESMTP id 02C984014F for <dev@dpdk.org>; Wed, 6 Apr 2022 16:56:09 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by lb.pantheon.sk (Postfix) with ESMTP id 5975E1AB919; Wed, 6 Apr 2022 16:56:08 +0200 (CEST) X-Virus-Scanned: amavisd-new at siecit.sk Received: from lb.pantheon.sk ([127.0.0.1]) by localhost (lb.pantheon.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k6rsMB59wxGu; Wed, 6 Apr 2022 16:56:07 +0200 (CEST) Received: from entguard.lab.pantheon.local (unknown [46.229.239.141]) by lb.pantheon.sk (Postfix) with ESMTP id AF58819E0DA; Wed, 6 Apr 2022 16:56:06 +0200 (CEST) From: =?utf-8?q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech> To: thomas@monjalon.net, david.marchand@redhat.com, Honnappa.Nagarahalli@arm.com, ohilyard@iol.unh.edu, lijuan.tu@intel.com Cc: dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech> Subject: [RFC PATCH v1 00/15] merge DTS core files to DPDK Date: Wed, 6 Apr 2022 14:55:51 +0000 Message-Id: <20220406145606.2913834-1-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
merge DTS core files to DPDK
|
|
Message
Juraj Linkeš
April 6, 2022, 2:55 p.m. UTC
These are the basic libraries that other libraries depend on. There's also the basic framework functionality related to test execution. Juraj Linkeš (15): dts: merge DTS dep/tclclient.tgz to DPDK dts: merge DTS dep/tgen.tgz to DPDK dts: merge DTS dts to DPDK dts: merge DTS framework/__init__.py to DPDK dts: merge DTS framework/asan_test.py to DPDK dts: merge DTS framework/checkCase.py to DPDK dts: merge DTS framework/dts.py to DPDK dts: merge DTS framework/exception.py to DPDK dts: merge DTS framework/logger.py to DPDK dts: merge DTS framework/packet.py to DPDK dts: merge DTS framework/project_dpdk.py to DPDK dts: merge DTS framework/serializer.py to DPDK dts: merge DTS framework/utils.py to DPDK dts: merge DTS main.py to DPDK dts: merge DTS version.py to DPDK dts/dep/tclclient.tgz | Bin 0 -> 199327 bytes dts/dep/tgen.tgz | Bin 0 -> 134392 bytes dts/dts | 9 + dts/framework/__init__.py | 30 + dts/framework/asan_test.py | 439 +++++++++++ dts/framework/checkCase.py | 214 ++++++ dts/framework/dts.py | 721 ++++++++++++++++++ dts/framework/exception.py | 148 ++++ dts/framework/logger.py | 474 ++++++++++++ dts/framework/packet.py | 1292 +++++++++++++++++++++++++++++++++ dts/framework/project_dpdk.py | 627 ++++++++++++++++ dts/framework/serializer.py | 115 +++ dts/framework/utils.py | 353 +++++++++ dts/main.py | 236 ++++++ dts/version.py | 21 + 15 files changed, 4679 insertions(+) create mode 100644 dts/dep/tclclient.tgz create mode 100644 dts/dep/tgen.tgz create mode 100755 dts/dts create mode 100644 dts/framework/__init__.py create mode 100644 dts/framework/asan_test.py create mode 100644 dts/framework/checkCase.py create mode 100644 dts/framework/dts.py create mode 100644 dts/framework/exception.py create mode 100644 dts/framework/logger.py create mode 100644 dts/framework/packet.py create mode 100644 dts/framework/project_dpdk.py create mode 100644 dts/framework/serializer.py create mode 100644 dts/framework/utils.py create mode 100755 dts/main.py create mode 100755 dts/version.py
Comments
On Wed, Apr 6, 2022 at 8:26 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote: > > These are the basic libraries that other libraries depend on. There's > also the basic framework functionality related to test execution. > > Juraj Linkeš (15): > dts: merge DTS dep/tclclient.tgz to DPDK > dts: merge DTS dep/tgen.tgz to DPDK > dts: merge DTS dts to DPDK > dts: merge DTS framework/__init__.py to DPDK > dts: merge DTS framework/asan_test.py to DPDK > dts: merge DTS framework/checkCase.py to DPDK > dts: merge DTS framework/dts.py to DPDK > dts: merge DTS framework/exception.py to DPDK > dts: merge DTS framework/logger.py to DPDK > dts: merge DTS framework/packet.py to DPDK > dts: merge DTS framework/project_dpdk.py to DPDK > dts: merge DTS framework/serializer.py to DPDK > dts: merge DTS framework/utils.py to DPDK > dts: merge DTS main.py to DPDK > dts: merge DTS version.py to DPDK merge->import > > dts/dep/tclclient.tgz | Bin 0 -> 199327 bytes > dts/dep/tgen.tgz | Bin 0 -> 134392 bytes Some top level comments: - I think, we should not check in binary files. - git commit comment should much more than "dts: merge DTS xxxx to DPDK" where the commit log should have details on check in. -Add the documentation from the first patch and update the documentation per patch based on the content.
07/04/2022 07:04, Jerin Jacob: > On Wed, Apr 6, 2022 at 8:26 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote: > > > > These are the basic libraries that other libraries depend on. There's > > also the basic framework functionality related to test execution. > > > > Juraj Linkeš (15): > > dts: merge DTS dep/tclclient.tgz to DPDK > > dts: merge DTS dep/tgen.tgz to DPDK > > dts: merge DTS dts to DPDK > > dts: merge DTS framework/__init__.py to DPDK > > dts: merge DTS framework/asan_test.py to DPDK > > dts: merge DTS framework/checkCase.py to DPDK > > dts: merge DTS framework/dts.py to DPDK > > dts: merge DTS framework/exception.py to DPDK > > dts: merge DTS framework/logger.py to DPDK > > dts: merge DTS framework/packet.py to DPDK > > dts: merge DTS framework/project_dpdk.py to DPDK > > dts: merge DTS framework/serializer.py to DPDK > > dts: merge DTS framework/utils.py to DPDK > > dts: merge DTS main.py to DPDK > > dts: merge DTS version.py to DPDK > > merge->import > > > > > dts/dep/tclclient.tgz | Bin 0 -> 199327 bytes > > dts/dep/tgen.tgz | Bin 0 -> 134392 bytes > > Some top level comments: > - I think, we should not check in binary files. +1 > - git commit comment should much more than "dts: merge DTS xxxx to DPDK" where > the commit log should have details on check in. +1 > -Add the documentation from the first patch and update the > documentation per patch > based on the content. +1 More comments: - Please don't send so many patches, it looks like spam. - Please let's start small with the very minimal code to run a dummy test. - Split by file does not make sense The process is going to be very long. The techboard said in the past that we must have a very careful review of an import piece by piece. So please be patient. Thank you
> -----Original Message----- > From: Thomas Monjalon <thomas@monjalon.net> > Sent: Thursday, April 7, 2022 9:34 AM > To: Juraj Linkeš <juraj.linkes@pantheon.tech> > Cc: Jerin Jacob <jerinjacobk@gmail.com>; David Marchand > <david.marchand@redhat.com>; Honnappa Nagarahalli > <Honnappa.Nagarahalli@arm.com>; ohilyard@iol.unh.edu; Tu, Lijuan > <lijuan.tu@intel.com>; dpdk-dev <dev@dpdk.org> > Subject: Re: [RFC PATCH v1 00/15] merge DTS core files to DPDK > > 07/04/2022 07:04, Jerin Jacob: > > On Wed, Apr 6, 2022 at 8:26 PM Juraj Linkeš <juraj.linkes@pantheon.tech> > wrote: > > > First of all, thanks for the feedback. We've already done a bit of a pre-review and this is where we decided we want the feedback from the rest of the community. We understand it's going to take time which is why we wanted to get feedback sooner than later. > > > These are the basic libraries that other libraries depend on. > > > There's also the basic framework functionality related to test execution. > > > > > > Juraj Linkeš (15): > > > dts: merge DTS dep/tclclient.tgz to DPDK > > > dts: merge DTS dep/tgen.tgz to DPDK > > > dts: merge DTS dts to DPDK > > > dts: merge DTS framework/__init__.py to DPDK > > > dts: merge DTS framework/asan_test.py to DPDK > > > dts: merge DTS framework/checkCase.py to DPDK > > > dts: merge DTS framework/dts.py to DPDK > > > dts: merge DTS framework/exception.py to DPDK > > > dts: merge DTS framework/logger.py to DPDK > > > dts: merge DTS framework/packet.py to DPDK > > > dts: merge DTS framework/project_dpdk.py to DPDK > > > dts: merge DTS framework/serializer.py to DPDK > > > dts: merge DTS framework/utils.py to DPDK > > > dts: merge DTS main.py to DPDK > > > dts: merge DTS version.py to DPDK > > > > merge->import > > Ack. I'll keep this in mind when composing commit messages in the future. > > > > > > dts/dep/tclclient.tgz | Bin 0 -> 199327 bytes > > > dts/dep/tgen.tgz | Bin 0 -> 134392 bytes > > > > Some top level comments: > > - I think, we should not check in binary files. > > +1 > Thanks, we'll make sure to make this a requirement - it makes sense to us as well. > > - git commit comment should much more than "dts: merge DTS xxxx to > > DPDK" where the commit log should have details on check in. > > +1 Right, this is part of the broader question of how exactly are we going to move the whole thing. More below. > > > -Add the documentation from the first patch and update the > > documentation per patch based on the content. > > +1 > > More comments: > > - Please don't send so many patches, it looks like spam. > - Please let's start small with the very minimal code to run a dummy test. > - Split by file does not make sense This is very useful. We started with splitting up DTS into logical chunks, but it's not quite what you have in mind. We'll move things around and send just the first part (containing just a dummy test (is a smoke test ok?) with the necessary code). Now that we have a better understanding of the first step we'll produce a more complete patch set, with proper commit messages and files in each commit. > > The process is going to be very long. > The techboard said in the past that we must have a very careful review of an > import piece by piece. So please be patient. > Thank you > >
<snip> > > > > 07/04/2022 07:04, Jerin Jacob: > > > On Wed, Apr 6, 2022 at 8:26 PM Juraj Linkeš > > > <juraj.linkes@pantheon.tech> > > wrote: > > > > > > First of all, thanks for the feedback. We've already done a bit of a pre-review > and this is where we decided we want the feedback from the rest of the > community. We understand it's going to take time which is why we wanted to > get feedback sooner than later. > > > > > These are the basic libraries that other libraries depend on. > > > > There's also the basic framework functionality related to test execution. > > > > > > > > Juraj Linkeš (15): > > > > dts: merge DTS dep/tclclient.tgz to DPDK > > > > dts: merge DTS dep/tgen.tgz to DPDK > > > > dts: merge DTS dts to DPDK > > > > dts: merge DTS framework/__init__.py to DPDK > > > > dts: merge DTS framework/asan_test.py to DPDK > > > > dts: merge DTS framework/checkCase.py to DPDK > > > > dts: merge DTS framework/dts.py to DPDK > > > > dts: merge DTS framework/exception.py to DPDK > > > > dts: merge DTS framework/logger.py to DPDK > > > > dts: merge DTS framework/packet.py to DPDK > > > > dts: merge DTS framework/project_dpdk.py to DPDK > > > > dts: merge DTS framework/serializer.py to DPDK > > > > dts: merge DTS framework/utils.py to DPDK > > > > dts: merge DTS main.py to DPDK > > > > dts: merge DTS version.py to DPDK > > > > > > merge->import > > > > > Ack. I'll keep this in mind when composing commit messages in the future. > > > > > > > > > dts/dep/tclclient.tgz | Bin 0 -> 199327 bytes > > > > dts/dep/tgen.tgz | Bin 0 -> 134392 bytes > > > > > > Some top level comments: > > > - I think, we should not check in binary files. > > > > +1 > > > > Thanks, we'll make sure to make this a requirement - it makes sense to us as > well. Should we move these to the external repo where the GPL file would be stored? I do not expect these files to change frequently. But the external repo also needs to be tagged (along with tagging the DPDK/DTS). > > > > - git commit comment should much more than "dts: merge DTS xxxx to > > > DPDK" where the commit log should have details on check in. > > > > +1 > > Right, this is part of the broader question of how exactly are we going to > move the whole thing. More below. > > > > > > -Add the documentation from the first patch and update the > > > documentation per patch based on the content. > > > > +1 > > > > More comments: > > > > - Please don't send so many patches, it looks like spam. > > - Please let's start small with the very minimal code to run a dummy test. > > - Split by file does not make sense > > This is very useful. We started with splitting up DTS into logical chunks, but it's > not quite what you have in mind. We'll move things around and send just the > first part (containing just a dummy test (is a smoke test ok?) with the > necessary code). Now that we have a better understanding of the first step > we'll produce a more complete patch set, with proper commit messages and > files in each commit. I think bringing in this big chunk of code is a challenge. We need to discuss more on how to get this into DPDK. From that perspective: 1) the current set of patches are just the framework and documentation, no test suites yet. I am not sure if there are further parts that can be stripped out. 2) agree, the split by files is too much. We have tried to group the files into some logical components. So, would it be fine if we remove the commit by file, but still keep the logical component split? i.e. we will squash the files in a series into a single commit and then combine the logical splits as commits in a single series. 3) In my private discussions with David Marchand, he expressed interest in getting the git log history. The current review process will not help in this regard. Is this a must? If yes, are there any known methods to do this? > > > > > The process is going to be very long. > > The techboard said in the past that we must have a very careful review > > of an import piece by piece. So please be patient. Agree, we need to give time for review. I would prefer to set some timeline so that we are working accordingly. Currently, we are working with the goal of merging this in 22.07 (assuming the reviews are coming through quick enough). But, we can align on a different timeline. > > Thank you > > > >
> > 3) In my private discussions with David Marchand, he expressed interest > in getting the git log history. The current review process will not help in > this regard. Is this a must? If yes, are there any known methods to do this? The git docs point to https://github.com/newren/git-filter-repo/, since apparently git filter-branch is full of footguns. From there, I would follow an example from Ruby, https://gist.github.com/x-yuri/9890ab1079cf4357d6f269d073fd9731, which was merging two reasonably-sized tooling repos. Instead of using git merge, we would be using git rebase. I did it locally to try it out, and it does have to replay the ENTIRE git history of dts. This means whoever does it may want to grab a server and do the rebase in /run. This would also mean that we would have to bypass the mailing list, since sending out more than 32000 emails doesn't seem reasonable. Possibly just a patch with the script to run to do the rebase instead of putting all the patches on the mailing list? Doing this would also probably mean all of our work would be in the DTS repo until we are ready to merge. That might make it harder to review changes to DTS in small parts.
<snip> > > On Wed, Apr 6, 2022 at 8:26 PM Juraj Linkeš <juraj.linkes@pantheon.tech> > wrote: > > > > These are the basic libraries that other libraries depend on. There's > > also the basic framework functionality related to test execution. > > > > Juraj Linkeš (15): > > dts: merge DTS dep/tclclient.tgz to DPDK > > dts: merge DTS dep/tgen.tgz to DPDK > > dts: merge DTS dts to DPDK > > dts: merge DTS framework/__init__.py to DPDK > > dts: merge DTS framework/asan_test.py to DPDK > > dts: merge DTS framework/checkCase.py to DPDK > > dts: merge DTS framework/dts.py to DPDK > > dts: merge DTS framework/exception.py to DPDK > > dts: merge DTS framework/logger.py to DPDK > > dts: merge DTS framework/packet.py to DPDK > > dts: merge DTS framework/project_dpdk.py to DPDK > > dts: merge DTS framework/serializer.py to DPDK > > dts: merge DTS framework/utils.py to DPDK > > dts: merge DTS main.py to DPDK > > dts: merge DTS version.py to DPDK > > merge->import > > > > > dts/dep/tclclient.tgz | Bin 0 -> 199327 bytes > > dts/dep/tgen.tgz | Bin 0 -> 134392 bytes > > Some top level comments: > - I think, we should not check in binary files. > - git commit comment should much more than "dts: merge DTS xxxx to DPDK" > where the commit log should have details on check in. > -Add the documentation from the first patch and update the documentation > per patch based on the content. The documentation is existing documentation. It has evolved based on how the code has evolved. I do not think it is possible to split that documentation now and bring it in on a per patch basis. The best way would be to maintain the history from git. Some documentation is also getting updated, but this is one time overall update.