[v8,00/16] drivers: introduce mlx5 crypto PMD

  v2:
- Add data-path part.

v3:
- Rebase.

v4:
- Rebase + Address the following Akhil comments:
- Set HW feature flag in the capability patch.
- Fix mp object release in session clear.
- Some spelling and word missing in doc.
- Squash data-unit adjustment to the session operations commit.
- Wording: device argument -> devarg.

v5: 
- Add mlx5 crypto tests into test library.
- Update documentation according to Akhil comments.
- Fix memory region management.
- Fix multi segment case in data-path code.
- Split documentation to the correct commits according to Akhil comments.
- Rebase to new version.
- Change license to Nvidia license.

v6:
- Rebase to new version.
- Address David's comment and update log accordingly. 
- Update testing app commits as suggested by Akhil.

v7:
- Rebase to new version.
- Add scatter gather and OOP test cases.

v8:
- Add a test case that was removed by error.



Shiri Kuzin (11):
  drivers: introduce mlx5 crypto PMD
  crypto/mlx5: add DEK object management
  crypto/mlx5: add session operations
  crypto/mlx5: add basic operations
  crypto/mlx5: add queue pairs operations
  crypto/mlx5: add dev stop and start operations
  crypto/mlx5: add memory region management
  crypto/mlx5: create login object using DevX
  test/crypto: add mlx5 crypto driver
  test/crypto: add data-unit and wrapped vectors
  test/crypto: add AES-XTS multi segment OOP tests

Suanming Mou (5):
  crypto/mlx5: add keytag devarg
  crypto/mlx5: add maximum segments devarg
  crypto/mlx5: add WQE set initialization
  crypto/mlx5: add enqueue and dequeue operations
  crypto/mlx5: add statistic get and reset operations

 MAINTAINERS                                |    4 +
 app/test/test_cryptodev.c                  |    7 +
 app/test/test_cryptodev.h                  |    4 +-
 app/test/test_cryptodev_aes_test_vectors.h | 1404 ++++++++++++++++++++
 app/test/test_cryptodev_blockcipher.c      |   10 +-
 app/test/test_cryptodev_blockcipher.h      |    2 +
 doc/guides/cryptodevs/features/mlx5.ini    |   37 +
 doc/guides/cryptodevs/index.rst            |    1 +
 doc/guides/cryptodevs/mlx5.rst             |  153 +++
 doc/guides/rel_notes/release_21_08.rst     |    5 +
 drivers/common/mlx5/mlx5_common.h          |    1 +
 drivers/common/mlx5/mlx5_common_pci.c      |   14 +
 drivers/common/mlx5/mlx5_common_pci.h      |   21 +-
 drivers/crypto/meson.build                 |    1 +
 drivers/crypto/mlx5/meson.build            |   27 +
 drivers/crypto/mlx5/mlx5_crypto.c          | 1177 ++++++++++++++++
 drivers/crypto/mlx5/mlx5_crypto.h          |   91 ++
 drivers/crypto/mlx5/mlx5_crypto_dek.c      |  161 +++
 drivers/crypto/mlx5/mlx5_crypto_utils.h    |   19 +
 drivers/crypto/mlx5/version.map            |    3 +
 20 files changed, 3130 insertions(+), 12 deletions(-)
 create mode 100644 doc/guides/cryptodevs/features/mlx5.ini
 create mode 100644 doc/guides/cryptodevs/mlx5.rst
 create mode 100644 drivers/crypto/mlx5/meson.build
 create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c
 create mode 100644 drivers/crypto/mlx5/mlx5_crypto.h
 create mode 100644 drivers/crypto/mlx5/mlx5_crypto_dek.c
 create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h
 create mode 100644 drivers/crypto/mlx5/version.map
  

> v2:
> - Add data-path part.
> 
> v3:
> - Rebase.
> 
> v4:
> - Rebase + Address the following Akhil comments:
> - Set HW feature flag in the capability patch.
> - Fix mp object release in session clear.
> - Some spelling and word missing in doc.
> - Squash data-unit adjustment to the session operations commit.
> - Wording: device argument -> devarg.
> 
> v5:
> - Add mlx5 crypto tests into test library.
> - Update documentation according to Akhil comments.
> - Fix memory region management.
> - Fix multi segment case in data-path code.
> - Split documentation to the correct commits according to Akhil comments.
> - Rebase to new version.
> - Change license to Nvidia license.
> 
> v6:
> - Rebase to new version.
> - Address David's comment and update log accordingly.
> - Update testing app commits as suggested by Akhil.
> 
> v7:
> - Rebase to new version.
> - Add scatter gather and OOP test cases.
> 
> v8:
> - Add a test case that was removed by error.
> 
> 
> 
> Shiri Kuzin (11):
>   drivers: introduce mlx5 crypto PMD
>   crypto/mlx5: add DEK object management

This is really bad, I was planning to apply this series today,
It had small issues, so was planning to fix them while applying.
But when I applied the 2/16, it does not compile.
Please fix this, we are into v8 and compilation is broken.

I am adding few more comments, please fix them too in your next version
As soon as possible. I was planning to close RC2 during weekend, but it looks
It will fall over next week.

  147 |      mlx5_crypto_dek_match_cb,
      |      ^~~~~~~~~~~~~~~~~~~~~~~~
      |      |
      |      int (*)(void *, struct mlx5_list_entry *, void *)
In file included from ../drivers/crypto/mlx5/mlx5_crypto.h:13,
                 from ../drivers/crypto/mlx5/mlx5_crypto_dek.c:14:
../drivers/common/mlx5/mlx5_common_utils.h:138:20: note: expected 'mlx5_hlist_remove_cb' {aka 'void (*)(struct mlx5_hlist *, struct mlx5_hlist_entry *)'} but argument is of type 'int (*)(void *, struct mlx5_list_entry *, void *)'
  138 | struct mlx5_hlist *mlx5_hlist_create(const char *name, uint32_t size,
      |                    ^~~~~~~~~~~~~~~~~
../drivers/crypto/mlx5/mlx5_crypto_dek.c:144:20: error: too many arguments to function 'mlx5_hlist_create'
  144 |  priv->dek_hlist = mlx5_hlist_create("dek_hlist",
      |                    ^~~~~~~~~~~~~~~~~
In file included from ../drivers/crypto/mlx5/mlx5_crypto.h:13,
                 from ../drivers/crypto/mlx5/mlx5_crypto_dek.c:14:
../drivers/common/mlx5/mlx5_common_utils.h:138:20: note: declared here
  138 | struct mlx5_hlist *mlx5_hlist_create(const char *name, uint32_t size,
      |                    ^~~~~~~~~~~~~~~~~
../drivers/crypto/mlx5/mlx5_crypto_dek.c: In function 'mlx5_crypto_dek_destroy':
../drivers/crypto/mlx5/mlx5_crypto_dek.c:26:1: error: control reaches end of non-void function [-Werror=return-type]
   26 | }
      | ^
../drivers/crypto/mlx5/mlx5_crypto_dek.c: In function 'mlx5_crypto_dek_prepare':
../drivers/crypto/mlx5/mlx5_crypto_dek.c:45:1: error: control reaches end of non-void function [-Werror=return-type]
   45 | }
      | ^
../drivers/crypto/mlx5/mlx5_crypto_dek.c: In function 'mlx5_crypto_dek_create_cb':
../drivers/crypto/mlx5/mlx5_crypto_dek.c:127:1: error: control reaches end of non-void function [-Werror=return-type]
  127 | }
      | ^
cc1: all warnings being treated as errors
  

> > v2:
> > - Add data-path part.
> >
> > v3:
> > - Rebase.
> >
> > v4:
> > - Rebase + Address the following Akhil comments:
> > - Set HW feature flag in the capability patch.
> > - Fix mp object release in session clear.
> > - Some spelling and word missing in doc.
> > - Squash data-unit adjustment to the session operations commit.
> > - Wording: device argument -> devarg.
> >
> > v5:
> > - Add mlx5 crypto tests into test library.
> > - Update documentation according to Akhil comments.
> > - Fix memory region management.
> > - Fix multi segment case in data-path code.
> > - Split documentation to the correct commits according to Akhil comments.
> > - Rebase to new version.
> > - Change license to Nvidia license.
> >
> > v6:
> > - Rebase to new version.
> > - Address David's comment and update log accordingly.
> > - Update testing app commits as suggested by Akhil.
> >
> > v7:
> > - Rebase to new version.
> > - Add scatter gather and OOP test cases.
> >
> > v8:
> > - Add a test case that was removed by error.
> >
> >
> >
> > Shiri Kuzin (11):
> >   drivers: introduce mlx5 crypto PMD
> >   crypto/mlx5: add DEK object management
> 
> This is really bad, I was planning to apply this series today,
> It had small issues, so was planning to fix them while applying.
> But when I applied the 2/16, it does not compile.
> Please fix this, we are into v8 and compilation is broken.
> 
> I am adding few more comments, please fix them too in your next version
> As soon as possible. I was planning to close RC2 during weekend, but it looks
> It will fall over next week.
> 
Please review this patchset thoroughly within Nvidia before sending next version.
Take reference to other crypto PMDs for patch splitting and sequencing.
  

> 
> > > v2:
> > > - Add data-path part.
> > >
> > > v3:
> > > - Rebase.
> > >
> > > v4:
> > > - Rebase + Address the following Akhil comments:
> > > - Set HW feature flag in the capability patch.
> > > - Fix mp object release in session clear.
> > > - Some spelling and word missing in doc.
> > > - Squash data-unit adjustment to the session operations commit.
> > > - Wording: device argument -> devarg.
> > >
> > > v5:
> > > - Add mlx5 crypto tests into test library.
> > > - Update documentation according to Akhil comments.
> > > - Fix memory region management.
> > > - Fix multi segment case in data-path code.
> > > - Split documentation to the correct commits according to Akhil
> comments.
> > > - Rebase to new version.
> > > - Change license to Nvidia license.
> > >
> > > v6:
> > > - Rebase to new version.
> > > - Address David's comment and update log accordingly.
> > > - Update testing app commits as suggested by Akhil.
> > >
> > > v7:
> > > - Rebase to new version.
> > > - Add scatter gather and OOP test cases.
> > >
> > > v8:
> > > - Add a test case that was removed by error.
> > >
> > >
> > >
> > > Shiri Kuzin (11):
> > >   drivers: introduce mlx5 crypto PMD
> > >   crypto/mlx5: add DEK object management
> >
> > This is really bad, I was planning to apply this series today,
> > It had small issues, so was planning to fix them while applying.
> > But when I applied the 2/16, it does not compile.
> > Please fix this, we are into v8 and compilation is broken.
> >
> > I am adding few more comments, please fix them too in your next version
> > As soon as possible. I was planning to close RC2 during weekend, but it
> looks
> > It will fall over next week.
> >
> Please review this patchset thoroughly within Nvidia before sending next
> version.
> Take reference to other crypto PMDs for patch splitting and sequencing.

Are we expecting next version today?
This is the only major patchset left for RC2.
Any update?
  

Hi Akhil,

> -----Original Message-----
> From: Akhil Goyal <gakhil@marvell.com>
> Sent: Tuesday, July 20, 2021 4:14 PM
> To: Shiri Kuzin <shirik@nvidia.com>; NBU-Contact-Thomas Monjalon
> <thomas@monjalon.net>; Matan Azrad <matan@nvidia.com>; Suanming Mou
> <suanmingm@nvidia.com>
> Cc: david.marchand@redhat.com; dev@dpdk.org
> Subject: RE: [EXT] [PATCH v8 00/16] drivers: introduce mlx5 crypto PMD
> 
> 
> >
> > > > v2:
> > > > - Add data-path part.
> > > >
> > > > v3:
> > > > - Rebase.
> > > >
> > > > v4:
> > > > - Rebase + Address the following Akhil comments:
> > > > - Set HW feature flag in the capability patch.
> > > > - Fix mp object release in session clear.
> > > > - Some spelling and word missing in doc.
> > > > - Squash data-unit adjustment to the session operations commit.
> > > > - Wording: device argument -> devarg.
> > > >
> > > > v5:
> > > > - Add mlx5 crypto tests into test library.
> > > > - Update documentation according to Akhil comments.
> > > > - Fix memory region management.
> > > > - Fix multi segment case in data-path code.
> > > > - Split documentation to the correct commits according to Akhil
> > comments.
> > > > - Rebase to new version.
> > > > - Change license to Nvidia license.
> > > >
> > > > v6:
> > > > - Rebase to new version.
> > > > - Address David's comment and update log accordingly.
> > > > - Update testing app commits as suggested by Akhil.
> > > >
> > > > v7:
> > > > - Rebase to new version.
> > > > - Add scatter gather and OOP test cases.
> > > >
> > > > v8:
> > > > - Add a test case that was removed by error.
> > > >
> > > >
> > > >
> > > > Shiri Kuzin (11):
> > > >   drivers: introduce mlx5 crypto PMD
> > > >   crypto/mlx5: add DEK object management
> > >
> > > This is really bad, I was planning to apply this series today, It
> > > had small issues, so was planning to fix them while applying.
> > > But when I applied the 2/16, it does not compile.
> > > Please fix this, we are into v8 and compilation is broken.
> > >
> > > I am adding few more comments, please fix them too in your next
> > > version As soon as possible. I was planning to close RC2 during
> > > weekend, but it
> > looks
> > > It will fall over next week.
> > >
> > Please review this patchset thoroughly within Nvidia before sending
> > next version.
> > Take reference to other crypto PMDs for patch splitting and sequencing.
> 
> Are we expecting next version today?
> This is the only major patchset left for RC2.
> Any update?

Sorry for the late reply, Shiri is not available these days,  so I'm trying to follow up the status and update the new version later today.
Regarding the compilation failure, it's due to the crypto patch series depends on some mlx5 common code. 
The mlx5 common code has some changes and I assume Shiri rebased the version with the dpdk-next-mlx and meanwhile the dpdk-next-crypto branch still did not get the latest mlx5 common code.

I just checked the dpdk-next-crypto branch, seems the mlx5 part is still not updated yet. So will you also sync up the crypto branch today? If no, then the failure will come again.
I'm not so familiar with the branch update procedure, but we will need the branch to be  updated to the version below:
http://git.dpdk.org/next/dpdk-next-net-mlx/commit/?id=7e1cf892711b9ba237563b6c876382caaec434f7

Thanks,
SuanmingMou
  

> > > > > v2:
> > > > > - Add data-path part.
> > > > >
> > > > > v3:
> > > > > - Rebase.
> > > > >
> > > > > v4:
> > > > > - Rebase + Address the following Akhil comments:
> > > > > - Set HW feature flag in the capability patch.
> > > > > - Fix mp object release in session clear.
> > > > > - Some spelling and word missing in doc.
> > > > > - Squash data-unit adjustment to the session operations commit.
> > > > > - Wording: device argument -> devarg.
> > > > >
> > > > > v5:
> > > > > - Add mlx5 crypto tests into test library.
> > > > > - Update documentation according to Akhil comments.
> > > > > - Fix memory region management.
> > > > > - Fix multi segment case in data-path code.
> > > > > - Split documentation to the correct commits according to Akhil
> > > comments.
> > > > > - Rebase to new version.
> > > > > - Change license to Nvidia license.
> > > > >
> > > > > v6:
> > > > > - Rebase to new version.
> > > > > - Address David's comment and update log accordingly.
> > > > > - Update testing app commits as suggested by Akhil.
> > > > >
> > > > > v7:
> > > > > - Rebase to new version.
> > > > > - Add scatter gather and OOP test cases.
> > > > >
> > > > > v8:
> > > > > - Add a test case that was removed by error.
> > > > >
> > > > >
> > > > >
> > > > > Shiri Kuzin (11):
> > > > >   drivers: introduce mlx5 crypto PMD
> > > > >   crypto/mlx5: add DEK object management
> > > >
> > > > This is really bad, I was planning to apply this series today, It
> > > > had small issues, so was planning to fix them while applying.
> > > > But when I applied the 2/16, it does not compile.
> > > > Please fix this, we are into v8 and compilation is broken.
> > > >
> > > > I am adding few more comments, please fix them too in your next
> > > > version As soon as possible. I was planning to close RC2 during
> > > > weekend, but it
> > > looks
> > > > It will fall over next week.
> > > >
> > > Please review this patchset thoroughly within Nvidia before sending
> > > next version.
> > > Take reference to other crypto PMDs for patch splitting and sequencing.
> >
> > Are we expecting next version today?
> > This is the only major patchset left for RC2.
> > Any update?
> 
> Sorry for the late reply, Shiri is not available these days,  so I'm trying to
> follow up the status and update the new version later today.
> Regarding the compilation failure, it's due to the crypto patch series depends
> on some mlx5 common code.
> The mlx5 common code has some changes and I assume Shiri rebased the
> version with the dpdk-next-mlx and meanwhile the dpdk-next-crypto branch
> still did not get the latest mlx5 common code.
> 
> I just checked the dpdk-next-crypto branch, seems the mlx5 part is still not
> updated yet. So will you also sync up the crypto branch today? If no, then the
> failure will come again.
> I'm not so familiar with the branch update procedure, but we will need the
> branch to be  updated to the version below:
> http://git.dpdk.org/next/dpdk-next-net-mlx/commit/?id=7e1cf892711b9ba237563b6c876382caaec434f7

The tree is in sync with main branch now.
  

> > Sorry for the late reply, Shiri is not available these days,  so I'm trying to
> > follow up the status and update the new version later today.
> > Regarding the compilation failure, it's due to the crypto patch series
> depends
> > on some mlx5 common code.
> > The mlx5 common code has some changes and I assume Shiri rebased the
> > version with the dpdk-next-mlx and meanwhile the dpdk-next-crypto
> branch
> > still did not get the latest mlx5 common code.
> >
> > I just checked the dpdk-next-crypto branch, seems the mlx5 part is still not
> > updated yet. So will you also sync up the crypto branch today? If no, then
> the
> > failure will come again.
> > I'm not so familiar with the branch update procedure, but we will need the
> > branch to be  updated to the version below:
> > http://git.dpdk.org/next/dpdk-next-net-
> mlx/commit/?id=7e1cf892711b9ba237563b6c876382caaec434f7
> 
> The tree is in sync with main branch now.

BTW, it was not mentioned in the cover note that it has dependency on other patchset.
  

> -----Original Message-----
> From: Akhil Goyal <gakhil@marvell.com>
> Sent: Tuesday, July 20, 2021 4:42 PM
> To: Suanming Mou <suanmingm@nvidia.com>; Shiri Kuzin <shirik@nvidia.com>;
> NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Matan Azrad
> <matan@nvidia.com>
> Cc: david.marchand@redhat.com; dev@dpdk.org
> Subject: RE: [EXT] [PATCH v8 00/16] drivers: introduce mlx5 crypto PMD
> 
> > > Sorry for the late reply, Shiri is not available these days,  so I'm
> > > trying to follow up the status and update the new version later today.
> > > Regarding the compilation failure, it's due to the crypto patch
> > > series
> > depends
> > > on some mlx5 common code.
> > > The mlx5 common code has some changes and I assume Shiri rebased the
> > > version with the dpdk-next-mlx and meanwhile the dpdk-next-crypto
> > branch
> > > still did not get the latest mlx5 common code.
> > >
> > > I just checked the dpdk-next-crypto branch, seems the mlx5 part is
> > > still not updated yet. So will you also sync up the crypto branch
> > > today? If no, then
> > the
> > > failure will come again.
> > > I'm not so familiar with the branch update procedure, but we will
> > > need the branch to be  updated to the version below:
> > > http://git.dpdk.org/next/dpdk-next-net-
> > mlx/commit/?id=7e1cf892711b9ba237563b6c876382caaec434f7
> >
> > The tree is in sync with main branch now.
> 
> BTW, it was not mentioned in the cover note that it has dependency on other
> patchset.

Thank you for the branch syncing. 
I assume the main reason is that the required patchset was already merged when the crypto patch v8 was sent, we are not so familiar with the branch syncing.
Will keep an eye on that next time.